AI Driven PM: S2E5 - Data Rules 2.0

Let me tell you something I learned from Six Sigma that changed the way I run projects:

If you can't measure it, you can't improve it.

I came up through the DMAIC era—Define, Measure, Analyze, Improve, Control. I love data. I live for metrics. Give me a spreadsheet and a story to tell, and I'm happy.

But here's the uncomfortable truth I've discovered after 30+ years and 150+ implementations:

Most of the metrics we track don't tell the story we actually need to tell.

We obsess over story points. Velocity. Task completions. Hours logged. Burn rate.

And then an executive asks, "Are we going to hit the launch date?"

And we stare at our dashboard.

And it doesn't answer the question.

That's not a data problem. That's a framing problem.

The Activity Metrics Trap

Here's what most PMs measure:

• Story points completed
• Task closes
• Hours logged
• Budget spent to date
• Number of commits to repo

You know what all of those have in common?

They measure busyness. Not achievement.

Activity doesn't equal value. And executives—even if they can't always articulate it—don't actually care about activity. They care about outcomes that drive business value.

I was working with a client recently who said their goal was a 2% net sales lift.

I asked, "If you land Walmart, do you win?"

They said, "No."

I said, "Okay—walk me through what this 2% net sales lift actually means."

And here's what was wild: They were using that metric to justify building a data warehouse, but they had no idea what data they were going to put into the warehouse to measure it.

The metric was real. The measurement plan was nonexistent.

That's the trap.

We pick metrics because they're easy to track, not because they answer the questions that actually matter.

We Are Lawyers. Our Sponsors Are Judges.

Here's a frame I use all the time that completely changes how PMs think about data:

Your job is to build a case. Your sponsor is the judge.

You gather evidence. You analyze the data. You present your recommendation. You let the judge decide.

And if you don't like the ruling? You don't argue in the courtroom.

You appeal.

You go back, review your data, figure out why it didn't tell the compelling story you needed, and you come back better prepared.

And here's the thing: If I have more data than you, I'm going to win the conversation.

Not because I'm louder or more senior or more confident.

Because data tells a story. And the PM who tells the better story wins.

The problem is we keep bringing the wrong data to court.

What You Should Actually Be Measuring

Here's what I suggest you measure instead of activity:

1. Value Delivered

Not "features shipped"—features in production being used.

I worked on the GrowthDay app build. We had all these features planned, but at the last minute, the founder said, "Wouldn't it be cool if we had a daily motivational segment—something that fires people up every morning?"

We almost cut it. Time pressure. Scope pressure.

We didn't cut it.

That tiny, last-minute feature became one of the stickiest in the whole app.

But here's the key: We only knew it because we measured how people were actually using the app. How many times. How long they stayed. Whether they came back.

If you're not measuring features being used, you don't know if you're delivering value or just shipping code.

2. Time to Impact

How fast do you go from idea to user value?

What's your average cycle time from "we need this" to "users are using this"?

That's a story executives actually care about.

3. Quality Signals

Defect rates. Technical debt. User satisfaction.

But here's the nuance: When you compress testing due to date pressure, defect growth becomes exponential—not linear.

I had this exact conversation in a live demo. The data showed:

• Sprint velocity dropped
• Defect rate rising
• Testing coverage shrinking

I told the team: "If testing continues to erode, expect 15 to 18 defects per sprint within four to six weeks. The rework alone will cost more time than the testing would have."

That's the conversation a PM should be having with a sponsor. Not "we're at 80% of story points." But: "We're trading short-term velocity for long-term quality debt. Here's what that actually costs us."

4. Team Health

Velocity stability (not just velocity), morale, attrition risk, sentiment analysis.

A velocity drop after adding a team member? That's normal. Expected, even.

Knowledge transfer consumes senior capacity. Code review loads increase. A 20% velocity drop after onboarding one person is common in the first two to four sprints.

But if it hasn't recovered in three sprints? That's structural. That's something else.

Know the difference.

5. Stakeholder Confidence

Sponsor engagement. Clarity of vision. Meeting attendance.

When your sponsor starts missing meetings, that's a leading indicator—not a footnote.

The Metric I'm Most Proud Of: Scope Stability Index

Here's one I love that most teams don't track:

Scope Stability Index = New story points added ÷ Total committed story points

If that number exceeds 15% mid-sprint, execution predictability collapses.

Let me make that concrete. You committed to 30 story points for the sprint. During the sprint, 10 new points get added. That's 10 ÷ 30 = 33%.

Your sprint just broke.

Not because the team is failing—but because the input changed faster than the output could absorb it.

This is the conversation you bring to a sponsor: "Every time we add scope mid-sprint, we pay a compounding tax. Here's what that tax looks like in data."

That's a case. That's a lawyer walking into court prepared.

How AI Fits Into All of This

Here's the piece most people miss:

AI can automate all the activity tracking.

I have agents that pull data from JIRA, Microsoft Planner, ServiceNow, spreadsheets—normalize it, format it, and report it. Automatically.

That means I'm not spending 60% of my week staring backwards at what already happened.

I'm spending it on outcome measurement, impact analysis, and inventing new metrics that tell the parts of the story nobody else is telling.

AI can correlate your leading indicators (velocity, quality, team sentiment) with your lagging indicators (revenue, retention, delivery dates). It can isolate trends across data sources you'd never have time to manually connect.

But here's the catch, and I said this right at the top of the episode:

AI can't do for you what it can't do through you.

The metrics it surfaces are only as good as the questions you're asking. You have to know what story you're trying to tell before AI can help you tell it.

That's what these three prompts are designed to do.

---

Prompt 1: Metrics Dashboard Designer

Start here. This builds you a dashboard that actually answers the questions your executives are asking.

What it creates:

• 3-5 outcome metrics that prove value delivery
• 3-5 activity health metrics as leading indicators
• Data sources and measurement approach for each
• Green/yellow/red thresholds
• How to present differently to executives vs. team vs. sponsor

What I got when I ran it for the Social Wishing app:

ChatGPT asked:

• "What would cause leadership to declare this a failure at month six or nine?"
• "How does this app make money in the first 12-18 months?"
• "Do you have any analytics tooling selected?"

That first question? Bring it to your sponsor. Seriously. Ask them: "What would cause you to declare this a failure at month six?" You'll learn more in that 10-minute conversation than in three weeks of status reports.

The outcome metrics it generated:

• New user signups per week
• 90-day active user rate
• Wish progress rate
• 30-day retention

The leading indicators:

• Visitor-to-signup conversion rate
• Invite rate (% of users who invite at least one friend)
• Time to first wish (median time from signup to first wish created)
• Sprint predictability

The dashboard format for founders: Single slide, three rows.

• Row 1: Growth (signups, conversion)
• Row 2: Engagement (wish progress, invite rate)
• Row 3: Retention (30-day retention, trend arrow)

Simple. Powerful. Tells the dream story.

---

Prompt 2: Predictive Risk Indicator Finder

This one is for when you feel something is wrong but can't quite articulate it yet.

What it does:

• Identifies leading indicators that predict trouble
• Correlates team health metrics with outcome metrics
• Surfaces data you're NOT capturing (but should be)
• Sets intervention thresholds
• Coaches you on how to communicate risk without panicking the room

I gave it this project context:

• Sprint velocity dropped from 35 to 28 points
• Defects up from 7 to 12 per sprint
• Code review cycle time averaging 2.3 days
• Sponsor missed last 2-3 meetings
• Team sentiment dropped from 8.0 to 6.5
• 8 new feature requests added this month, 3 original features cut

What AI told me:

"Velocity drop after adding a person is classic onboarding drag. A 20% drop is common in the first two to four sprints. If it doesn't recover within three sprints, the issue is structural—not onboarding."

"Your defect increase combined with shrinking testing coverage is the highest risk signal in your data. When testing coverage drops due to date pressure, defect growth becomes exponential, not linear. Expect 15 to 18 defects per sprint within four to six weeks if nothing changes."

And then—the one I loved most—it surfaced metrics I wasn't tracking:

Code review comment density per PR.

"High comment density means complexity or standards drift. Low comment density with long cycle times means avoidance. These require completely different interventions."

I would never have thought to track that. That's AI as a thinking partner.

---

Prompt 3: Vanity vs. Value Metrics Audit

This is the one that will save you from the most painful meeting of your career.

You know the meeting. Twenty executives in the room. You walk through your status report. They eat you alive.

Because your report answered "Are we busy?" instead of "Are we going to succeed?"

I've been in that meeting. I never want you to experience it.

I gave AI this list of my current metrics:

• Tasks completed this week
• Story points burned
• Budget spent to date
• Team utilization
• Number of commits to repo
• Lines of code written
• Meetings held
• Risks identified

And I told it: "Executives keep asking if we'll hit the launch date and user targets. My metrics don't answer that question."

Claude's response was brutal. And perfect:

"Why are you asking me questions, Rick? You've already diagnosed the problem yourself. Your metrics answer 'Are we busy?'—not 'Are we going to succeed?'"

"You have six vanity metrics out of eight. The executives are asking the right question. Your dashboard is giving them the wrong answer. It's not a data problem. It's a framing problem. You're reporting inputs when they're asking about outcomes."

Couldn't have said it better myself.

Then it gave me the swap:

Vanity Metric	Why It's Vanity	Replace With
Story points burned	Doesn't predict completion without trend	Forecasted completion date based on velocity trend
Team utilization	Measures busyness, not throughput	Scope stability index
Commits to repo	More commits can signal churn, not progress	Defect escape rate
Lines of code	More code often = more defects	Time to value / time to first wish

That table is a career-saver.

---

Your Non-Negotiable Experiment This Week

Two challenges:

1. Build your outcome metrics dashboard using Prompt 1. Take your current project and identify 3-5 metrics that actually answer your executive's burning questions.

2. Replace at least one vanity metric in your next status report with a value metric.

Just one swap.

Here's what I want you to notice:

• How do stakeholders react when your report answers their actual questions?
• Does better data help you spot risks earlier?
• Do you feel more confident walking into that executive meeting?

Because here's the truth: The PM who tells the better story with better data wins.

Not because they're louder. Because they came prepared.

---

Next time: Net Operating Value—the metric I use for portfolio decisions. How to stack-rank your portfolio, make trade-off decisions, and help executives choose between good ideas using data that actually reflects business value.

Want these prompts ready to copy/paste? Head to PMThatWorks.com for the full library.

Now go build that dashboard.

— Rick A. Morris

---

The Prompts (Copy/Paste Ready)

Prompt 1 - Metrics Dashboard Designer

You are a data-driven PM coach and metric strategist.

First, ask me 4–5 questions about the project goals, stakeholders, team, and what success means in business terms.

Then help me design a metrics dashboard by answering:

1. What are the 3–5 outcome metrics that prove the project is delivering value?
2. What are 3–5 activity health metrics that are leading indicators of those outcomes?
3. For each metric, what is the data source and how do we measure it?
4. What thresholds or targets indicate green, yellow, red status for each metric?
5. How should I present these metrics to executives vs. the team vs. the sponsor?

Project context: [Enter Context]

---

Prompt 2 - Predictive Risk Indicator Finder

You are a predictive analytics expert for project management.

Ask me 3–4 questions about current project metrics, team dynamics, and any early warning signs I'm seeing.

Then analyze potential risk patterns by answering:

1. Based on the metrics I'm tracking, what are the 3–5 leading indicators that typically predict project trouble?
2. What correlation exists between team health metrics (velocity, morale) and outcome metrics (quality, delivery)?
3. What data am I not currently capturing that would give me an earlier warning sign of risk?
4. What specific metric threshold should trigger a project health intervention?
5. How do I communicate risk using data without sounding alarmist?

Current metrics: [Enter Metrics and Current Project Context]

---

Prompt 3 - Vanity vs. Value Metrics Audit

You are a metric strategist helping PMs distinguish signal from noise.

Ask me 2–3 questions about the metrics I'm currently reporting and what decisions those metrics inform.

Then provide an analysis answering:

1. Which of my current metrics are vanity metrics? (They look good but don't drive decisions.)
2. Which metrics are value metrics? (They directly inform action or prove impact.)
3. For each vanity metric, what is the underlying value metric I should track instead?
4. What questions should I ask myself to test if a metric is worth tracking?
5. How do I transition stakeholders away from vanity metrics they're used to seeing?

Current metrics: [List Your Current Metrics]

 

AI Driven PM: S2E4 - Do We Have to be the Domain Expert?

I get this question all the time when I'm working with a new client or interviewing for a role.

"So Rick, are you an expert in the insurance industry?"

"No."

"Well... shouldn't you be? I mean, shouldn't a project manager understand the domain they're managing projects in?"

"Not really."

They look confused. So I follow up:

"How many people work here?"

"About 5,000."

"Great. So you've got 5,000 experts in insurance. What you need is someone like me who can get the best out of those 5,000 people."

That's the difference between domain expertise and project management expertise.

And most organizations don't understand it.

They prioritize hiring PMs who "speak the language of the business" over PMs who know how to facilitate, translate, and orchestrate expertise.

And that hiring bias? It backfires more often than you think.

The Conventional Wisdom (And Why It's Wrong)

Here's what most organizations believe:

• You need a PM who understands healthcare, finance, manufacturing, [insert industry here]
• They have to "speak the language" of the business
• Domain expertise signals credibility and competence
• Deep knowledge = better decisions

The reality? Domain expertise can be a hindrance just as often as it's a help.

Here's why.

When Domain Expertise Backfires

1. Domain Expert PMs Micromanage

When you know how to do the work, it's really hard not to tell people how to do the work.

A PM with deep domain expertise will hear an engineer say, "That'll take 10 hours," and think, "I could do that in two hours."

And then they start challenging estimates. Second-guessing approaches. Offering "suggestions" that aren't really suggestions.

That's not project management. That's micromanagement.

2. They Focus on WHAT, Not HOW

Domain expert PMs get obsessed with what's being built instead of how the team is building it.

They care more about the technical solution than the team dynamics, the stakeholder alignment, or the energy in the room.

And that leads to projects that might be technically perfect but operationally a disaster.

3. Personal Bias Takes Over

When you have deep domain experience, you carry bias—whether you're aware of it or not.

You think, "I've been there before. I've seen that approach. It doesn't work."

But here's the thing: Just because it didn't work when YOU tried it doesn't mean it won't work now.

Context matters. Teams matter. Timing matters.

And sometimes the team needs to try something, discover it doesn't work, and pivot. That's how ownership and learning happen.

When a domain expert PM shuts that down with "I already know that won't work," they kill ownership and innovation.

4. They Become Decision Bottlenecks

Domain expert PMs feel like they need to be involved in every decision because they "understand the implications."

So they become the bottleneck.

Every technical choice, every scope question, every trade-off discussion has to run through them.

That's not leadership. That's dependency.

What Great PMs Actually Do

Here's the truth most organizations miss:

A great PM knows how to ask the right questions—not provide the right answers.

Let me say that again for the people in the back:

You don't make decisions on scope, budget, timeline, or what's in or out. You make recommendations. You influence. But you don't own the decision.

Your job is to:

• Translate between domains (tech to business, business to customer, customer to tech)
• Facilitate expertise (create the conditions for experts to do their best work)
• Ask the "dumb" question (the one everyone assumed was already answered)
• Frame trade-offs (so the right people can make informed decisions)
• Orchestrate, not dictate (you're the conductor, not the soloist)

And here's the magic: Facilitation expertise + deep learning desire > domain expertise.

Why?

Because you're unafraid to ask the next question. You're not stuck in "how it's always been done." You bring fresh eyes, challenge assumptions, and force clarity where experts have gotten comfortable with ambiguity.

When Domain Expertise DOES Matter

I'm not saying domain expertise is useless. There are contexts where it absolutely matters:

1. Highly Regulated Industries

Healthcare, finance, government—anywhere compliance is complex and non-negotiable.

In these environments, knowing which questions to ask requires baseline domain knowledge. You need to know what regulations exist so you know who to pull into the conversation.

But even then, you don't need to be the compliance expert. You just need to know when to engage one.

2. Deeply Technical Domains

If you're building your own AI/ML systems, embedded systems, or highly specialized technology, some technical fluency helps with translation.

But notice I said fluency, not mastery.

You don't need to code the solution. You need to understand enough to ask, "What are the trade-offs?" and "What happens if we're wrong?"

3. When You're the Only Person in the Room

If you're a solo PM in a startup with no dedicated domain experts, then yeah—you might need to wear both hats for a while.

But even then, your job is to build the team that replaces your domain gaps as fast as possible.

How AI Helps You Bridge Domain Gaps in Days, Not Months

This is where it gets fun.

One of the most powerful uses of AI for project managers isn't writing status reports or generating meeting notes.

It's becoming a domain learning accelerator.

You can use AI to:

• Get up to speed on unfamiliar domains in days instead of months
• Build stakeholder expertise maps so you know who to ask what
• Generate facilitation scripts so you can lead technical debates without pretending to be the expert

Let me show you.

---

Prompt 1: Domain Knowledge Accelerator (Your Non-Negotiable)

This is your experiment for this week. Use AI to get up to speed on an unfamiliar domain—fast.

What it does:

• Identifies 5-7 core concepts you need to understand
• Explains each concept in plain language with analogies
• Maps key stakeholder types and what they care about
• Surfaces common PM pitfalls in that domain
• Generates questions to ask experts

What I got when I ran it for the Social Wishing app:

ChatGPT gave me concepts like:

• OAuth and API authorization flows
• Graph API rate limits
• Data privacy classification
• Viral growth and infrastructure scaling

And then—here's what I loved—it gave me analogies.

For "viral growth and infrastructure scaling," it said:

Plain language: If growth spikes, your system must handle sudden load increases.

Analogy: It's like a small coffee shop that suddenly gets national press—but you only have one espresso machine. Service will collapse.

That's gold.

Now I can explain infrastructure risk to a business stakeholder without using the word "horizontal scaling."

I can say, "We just got national press, and we've got a line around the block—but we only have one espresso machine. We need to decide: Do we buy more machines now, or risk turning customers away?"

That's translation. That's facilitation. That's what great PMs do.

---

Prompt 2: Stakeholder Expertise Mapper

This one helps you figure out who knows what and who cares about what on your project.

What it creates:

• 8-12 key stakeholders by role (not name)
• What domain expertise each brings
• What each stakeholder's "win condition" is
• Who to rely on for domain expertise vs. business context vs. technical decisions
• Questions to ask each stakeholder type

What I got:

ChatGPT mapped out:

• Executive sponsor (cares about market differentiation and user growth)
• Product owner (cares about MVP clarity and scope control)
• Back-end engineer (cares about API stability and Facebook integration)
• Marketing director (cares about launch readiness and campaign metrics)
• QA engineer (cares about testing strategy for third-party integrations)

Then it gave me questions tailored to each stakeholder.

For the marketing director: "What needs to be true in terms of experience or metrics for you to feel confident running a full launch campaign?"

For the QA engineer: "If you were going to design the testing strategy for the Facebook API integration from scratch, what would you prioritize?"

These aren't generic questions. They're role-specific, expertise-tapping questions that show you're learning—and give you credibility without pretending to know.

---

Prompt 3: Facilitation Over Expertise Script

This is the one I use when the team is stuck in a heated debate and I don't have the technical chops to declare a winner.

The scenario I gave it:

The Social Wishing engineering team is debating architecture.

Option A: Microservices from day one (more complex, scales better)
Option B: Monolith first, split later (faster to MVP, potential refactor pain)

I don't have strong back-end architecture expertise. Two senior engineers are dug in on opposite sides. The debate is getting heated, and we're burning time.

How do I lead this without pretending I know what's technically right?

What AI gave me:

Questions to draw out expertise:

• "What's each of us assuming about how fast this app will scale—and are those assumptions written down anywhere?"
• "What would have to be true about our growth trajectory for Option A to be clearly the right call? Or Option B?"
• "Has anyone on the team built something similar before, and what happened?"
• "Is there anything about our team's current skills or bandwidth that should factor into this choice that we haven't mentioned yet?"

Framework to organize the discussion:

Claude suggested:

1. Structured input from both sides (5 minutes each, no interruptions)
2. Engineering lead makes recommendation
3. I confirm alignment with business constraints
4. If no clear owner exists, escalate ownership before debating substance

That last one is killer: Find out who's going to make the call before you go into a full debate.

Authority without expertise:

ChatGPT gave me this framing:

"I'm not here to declare the technically pure answer. I'm here to ensure we understand the trade-offs and align the architecture to our business goals."

That's leadership.

You're not pretending to know the answer. You're facilitating the process that gets to the right answer.

---

Your Non-Negotiable Experiment This Week

Use the Domain Knowledge Accelerator (Prompt 1) on an unfamiliar area of your current project.

Then ask at least one question from the expert question list AI generates for you.

Here's what I want you to notice:

1. Did asking questions instead of pretending to know earn you more credibility?
   (It almost always does.)
2. How much faster can you learn with AI as a tutor?
   (Days instead of months.)
3. Did the "dumb" question you asked surface something nobody else was saying out loud?
   (That's where breakthroughs happen.)

Because here's the truth: Asking questions doesn't make you look weak. It makes you look curious, coachable, and confident enough to admit what you don't know.

And that earns trust faster than pretending to be the expert ever will.

---

The Takeaway

Domain expertise is overrated for project managers.

Facilitation expertise is underrated.

Great PMs don't have all the answers. They ask the right questions and create the conditions for experts to thrive.

And with AI as your learning partner, you can bridge domain knowledge gaps in days—not months—so you can lead with confidence even when you're not the expert in the room.

So stop worrying about whether you "know the industry."

Start worrying about whether you know how to get the best out of the people who do.

---

Next time: Data-Driven Metrics 2.0—What metrics actually matter in the AI era, and how do we use AI to surface what's really going on in our projects?

If you would like to see the podcast live, check out this link: https://youtu.be/zqrspMN0gCM

Now go ask a "dumb" question. Your team is waiting.

— Rick A. Morris

---

The Prompts (Copy/Paste Ready)

Prompt 1 - Domain Knowledge Accelerator

You are a strategic learning coach helping a project manager quickly understand a new domain.

First, ask me 2–3 questions about the project, the domain, and what I specifically need to understand to lead effectively.

Then provide a learning plan answering:

1. What are the 5–7 core concepts or frameworks I must understand in this domain?
2. For each concept, explain it in plain language with an analogy to something more familiar.
3. What are the key stakeholder types in this domain and what does each care most about?
4. What are the 3–5 most common pitfalls or mistakes PMs make when they don't understand this domain?
5. What questions should I ask domain experts to demonstrate I'm learning and to uncover critical constraints?

Domain and project context: [Enter Context]

---

Prompt 2 - Stakeholder Expertise Mapper

You are a project stakeholder analyst.

Ask me 3–4 questions about the project, its goals, and who's involved or affected.

Then create a stakeholder expertise map answering:

1. Who are the 8–12 key stakeholders (by role, not name)?
2. For each stakeholder, what domain expertise or knowledge do they bring?
3. What does each stakeholder care most about (their "win condition")?
4. Which stakeholders should I rely on for domain expertise vs. business context vs. technical decisions?
5. What questions should I ask each stakeholder type to tap their expertise effectively?

Project context: [Add Context]

---

Prompt 3 - Facilitation over Expertise Script

You are a coaching expert helping PMs lead through facilitation rather than expertise.

Ask me 2–3 questions about a specific domain decision or technical choice the team is debating.

Then help me facilitate the decision by providing:

1. What open-ended questions should I ask to draw out the team's expertise?
2. What framework or structure can I offer to organize the discussion (without dictating the answer)?
3. How do I acknowledge my knowledge gaps while still leading with authority?
4. What decision-making process should I facilitate (consensus, consultative, executive call)?
5. How do I summarize and communicate the decision in a way that shows I understand the "why" even if I didn't provide the "what"?

Situation: [Enter Situation]