AI Driven PM: S2E5 - Data Rules 2.0

Let me tell you something I learned from Six Sigma that changed the way I run projects:

If you can't measure it, you can't improve it.

I came up through the DMAIC era—Define, Measure, Analyze, Improve, Control. I love data. I live for metrics. Give me a spreadsheet and a story to tell, and I'm happy.

But here's the uncomfortable truth I've discovered after 30+ years and 150+ implementations:

Most of the metrics we track don't tell the story we actually need to tell.

We obsess over story points. Velocity. Task completions. Hours logged. Burn rate.

And then an executive asks, "Are we going to hit the launch date?"

And we stare at our dashboard.

And it doesn't answer the question.

That's not a data problem. That's a framing problem.

The Activity Metrics Trap

Here's what most PMs measure:

• Story points completed
• Task closes
• Hours logged
• Budget spent to date
• Number of commits to repo

You know what all of those have in common?

They measure busyness. Not achievement.

Activity doesn't equal value. And executives—even if they can't always articulate it—don't actually care about activity. They care about outcomes that drive business value.

I was working with a client recently who said their goal was a 2% net sales lift.

I asked, "If you land Walmart, do you win?"

They said, "No."

I said, "Okay—walk me through what this 2% net sales lift actually means."

And here's what was wild: They were using that metric to justify building a data warehouse, but they had no idea what data they were going to put into the warehouse to measure it.

The metric was real. The measurement plan was nonexistent.

That's the trap.

We pick metrics because they're easy to track, not because they answer the questions that actually matter.

We Are Lawyers. Our Sponsors Are Judges.

Here's a frame I use all the time that completely changes how PMs think about data:

Your job is to build a case. Your sponsor is the judge.

You gather evidence. You analyze the data. You present your recommendation. You let the judge decide.

And if you don't like the ruling? You don't argue in the courtroom.

You appeal.

You go back, review your data, figure out why it didn't tell the compelling story you needed, and you come back better prepared.

And here's the thing: If I have more data than you, I'm going to win the conversation.

Not because I'm louder or more senior or more confident.

Because data tells a story. And the PM who tells the better story wins.

The problem is we keep bringing the wrong data to court.

What You Should Actually Be Measuring

Here's what I suggest you measure instead of activity:

1. Value Delivered

Not "features shipped"—features in production being used.

I worked on the GrowthDay app build. We had all these features planned, but at the last minute, the founder said, "Wouldn't it be cool if we had a daily motivational segment—something that fires people up every morning?"

We almost cut it. Time pressure. Scope pressure.

We didn't cut it.

That tiny, last-minute feature became one of the stickiest in the whole app.

But here's the key: We only knew it because we measured how people were actually using the app. How many times. How long they stayed. Whether they came back.

If you're not measuring features being used, you don't know if you're delivering value or just shipping code.

2. Time to Impact

How fast do you go from idea to user value?

What's your average cycle time from "we need this" to "users are using this"?

That's a story executives actually care about.

3. Quality Signals

Defect rates. Technical debt. User satisfaction.

But here's the nuance: When you compress testing due to date pressure, defect growth becomes exponential—not linear.

I had this exact conversation in a live demo. The data showed:

• Sprint velocity dropped
• Defect rate rising
• Testing coverage shrinking

I told the team: "If testing continues to erode, expect 15 to 18 defects per sprint within four to six weeks. The rework alone will cost more time than the testing would have."

That's the conversation a PM should be having with a sponsor. Not "we're at 80% of story points." But: "We're trading short-term velocity for long-term quality debt. Here's what that actually costs us."

4. Team Health

Velocity stability (not just velocity), morale, attrition risk, sentiment analysis.

A velocity drop after adding a team member? That's normal. Expected, even.

Knowledge transfer consumes senior capacity. Code review loads increase. A 20% velocity drop after onboarding one person is common in the first two to four sprints.

But if it hasn't recovered in three sprints? That's structural. That's something else.

Know the difference.

5. Stakeholder Confidence

Sponsor engagement. Clarity of vision. Meeting attendance.

When your sponsor starts missing meetings, that's a leading indicator—not a footnote.

The Metric I'm Most Proud Of: Scope Stability Index

Here's one I love that most teams don't track:

Scope Stability Index = New story points added ÷ Total committed story points

If that number exceeds 15% mid-sprint, execution predictability collapses.

Let me make that concrete. You committed to 30 story points for the sprint. During the sprint, 10 new points get added. That's 10 ÷ 30 = 33%.

Your sprint just broke.

Not because the team is failing—but because the input changed faster than the output could absorb it.

This is the conversation you bring to a sponsor: "Every time we add scope mid-sprint, we pay a compounding tax. Here's what that tax looks like in data."

That's a case. That's a lawyer walking into court prepared.

How AI Fits Into All of This

Here's the piece most people miss:

AI can automate all the activity tracking.

I have agents that pull data from JIRA, Microsoft Planner, ServiceNow, spreadsheets—normalize it, format it, and report it. Automatically.

That means I'm not spending 60% of my week staring backwards at what already happened.

I'm spending it on outcome measurement, impact analysis, and inventing new metrics that tell the parts of the story nobody else is telling.

AI can correlate your leading indicators (velocity, quality, team sentiment) with your lagging indicators (revenue, retention, delivery dates). It can isolate trends across data sources you'd never have time to manually connect.

But here's the catch, and I said this right at the top of the episode:

AI can't do for you what it can't do through you.

The metrics it surfaces are only as good as the questions you're asking. You have to know what story you're trying to tell before AI can help you tell it.

That's what these three prompts are designed to do.

---

Prompt 1: Metrics Dashboard Designer

Start here. This builds you a dashboard that actually answers the questions your executives are asking.

What it creates:

• 3-5 outcome metrics that prove value delivery
• 3-5 activity health metrics as leading indicators
• Data sources and measurement approach for each
• Green/yellow/red thresholds
• How to present differently to executives vs. team vs. sponsor

What I got when I ran it for the Social Wishing app:

ChatGPT asked:

• "What would cause leadership to declare this a failure at month six or nine?"
• "How does this app make money in the first 12-18 months?"
• "Do you have any analytics tooling selected?"

That first question? Bring it to your sponsor. Seriously. Ask them: "What would cause you to declare this a failure at month six?" You'll learn more in that 10-minute conversation than in three weeks of status reports.

The outcome metrics it generated:

• New user signups per week
• 90-day active user rate
• Wish progress rate
• 30-day retention

The leading indicators:

• Visitor-to-signup conversion rate
• Invite rate (% of users who invite at least one friend)
• Time to first wish (median time from signup to first wish created)
• Sprint predictability

The dashboard format for founders: Single slide, three rows.

• Row 1: Growth (signups, conversion)
• Row 2: Engagement (wish progress, invite rate)
• Row 3: Retention (30-day retention, trend arrow)

Simple. Powerful. Tells the dream story.

---

Prompt 2: Predictive Risk Indicator Finder

This one is for when you feel something is wrong but can't quite articulate it yet.

What it does:

• Identifies leading indicators that predict trouble
• Correlates team health metrics with outcome metrics
• Surfaces data you're NOT capturing (but should be)
• Sets intervention thresholds
• Coaches you on how to communicate risk without panicking the room

I gave it this project context:

• Sprint velocity dropped from 35 to 28 points
• Defects up from 7 to 12 per sprint
• Code review cycle time averaging 2.3 days
• Sponsor missed last 2-3 meetings
• Team sentiment dropped from 8.0 to 6.5
• 8 new feature requests added this month, 3 original features cut

What AI told me:

"Velocity drop after adding a person is classic onboarding drag. A 20% drop is common in the first two to four sprints. If it doesn't recover within three sprints, the issue is structural—not onboarding."

"Your defect increase combined with shrinking testing coverage is the highest risk signal in your data. When testing coverage drops due to date pressure, defect growth becomes exponential, not linear. Expect 15 to 18 defects per sprint within four to six weeks if nothing changes."

And then—the one I loved most—it surfaced metrics I wasn't tracking:

Code review comment density per PR.

"High comment density means complexity or standards drift. Low comment density with long cycle times means avoidance. These require completely different interventions."

I would never have thought to track that. That's AI as a thinking partner.

---

Prompt 3: Vanity vs. Value Metrics Audit

This is the one that will save you from the most painful meeting of your career.

You know the meeting. Twenty executives in the room. You walk through your status report. They eat you alive.

Because your report answered "Are we busy?" instead of "Are we going to succeed?"

I've been in that meeting. I never want you to experience it.

I gave AI this list of my current metrics:

• Tasks completed this week
• Story points burned
• Budget spent to date
• Team utilization
• Number of commits to repo
• Lines of code written
• Meetings held
• Risks identified

And I told it: "Executives keep asking if we'll hit the launch date and user targets. My metrics don't answer that question."

Claude's response was brutal. And perfect:

"Why are you asking me questions, Rick? You've already diagnosed the problem yourself. Your metrics answer 'Are we busy?'—not 'Are we going to succeed?'"

"You have six vanity metrics out of eight. The executives are asking the right question. Your dashboard is giving them the wrong answer. It's not a data problem. It's a framing problem. You're reporting inputs when they're asking about outcomes."

Couldn't have said it better myself.

Then it gave me the swap:

Vanity Metric	Why It's Vanity	Replace With
Story points burned	Doesn't predict completion without trend	Forecasted completion date based on velocity trend
Team utilization	Measures busyness, not throughput	Scope stability index
Commits to repo	More commits can signal churn, not progress	Defect escape rate
Lines of code	More code often = more defects	Time to value / time to first wish

That table is a career-saver.

---

Your Non-Negotiable Experiment This Week

Two challenges:

1. Build your outcome metrics dashboard using Prompt 1. Take your current project and identify 3-5 metrics that actually answer your executive's burning questions.

2. Replace at least one vanity metric in your next status report with a value metric.

Just one swap.

Here's what I want you to notice:

• How do stakeholders react when your report answers their actual questions?
• Does better data help you spot risks earlier?
• Do you feel more confident walking into that executive meeting?

Because here's the truth: The PM who tells the better story with better data wins.

Not because they're louder. Because they came prepared.

---

Next time: Net Operating Value—the metric I use for portfolio decisions. How to stack-rank your portfolio, make trade-off decisions, and help executives choose between good ideas using data that actually reflects business value.

Want these prompts ready to copy/paste? Head to PMThatWorks.com for the full library.

Now go build that dashboard.

— Rick A. Morris

---

The Prompts (Copy/Paste Ready)

Prompt 1 - Metrics Dashboard Designer

You are a data-driven PM coach and metric strategist.

First, ask me 4–5 questions about the project goals, stakeholders, team, and what success means in business terms.

Then help me design a metrics dashboard by answering:

1. What are the 3–5 outcome metrics that prove the project is delivering value?
2. What are 3–5 activity health metrics that are leading indicators of those outcomes?
3. For each metric, what is the data source and how do we measure it?
4. What thresholds or targets indicate green, yellow, red status for each metric?
5. How should I present these metrics to executives vs. the team vs. the sponsor?

Project context: [Enter Context]

---

Prompt 2 - Predictive Risk Indicator Finder

You are a predictive analytics expert for project management.

Ask me 3–4 questions about current project metrics, team dynamics, and any early warning signs I'm seeing.

Then analyze potential risk patterns by answering:

1. Based on the metrics I'm tracking, what are the 3–5 leading indicators that typically predict project trouble?
2. What correlation exists between team health metrics (velocity, morale) and outcome metrics (quality, delivery)?
3. What data am I not currently capturing that would give me an earlier warning sign of risk?
4. What specific metric threshold should trigger a project health intervention?
5. How do I communicate risk using data without sounding alarmist?

Current metrics: [Enter Metrics and Current Project Context]

---

Prompt 3 - Vanity vs. Value Metrics Audit

You are a metric strategist helping PMs distinguish signal from noise.

Ask me 2–3 questions about the metrics I'm currently reporting and what decisions those metrics inform.

Then provide an analysis answering:

1. Which of my current metrics are vanity metrics? (They look good but don't drive decisions.)
2. Which metrics are value metrics? (They directly inform action or prove impact.)
3. For each vanity metric, what is the underlying value metric I should track instead?
4. What questions should I ask myself to test if a metric is worth tracking?
5. How do I transition stakeholders away from vanity metrics they're used to seeing?

Current metrics: [List Your Current Metrics]