This is my operating model for AI programs — how I'd run one end to end: the ways teams use AI, the lifecycle, how to prove a model is good, the risks that are unique to AI, and the gates that get pilots to production. It's a how-I-work playbook, not a claim that I've shipped a giant model — the point is the judgment, and where it comes from work I've already done.
AI programs don't fail on the model. They fail on the program. The research is blunt: ~95% of GenAI pilots never scale, and the causes are program problems — no business outcome, cost overruns, weak data, underestimated integration, governance bolted on too late.
Those are the exact things I already manage. So my pitch isn't "I learned AI." It's: the discipline AI programs need is the discipline I already run — evaluation, staged rollout, drift monitoring, risk gating, human sign-off — proven on data- and ML-adjacent delivery, and I build with AI hands-on.
The bridge
A translation chart: what I already run on the left, what the AI world calls the same thing on the right. The point — I'm not starting from zero.
| What I already run | The AI equivalent | In plain words |
|---|---|---|
| Structured A/B testing | Model / LLM evaluation | Measuring "is A better than B, with data" — the same job as scoring an AI's answers. |
| UAT → cutover → hypercare | MLOps release | Test it, roll it out slowly, watch it closely after — the same launch discipline. |
| RAID risk log | AI-specific RAID | Same risk habit, new risks: hallucination, bias, leakage, prompt injection. |
| Capacity + Ready/Done | Sequencing experiments | Plan in "try it and see" instead of fixed tasks — the planning habit still holds. |
| Stage-gates | PoC → pilot → prod → scale | Go/no-go checkpoints — the same gating, AI's four stages. |
| Steering committee | Responsible-AI review | Same approval forum, one new question: is it safe/fair, and does a human check it? |
| Migration reconciliation gate | Eval gate before ship | Nothing goes live until it passes — parity check, or a quality score. |
| Fuzzy entity resolution (shipped) | ML entity resolution | My fuzzy matching is real machine learning — I've already shipped a piece of AI. |
| Governed metric layer (shipped) | Grounding for RAG | The trusted source an AI pulls from to avoid making things up — I've built exactly that. |
The takeaway: "I already run the discipline AI programs need — I just learned its vocabulary."
How AI actually gets used
Every model shows up in three ways of working. Knowing which to reach for is basic fluency — and a real decision, because each has a different cost, speed, and risk. The simplest lens: how much the AI is allowed to touch.
| Mode | What it is | What it touches | What it's for |
|---|---|---|---|
| Chat | The conversation box (ChatGPT, Claude, Gemini) | Nothing — it just talks | Thinking, learning, drafting, deciding |
| Code | AI working inside your codebase (Claude Code, Cursor, Copilot) | Your files & systems — reads, writes, runs | Building software, automating technical work |
| Co-work | An AI teammate inside your work tools | Your whole workspace — multi-step tasks | Getting real work done with you |
Chat advises; code and co-work actually do the work. So picking the mode is a risk decision — the more the AI does on its own, the more I add the same human-in-the-loop sign-off I'd put on any risky launch. That's the whole job in one line: match the mode to the task, and gate the ones that act.
Every stage, who owns it, where it breaks, and where my governance sits.
The one big build decision
The most common enterprise question, as a simple ladder — start simple, add only what you need.
The prompt playbook
I don't just talk about AI — I use it to produce program work. These run in chat mode (draft) or co-work mode (pulling from real Jira/docs). The rule from the thesis: AI drafts, I govern — every prompt ends with a human check.
Role (who the AI acts as) + Context (my situation) + Task (what to make) + Format (table? word count?) + Constraints (what to avoid).
Act as a technical program manager. Program context: [paste one paragraph — goal, systems, teams, timeline]. Build a phased roadmap from intake → launch → post-launch. For each phase give: goal, key workstreams, entry/exit gate criteria, duration estimate, top risk. Output as a table, 5–7 phases, plain language.
Human check: are the gate criteria real for your program? Fix durations to your reality.
You are a TPM. Program context: [paste]. Draft a RAID register table — columns: Type (Risk/Assumption/Issue/Dependency), Description, Impact (H/M/L), Likelihood (H/M/L), Owner, Mitigation/next step. Give 8–12 entries specific to THIS program — no generic filler. Flag the top 3 by exposure.
Human check: delete anything generic, assign real owners, add what the AI missed.
From these notes: [paste]. Extract every actual decision made (not discussion). For each: Decision, Date, Owner, Options considered, Rationale, Reversible? (Y/N). Output as a table.
Human check: confirm each is a real decision and the rationale is right.
Act as a TPM writing a weekly executive update. Current state: [paste RAID + progress]. Write it BLUF: (1) Bottom line — on track / at risk + why, 2 sentences; (2) Progress this week — 3 bullets; (3) Top 3 risks + mitigation; (4) Decisions/asks needed. Plain, under 200 words.
Human check: does the bottom line match reality? Sharpen the asks.
From this transcript: [paste]. Extract four tables: (1) decisions, (2) action items with owner + due date, (3) open questions, (4) risks raised. Don't invent owners — mark 'unassigned' if unclear.
Human check: assign the unassigned, verify due dates.
Evaluation
You can't ship what you can't measure. This is my A/B-testing muscle applied to AI answers — four ways to check, simplest to most rigorous.
| Method | What it is | Like… |
|---|---|---|
| Gold set (offline) | Fixed questions with known-right answers; score % correct | A practice exam with an answer key |
| Human eval | People rate outputs on a rubric (accurate? helpful? safe?) | Grading essays with a rubric |
| Online A/B test | Release to a slice of real users, compare vs. the old way | Exactly my A/B program |
| Live tracking | Monitor accuracy / hallucinations / complaints after launch | Post-launch hypercare |
One concrete line
"A trusted-metrics assistant had to answer 50 known KPI questions; we shipped only when it hit ≥95% correct with zero wrong numbers — the same go/no-go discipline as a migration reconciliation gate."
Risk
The RAID that proves I can run risk on an AI program specifically — a ready-to-use template.
| AI risk | What it looks like | How I catch it | How I mitigate |
|---|---|---|---|
| Hallucination | Confident, wrong answer | Gold-set eval + require citations | RAG grounding; human review on high-stakes |
| Bias | Unfair results for a group | Test eval set across segments | Diverse eval data; fairness threshold |
| Data leakage | Private data comes out via the AI | Red-team prompts; output scanning | No sensitive data in prompts; access controls |
| Prompt injection | Bad input tricks it into breaking rules | Injection test cases | Input validation; limit what the AI can do |
| Cost blowup | Token / API spend runs away | Usage dashboard + budget alerts | Rate limits; caching; cheaper model tiers |
| Drift | Accuracy quietly drops over time | Monitor vs. a baseline | Scheduled re-eval; retrain trigger |
Three frameworks are now table stakes, not extras: the EU AI Act (full enforcement in 2026), NIST AI RMF (Govern / Map / Measure / Manage), and ISO 42001 (a certifiable AI management system). I treat these the way I treat any compliance gate — mapped to controls, signed off before launch.
Cost is now a core program control, not an afterthought — enterprise AI builds run $60K to $300K+, and integration + governance can eat up to 60% of the budget. Production spend routinely lands 3–5× the first estimate. What I put in place:
The 2026 frontier: agents that act autonomously — multiple steps, without a human in the loop for every one. Three risks to gate:
The business case
The failure rate is the whole reason a delivery leader matters here: ~88% of AI PoCs never reach wide deployment, ~95% of GenAI pilots fail to scale, and only ~12% reach production. Every cause is a program problem — which is to say, one I already prevent.
| Why pilots die | The discipline that prevents it — mine |
|---|---|
| No business outcome defined (95% fail) | Tie every program to a business metric at the gate |
| Cost runs 3–5× the estimate | AI cost governance — budget gates + monitoring |
| Data quality | A data-readiness gate (my reconciliation habit) |
| Integration underestimated by an order of magnitude | My literal migration / integration experience |
| Governance built too late | I build governance in from day one |
| Org misalignment | Stakeholder alignment — my core strength |
| Gate | Prove this | The trap it kills |
|---|---|---|
| PoC | It can work, on a small sample | Falling in love with a demo |
| Pilot | It works for real users and moves a business metric | "Pilot purgatory" — no outcome defined |
| Production | It holds at scale — cost, latency, integration, drift monitoring, rollback | Cost shock + integration underestimated |
| Scale | It's repeatable and adopted | Scaling before it's repeatable |
The line that matters: define the business outcome at the pilot gate, not after — a model with 95% accuracy that doesn't move a number is a curiosity, not an asset.
In practice — the real story
The playbook isn't theory. On a recent enterprise program I turned weeks of manual synthesis into days — with a human governing every step that carried risk. Open each View how to see exactly what I did.
The problem: 30+ recorded stakeholder interviews — marketing ops, campaign & partner managers, sales ops, report consumers, finance, regional leads — to turn into requirements and a roadmap. By hand that's ~3 weeks of reading and tagging.
Loaded all 30+ transcripts into an AI workflow — code mode to clean them, chat mode to read them.
The AI collapsed ~200 raw pain-point quotes into ~9 themes — each with a count of how many of the 30 raised it.
| Theme the AI grouped | Raised by | Example quote |
|---|---|---|
| "Pipeline means different things" (definition drift) | 22 / 30 | "Sales' number never matches marketing's." |
| "Can't tie partner / MDF spend to revenue" | 18 / 30 | "We fund events but can't prove what they sourced." |
| "I rebuild the same report every week" | 15 / 30 | "Monday mornings are copy-paste into Excel." |
| "The three systems disagree" | 12 / 30 | "Which account ID is the real one?" |
| "Numbers are two weeks stale" | 4 / 30 | "By the time I see it, it's old." |
How often a theme came up became an early priority signal.
I checked every theme against the transcripts — fixing what the AI got wrong and cutting what it made up.
The AI grouped; I decided what was true, in scope, and correctly owned.
Turned the validated themes into PRDs, wireframes, and acceptance criteria for 15+ contributors.
A "validated theme" is a corrected theme carrying: the pain → who's affected → the requirement → how we'll know it's done. The definition-drift theme became this PRD entry:
Repeated across ~9 themes → the PRD set that 15+ contributors built from.
Sequenced the roadmap by evidence — how many stakeholders raised each theme × business impact — not by who shouted loudest.
| Theme | Raised by | Phase |
|---|---|---|
| Definition drift | 22 / 30 | Phase 1 |
| Attribution blind spot | 18 / 30 | Phase 1 |
| Weekly report rebuild | 15 / 30 | Phase 2 |
| System reconciliation | 12 / 30 | Phase 2 |
| Latency | 4 / 30 | Later |
Evidence, not opinion, set the order. (A RICE score would formalize this exact call — a planned add.)
Research-to-roadmap went from weeks to days, ~60% less manual analysis, every stakeholder aligned on the same decisions.
Honest split: AI compressed the first 80%; the 20% that carried risk stayed human.
Most PMs talk about AI. This is the discipline run on a live program — and it maps straight onto the playbook: the modes (chat / code), the thesis (AI drafts, human governs), and human-in-the-loop sign-off.
Putting it together
A two-minute walk-through — a trusted-metrics assistant, built on the governed-data work I've actually done.
Why this is credible
Real ML-adjacent delivery
I've shipped entity resolution, attribution, and a governed semantic layer — the genuine data/ML lifecycle, not a course project.
I build with AI, hands-on
This portfolio, a 200-question training tool, and an interactive simulator — all built by directing AI in code mode. Most PMs can't say that.
A real operating model
This playbook itself — modes, lifecycle, eval, risk, cost, gates — the discipline that gets AI pilots to production.
The one line
"I run the program discipline AI actually needs — evaluation, staged rollout, drift monitoring, risk gating, human-in-the-loop sign-off — proven on real data delivery, and I build with AI hands-on."
Competencies