Role-Based AI Playbooks for Delivery Teams

By Sergii Sindikaiev · Director of AI Innovations · Pillar 1 - AI Enablement of Delivery Teams

Most delivery organizations are running AI training as if every role does the same job. They don't. The developer running Copilot, the QA running an AI test generator, the PM running a meeting-summary tool, the BA writing a requirements draft, the architect sketching a decision record - these are five different cognitive workloads with five different failure modes. The training programs that anchor most rollouts treat them as one.

This is one reason the numbers often don't move.

I have led AI-enabled delivery transformation across PM, QA, Dev, SA, and BA roles, and the same observation keeps surfacing: the tools land, usage looks healthy for a quarter, and then it splits. Seniors quietly drop the tools because the friction outweighs the gain at their level. Juniors over-rely on them because the gain outweighs the friction at theirs. Nobody's role changed - the dashboards still measure license counts, not workflow redesign.

The fix is not more training. It is not better prompts. It is a per-role playbook that redesigns the workflow itself - what the role does, what it decides, and what evidence of competency looks like when AI is genuinely embedded. Five roles, five playbooks. This article is what those playbooks look like.

One-size training fails because five roles are five jobs

A developer's primary AI failure mode is autocomplete drift: code that looks reasonable, compiles, and quietly bypasses a constraint nobody wrote down. The remedy lives in workflow rules - when the agent runs, what it must produce alongside the code, who reviews what.

A product manager's primary AI failure mode is the opposite. The PM doesn't generate too much; the PM accepts too much. A meeting-summary tool produces a confident, well-structured summary of a conversation that contained three unresolved decisions, and the PM ships the summary as if the decisions were resolved. Slide polish goes up. Decision quality goes flat or down.

These two failure modes are not solved by the same training session. Not by the same prompts. Not by the same governance. The Dev playbook has to make the agent's output more conservative and more inspectable. The PM playbook has to make the human more skeptical of well-structured outputs - the exact opposite cognitive move.

When the same 90-minute "Intro to AI Tools" training is run across both roles, the Devs leave with rules they don't follow and the PMs leave with confidence they shouldn't have. The training was designed for the tools, not for the roles. The roles paid for it.

The QA, BA, and SA failure modes are different again, and we will get to each. The point this section makes is structural: generic AI training across heterogeneous roles is not a delivery investment. It is a procurement event with a presentation deck attached.

What a role playbook actually contains

Before walking through the five, a definition. A role playbook is not a prompt library. A prompt library is a starter pack - useful, two days of work to assemble, six weeks before it goes stale. A playbook is the operating description of the role after AI is embedded into it. Four components, all four required:

1. The failure mode the role hits when AI is added without redesign. Specific, named, observable. Not "lower quality." Something like "autocomplete drift on framework conventions" or "summary-as-resolution conflation."
2. The workflow redesign. What the agent does, what the human does, what review gate exists between them, what artifact the agent must produce alongside its primary output so the human can inspect what the agent decided. This is the load-bearing part. Prompt libraries skip it.
3. The senior-vs-junior pathway. Seniors and juniors fail differently and gain differently from AI. The playbook spells out what good AI use looks like at each level, and what each level needs to demonstrate before progressing. Without this, seniors disengage and juniors plateau.
4. The evidence-of-competency signal a manager can inspect. Not "ask them how it's going." Something a manager can pull up in five minutes - a PR, a test plan, a ticket, an ADR - and say with confidence that this person is using AI well or not.

If the playbook lacks any of these four, you have a slide deck. You do not have an operating model.

Here is what the five role playbooks look like in one table:

The next five sections are five playbooks built to this template.

Developers need workflow rules, not prompt libraries

The Dev failure mode I see across transformation engagements is autocomplete drift. The agent suggests a function, the developer accepts it, and the function quietly violates a project-level convention that the developer either didn't internalize or assumed the agent knew. Naming conventions, error-handling patterns, the specific subset of a library the codebase has standardized on, the security boundary that says certain calls must go through the audit logger.

The agent doesn't violate these out of malice. It violates them because nothing in its context told it not to.

The workflow redesign starts with what the agent must produce alongside the code. An effective Dev playbook requires three things on every agent-assisted commit: the code, the test that exercises the new path, and a one-line note in the PR description that says what the developer asked the agent to do. The note is not for the developer's manager. It is for the developer's future self in six months. Without it, the codebase fills with code nobody can confidently change.

The review gate moves earlier. In a non-AI workflow, the reviewer looks at code and decides whether it ships. In the AI workflow, the reviewer is also looking at whether the agent was given a coherent task. A well-bounded prompt that produced a small, focused diff is fine. A 200-line diff with a vague "implement the feature" prompt is a smell, and reviewers should be trained to call it out. The convention is enforced by review, not by tooling - tooling-only enforcement creates a game that everyone learns to play around.

Senior-vs-junior split: seniors are allowed and expected to use the agent on plumbing, refactors, and well-scoped feature additions. They are not expected to use it on architecture decisions or on debugging an outage. Juniors are expected to practice agent-assisted work on bounded tasks - the goal is not usage volume but workflow literacy - and their PRs are reviewed more closely for the well-bounded-prompt convention. Pair sessions with seniors are weighted toward "did the agent run the way the team runs it" more than syntax correctness.

Evidence of competency a manager can inspect: open three PRs from the last week. Each should have a one-line intent note, a coherent diff size, and a test for any new path. If the intent notes are vague, the diffs are sprawling, and tests are missing on the AI-assisted ones specifically, the developer is not yet a competent AI-using developer. The diffs tell you that in five minutes.

QA's failure mode is over-trusting the agent that wrote the test

A QA engineer who runs an AI test-generation tool against a feature gets twenty-eight tests in fifteen seconds. They pass. The feature ships. Two weeks later production hits an edge case nobody tested, and the post-mortem traces the gap to a behavior the AI never thought to exercise - because the AI tested what the feature does, not what could break it.

The Dev agent over-produces code; you can see the over-production in the diff size. The QA agent under-produces tests, but does so confidently and with a green dashboard, which is worse. A short list of failing tests gets fixed. A long list of passing tests gets shipped.

The workflow redesign separates two activities the old QA role used to combine. Test generation is delegated to the agent. Test strategy - what should be tested, what's the failure surface, where would adversarial users find a way in - stays with the human. The new QA artifact is not the generated test suite. It is the human-authored risk map. The artifact the agent must produce alongside the tests is a coverage gap analysis: what categories of failure these tests do not exercise. Without that, the QA engineer has no way to evaluate the agent's work except by re-doing it.

The redesign is also where QA finally lands the shift-left move the discipline has been talking about for a decade. Shift-left meant testing earlier in the cycle - at requirements, at design, at the time a story is written rather than the day it lands in a build. In a pre-AI workflow that ambition was structurally underfunded: writing test cases at requirements time required QA hours the team didn't have, so testability review collapsed into a downstream activity by default. With agent-generated cases, the marginal cost becomes low enough to move testability review earlier. The QA workflow moves into the discovery and refinement meetings: the agent drafts a candidate test set from the acceptance criteria the moment they exist; the QA engineer reviews the set for the categories the criteria failed to cover, and the gap goes back to the BA before engineering picks up the story. Shift-left in this redesign is not a slogan layered over the same workflow - it is the new place QA's testability judgment actually lives, because the agent removed the cost wall that kept it downstream.

The review gate is the test-plan review, which used to be a perfunctory check and is now the load-bearing artifact of the role. A test plan written or co-written by the QA engineer says: here are the categories of risk, here is the failure surface, here is what we are deliberately not testing and why. When the agent generates tests, those tests are checked against the plan, not against each other. The plan is the human's contribution. The tests are the agent's.

Senior-vs-junior split: seniors author test plans and review the agent's coverage gap analyses. Their AI-generated tests are not reviewed line by line - the plan is what matters. Juniors generate tests under a senior-authored plan and are evaluated on whether they caught and challenged at least one gap the plan did not anticipate. A junior who never finds a gap is a junior who is reading the plan, not testing the system.

Evidence of competency: ask a QA engineer for the last three test plans they authored or co-authored. If the plans are detailed, name specific failure modes, and explicitly list categories the team chose not to cover, the QA engineer is doing the new job. If they hand you twenty test files instead, they are still doing the old one with new tools.

PMs ship faster slides, not better decisions

The PM failure mode is the most subtle of the five and the easiest to miss because the surface metrics improve. Time-to-summarize a meeting drops. Time-to-draft-a-spec drops. Time-to-prepare-a-stakeholder-update drops. The PM is visibly more productive. The decisions inside the work are not visibly better. Often, they are quietly worse.

The mechanism is well-structured-output bias. A confident, formatted, three-bullet summary of a conversation makes the conversation feel resolved even when it wasn't. A clean spec draft makes the spec feel reviewed even when stakeholders haven't seen it. A neat stakeholder update makes the project feel on-track even when two of the three workstreams are at risk. The PM, like everyone else, weights structure higher than it should.

The workflow redesign reintroduces friction in specific places. After a meeting, the AI produces a draft summary. The PM is required to spend at least five minutes editing that summary before sending - not to polish it, but to sort what happened into four explicit buckets: decided, assumed, still open, escalated. The artifact is a short section at the bottom listing each item by name, written by the PM. If "still open" is empty, the PM either had an unusually decisive meeting or wrote the summary without reading the room. The convention exists to make the second case visible.

The review gate is moved off the artifact and onto the decision register. Specs are not reviewed against earlier specs; they are reviewed against an explicit list of decisions the PM claims have been made. If a spec depends on a decision not on the list, the spec goes back. The agent makes the spec drafting cheap. The decision register makes it accountable.

Senior-vs-junior split: seniors are expected to use the agent for drafting and not for resolution. They own the open-decisions list explicitly. Juniors are required to use the agent for both drafting and a first-pass resolution suggestion, but every suggestion they ship has to be reviewed by a senior PM or an engineering lead within forty-eight hours. The senior is checking whether the junior is using the agent to do the work or to skip the work.

Evidence of competency a manager can inspect: pull a PM's last three meeting summaries. Look at the "still open" sections. If they are empty or short, the PM is shipping AI-polished summaries without the human contribution. If they are present, specific, and tracked into the next meeting's agenda, the PM is doing the new role. Five minutes. No interview needed.

BAs need stakeholder calibration, not requirement generation

Business analysts hit a different failure mode again, and it is the one nobody wants to raise publicly. An AI tool that drafts a requirements document does so plausibly. The BA reviews the document, finds it well-organized, and ships it to engineering. Engineering builds against the document. Two iterations in, a stakeholder reads the working product and says something the BA has heard before and dreaded: that isn't what we meant.

The agent did not get the requirements wrong. The agent got the requirements the BA gave it. The BA didn't have all the requirements when they wrote the prompt, and the agent's confident output made the gap invisible. The BA's old job was partly about feeling out where the gaps were. AI-generated requirements documents short-circuit that feeling-out step exactly when it matters most.

The workflow redesign moves the agent earlier and the BA later. The BA's AI workflow should start before the requirements document exists, not after. The agent's job is to draft the questions the BA should ask stakeholders, not to draft the requirements those stakeholders supposedly already gave. The agent reads the existing context - prior projects, similar features, common gap categories - and produces a stakeholder-interview agenda. The BA runs the interview. The agent then drafts the requirements from the interview transcript. The artifact the agent must produce alongside the requirements is a list of assumptions the requirements make that were not explicitly confirmed in the interview. Those assumptions are the BA's follow-up list.

The review gate is the assumption list. Engineering doesn't start on a requirement until the BA has either confirmed the load-bearing assumptions or marked them as acceptable risk with the product owner. This is slower than letting the agent draft the document end-to-end. It is also the difference between building what they meant and building what they drafted.

Senior-vs-junior split: seniors are expected to use the agent to compress the prep and the drafting, not the interview. They own assumption surfacing as a deliverable. Juniors are required to use the agent for prep, drafting, and one round of assumption review, but their assumption lists are co-reviewed with a senior BA before engineering sees the requirements. The senior is checking calibration - does the junior know which assumptions matter?

Evidence of competency: ask a BA for the last three requirements documents and their accompanying assumption lists. Read the assumption lists, not the requirements. If the lists are short, generic, or absent, the BA is using the agent as a typing-faster tool. If the lists are specific, name the stakeholder who would need to confirm each one, and show evidence of follow-up, the BA is doing the new job.

SAs gain the most where the architecture rationale lives, not where the diagram does

Solution architects get the most overlooked role redesign of the five, partly because the visible artifact of architecture work - the diagram - is the part AI is worst at. So the assumption forms that the role doesn't change much. The role changes more than it first appears.

The SA's old work product was the architecture decision: this database, this messaging pattern, this auth model, this deployment topology. The artifact was a diagram and a paragraph. The rationale lived in the architect's head and occasionally in a wiki page nobody read again. When the architect left the team or the project paused for six months, the rationale evaporated and the next person inherited the diagram without the reasoning.

The agent is still unreliable as the final owner of architecture diagrams. The agent can write a much better rationale, and it can write it at the time of decision rather than two years later when someone asks. The workflow redesign assigns the SA the role of decision-author and assigns the agent the role of decision-recorder. The SA makes the architecture call. The agent drafts the ADR - the architecture decision record - capturing what was decided, what was rejected, why, what assumptions are load-bearing, and what would trigger revisiting the decision. The SA edits and signs the ADR. The diagram becomes a derived artifact.

The review gate moves to the ADR. Architecture reviews used to be presentations of a target state. They become walkthroughs of the rationale chain: here are the three decisions we made, here are the alternatives we rejected and why, here is what would have to change for us to revisit. The agent's draft makes the rationale chain cheap to produce; the SA's edit makes it accurate; the review makes it shared.

Senior-vs-junior split: seniors author decisions and edit the agent's ADR drafts. Their evidence of competency is a coherent chain of ADRs that another senior architect can read and reconstruct the project's history from. Juniors are paired on decisions, draft their own ADRs first (without the agent), then compare to the agent's draft and learn from the diff. The diff is the training mechanism. By the time a junior is authoring ADRs solo, they have read fifty decisions worth of "what the agent would have said vs. what I said" and they have a calibrated intuition for what a real rationale looks like.

Evidence of competency: open the last six months of ADRs from a project. If they exist, are specific, name rejected alternatives, and have triggers for revisiting, the SA is doing the new job. If they don't exist, the architecture rationale is still trapped in someone's head and AI hasn't touched the role.

DevOps's failure mode is over-trusting agent-drafted runbooks during incidents

One adjacent role makes the pattern clearer. The article focused on the five delivery roles that share an article-length redesign argument, but the operating-model logic generalizes - and the role it most obviously generalizes to is DevOps. The shape is the same; the artifacts differ.

The DevOps failure mode under AI assist is incident-pressure overconfidence. An agent that drafts a runbook during a calm Tuesday afternoon is useful and inspectable. The same agent invoked during a production incident, when the on-call has a paging app open and three Slack threads moving, produces a runbook that looks operationally crisp and gets followed faster than it gets read. Two months later the post-incident review traces a cascading failure to a runbook step that was confidently wrong and confidently followed.

The workflow redesign separates the runbook from the risk register. The agent drafts the runbook - steps, commands, rollback paths - from prior incident transcripts and infrastructure context. The human-authored artifact is the risk register sitting alongside it: what could go wrong if this runbook is followed under the wrong assumption, what alternate paths exist if step three fails, what data is irreversible if it is run incorrectly. The agent produces the procedure. The human produces the failure imagination.

The senior-vs-junior split mirrors the QA one: seniors author the risk register and edit the agent's runbooks. Juniors run agent-drafted runbooks under a senior-authored risk register, and are evaluated on whether they paused at the right step rather than how fast they completed it. Evidence of competency is the last three post-incident reviews, with the role of AI-suggested versus human-judgment calls documented. If those reviews don't separate the two, the agent has effectively been operating under the senior's name without the senior's accountability.

DevOps has its own published L1–L4 progression in the same AI Adoption framework as the other five roles, with infrastructure-specific evidence at each level. The next section maps the ladder across all six roles in one frame.

The maturity ladder: what L1–L4 looks like per role

The senior-vs-junior split in each playbook is the conceptual frame. The operating substance is a four-level maturity ladder that holds across all five roles, with role-specific evidence at each step. The full rubric is the 4-level AI adoption evaluation model. The structure is the same; the artifacts differ.

L1 - Foundations. Daily use of agentic tools, structured prompting, output verification, at least one custom agent encoding role-specific knowledge. Dev: custom agents in CLAUDE.md and structured prompts on every task. QA: a test-case-generator agent the engineer reviews and refines. PM: three or more reusable workflows for meeting processing, reports, and risk drafting. BA: a requirements-writer agent that reads discovery transcripts. SA: a decision-record skill that drafts ADRs.

L2 - Automated flows and context engineering. Repeatable end-to-end workflows wired into the role's regular rhythm. Project context - CLAUDE.md, rules, memory - actively improves AI output. The compounding loop runs every cycle: plan, delegate, assess, codify. Dev: feature-implementation flow with linter hook and tests. QA: an API-test agent plus Playwright UI conversion plus at least one LLM-evaluation practice. PM: meeting processing, reporting, metrics review, risk tracking, stakeholder tracking, quality-gates reporting - all on a rhythm. BA: discovery synthesis and requirements quality gates. SA: framework hierarchy and MCP integrations defined for the project.

L3 - Harness engineering and quality gates. Feedback loops, hooks, multi-model orchestration where the model that implements is not the model that reviews, automated quality gates that block bad output. The system catches common failures before manual review. Dev: auto-lint and verify-build PostToolUse hooks. QA: traceability and coverage hooks that block non-compliant test-suite merges. PM: project-level governance dashboards and measurement. BA: self-improving analysis with defect and feedback loops. SA: CI+AI pipeline bots - documentation on merge, security review on PRs, dependency upgrades with automated test suites.

L4 - Scaled agent systems. Multiple agents coordinating across phases. Cost controls - model selection per task - and quality monitoring on success rates and failure modes. Cross-project portability so the setup ports to new codebases without rebuilding. Dev: three or more concurrent agents on a shared codebase with CI guardrails. QA: multi-agent orchestration across functional, API, and UI testing. PM: cross-project influence and role expansion. BA: multi-agent analysis systems with cost and quality controls. SA: cross-project AI development platforms and framework evolution at scale.

A few things are worth naming explicitly.

The ladder is discontinuous, not linear. Going from L1 to L2 is not "do L1 more." It is a different practice - context engineering, the compounding loop, automated flows. Going from L2 to L3 again changes the underlying activity, from running flows to building the harness those flows run inside. Treating the ladder as a slope rather than a set of steps is what produces the plateau most teams end up on: comfortable at L1, unable to articulate what L2 would require.

The ladder is role-specific in evidence, not in shape. The same four levels apply to a developer and a business analyst, but the L1 evidence for one is custom agents and PR-intent notes; for the other it is a requirements-writer agent and a stakeholder-question template. A manager assessing AI maturity should be looking at role-specific evidence, not asking "how often do you use the tool." The artifact answers the question.

The ladder is the senior-vs-junior pathway, made concrete. The senior-vs-junior split in each playbook above is the frame; the L1–L4 ladder is the substance. A junior developer's career path is L1 → L2 over twelve to eighteen months. A senior developer's path is L2 → L3 → L4 over a similar window, with the L3 and L4 milestones requiring genuine harness engineering and multi-agent orchestration that is closer to platform work than to feature work. The DevOps role has its own framework with the same four levels and infrastructure-specific evidence - when a team is mature enough to ask about it, the levels are already named.

Why the experience curve doesn't flatten

A common board-level assumption is that AI flattens the experience curve - that it lets juniors operate at senior levels and makes years of experience less load-bearing. In delivery work, on the evidence I have seen, this is the opposite. AI raises the ceiling on judgment more than it raises the floor on tasks. Junior PRs are noticeably better because the agent catches obvious issues. Senior PRs are noticeably better because the senior knows which agent suggestions to ignore. The senior's compounding advantage is now compounding on top of an agent rather than instead of one. Without an explicit progression curve - named per role, with the L1–L4 evidence above - the senior plateau and the junior over-reliance problem become permanent.

How to start Monday

If the analysis lands and you are looking at where to begin, the first four weeks have a defensible shape. Pick one role - usually Dev or QA, because the artifacts are most inspectable - and run this:

Four weeks. One role. One playbook. One review gate. One inspection ritual. After that the second role gets cheaper, because the manager already understands the shape and the team has stopped expecting a 90-minute training to do the work.

What changes in your operating model if you take this seriously

The operating-model implication is straightforward and unglamorous. If the analysis lands, the move is not another tool purchase. Not another training program. Not another center of excellence producing decks.

The move is to commission five role playbooks - Dev, QA, PM, BA, SA - and to commit the organization to measuring AI adoption against the evidence-of-competency signals each playbook names, not against license counts or training-completion rates. The playbooks can be adapted from the five above, or written from scratch by the role leads. What matters is that they exist, that the four components are present in each, and that the organization stops running "AI training" as if the roles are interchangeable.

This is a smaller commitment than another platform rollout and a larger one than another training session. It is operating-model work. It changes how roles are described in job postings, how performance is reviewed, how managers spend their inspection time, and what counts as evidence in a board-level AI program update.

The manager-inspection-signal column is also the audit-trail column for the AI regulations now landing on every CTO's desk. The EU AI Act's deployer obligations (Article 26 and the associated risk-management framing) require organizations to document how AI is used in high-impact decisions, who is accountable, and what evidence exists that human judgment owned the decision rather than the model. Role playbooks turn out to be the artifact regulators will increasingly ask about. The intent note on the agent-assisted PR, the QA risk register sitting alongside the agent-drafted tests, the still-open / decided / assumed / escalated sections of the PM's meeting summary, the BA's assumption list, the SA's ADR chain, the DevOps risk register beside the runbook - these are the audit trail. Programs that build them for operational reasons land model-risk-management compliance as a side effect. Programs that don't build them will end up reverse-engineering them under deadline pressure when the regulator asks. The operating-model work and the compliance work converge in the same playbook; the question is whether the organization writes it deliberately or under audit.

The reason most AI transformation programs are stalling at this stage is not that the tools are bad. In many stalled programs, the tool is not the main constraint. The role was never redesigned. The dashboards still measure the procurement decision rather than the operating-model decision. The five playbooks are the operating-model decision, written down.

Reading adoption through role-level artifacts like this is the lens Shift Harness applies.

Before you buy another AI tool

Before approving the next AI license expansion, ask each delivery lead for one artifact: the role playbook. If they cannot show the failure mode, the redesigned workflow, the senior/junior pathway, and the manager inspection signal, the organization is not scaling AI adoption. It is scaling access. The first is operating-model work. The second is a procurement decision the dashboard will eventually flatter and the delivery numbers will eventually not.

Pick the role. Read the framework. Collect the ten artifacts. Write the one-page playbook. Add the one review gate and the one inspection ritual. The next AI program update at your company should reference at least one of the four playbook components by name. If it does not, the program is measuring something else.

Frequently Asked Questions