Introduction
Use a fixed pilot record to make the decision in this order: confirm the workload and controls, check output acceptance, calculate the complete labor change, assess commercial assumptions, and record the authorized next action. For the fictional case below, the decision is revise and retest. Only 180 of 200 AI-assisted drafts are accepted, below the organization's 95% acceptance threshold. Positive capacity value cannot override that failed quality gate.
This is a no-code, document-only exercise for AB-731 business leaders. The organization, documents, permissions, observations, thresholds, times, and monetary values are fictional teaching inputs. They are not measured Microsoft product performance, vendor prices, exam questions, or evidence of an actual deployment. You can complete the exercise without a tenant, subscription, real customer data, or cloud resources.
Prerequisites and Exercise Boundary
Before starting, be able to distinguish a business baseline from an AI output count, include failed-attempt effort, explain permission-aware grounding, and assign governance and adoption owners. AB-731: Generative AI Cost Drivers, Tokens, and ROI supplies the accounting foundation; the related data, governance, adoption, and commercial-readiness guides supply the control concepts. This exercise applies those concepts to one decision rather than teaching each one again.
Prepare a worksheet with sections for requirements, evidence, calculations, failed checks, decision, and closure. Keep these constraints throughout:
- Use only the synthetic inputs supplied here; do not upload confidential information.
- Evaluate internal account-review drafts. Sending customer messages, changing records, promising discounts, and approving business actions are excluded.
- Require a human to review every attempted draft. A failed draft goes to the manual workflow; it is not sent onward merely because it sounds plausible.
- Keep the same 200 completed account reviews in the baseline and assisted comparisons.
- Treat the scenario's permission and entitlement statements as fictional case facts. A real implementation would need current administrator verification and technical tests.
Fixed Scenario and Inputs
Fictional Northbridge Services wants ten account managers to prepare internal weekly account-review drafts faster. Each draft should summarize approved account facts, identify an unresolved issue, and state the next human decision. The business owner wants to expand a small pilot to additional teams.
The proposed starting option is a built-in Microsoft Copilot experience using approved Microsoft 365 content. Microsoft's documentation describes Copilot coordinating models, Microsoft Graph content, and Microsoft 365 apps within existing access boundaries. That product description establishes a possible fit; it does not prove the quality, permissions, entitlement, or economics of this fictional use case.
Supplied requirements and ownership
| Fixed case input | Value or constraint |
|---|---|
| Weekly workload | 200 account reviews for the same cohort |
| Baseline | Manual preparation takes 12 minutes per completed review |
| Output | Internal draft with source identifiers, unresolved issues, and no unsupported commitments |
| Quality gate | At least 95% of attempted drafts accepted without substantive factual correction |
| Human review | Every attempted draft; review capacity is 450 minutes per week |
| Access and action gate | No unauthorized content, external sending, or record changes |
| Business owner | Account Operations Lead; owns the expansion decision and outcomes |
| Control owners | Data Steward owns source currency; Security Lead owns access checks; Service Owner owns support and incidents |
| Adoption owner | Adoption Lead coordinates user guidance and feedback; champions collect issues but do not approve exceptions |
| Commercial assumption | Fictional administrator record says the proposed cohort has the required entitlement for the selected experience; real eligibility remains a separate implementation check |
These thresholds and role assignments are the fictional organization's choices, not Microsoft requirements. Microsoft's roles guidance supports making accountability explicit; its responsible AI guidance supports proportionate assessment, human oversight, and monitoring.
Synthetic source pack and permission rules
Use this small source pack to reason about the control boundary. The 200-attempt observation ledger below represents the larger fictional workload; these snippets illustrate its rules rather than claiming to reproduce all 200 records.
| Source identifier | Supplied fictional content | Rule |
|---|---|---|
| POL-7 | Current account-review policy: use the account's recorded renewal date; never invent a discount or customer commitment | Approved policy |
| POL-6 | Superseded policy with an older renewal-handling rule | Excluded from current grounding |
| ACC-A | Account A: renewal recorded as November 30; two unresolved support issues; next action requires the account owner's judgment | Available to Account A's assigned manager |
| ACC-B | Account B: renewal date is missing; source owner must confirm it | Available to Account B's assigned manager; missing information must be reported |
| ACC-C | Restricted Account C note | Unavailable to managers outside Account C's authorized group |
An expected draft for ACC-B should identify the missing date and route the question to its owner. It should not infer a renewal date from POL-6 or another account. An unauthorized user requesting ACC-C should receive no restricted content. These are expected acceptance behaviors, not captured Copilot outputs or proof that a technical access test passed.
Step 1: Select the Least Complex Suitable Option
Record the selection before considering the favorable time calculation:
| Option | Case decision and reason |
|---|---|
| Built-in Microsoft Copilot experience | Selected for evaluation: the case requires an internal draft from approved Microsoft 365 content, with human review and no automated action |
| Microsoft Copilot Studio agent | Defer: a separate managed agent or action workflow has not been justified by the fixed requirements |
| Custom application and model stack | Defer: the case has no requirement for a separate interface or custom orchestration |
| Existing manual workflow | Retain as fallback and as the option if quality or control problems cannot be resolved |
This is a requirement-based choice for the fictional case, not a claim that another product cannot perform the task. Copilot Studio's cited billing documentation specifically addresses standard-harness agents and their applicable credit options. Creating such an agent would add entitlement, environment, consumption, support, and lifecycle questions without yet fixing the case's source-quality problem.
Confirm the current experience and billing path if requirements change. Copilot Credits apply to eligible services and do not establish universal access or one management path for every Copilot interaction. The fictional weekly commercial allowance below is an assumption, not a quote derived from a credit rate.
Step 2: Trace the Review and Fallback Operation
Walk one attempt through the proposed process on paper:
- The assigned manager selects the permitted account and the current policy source.
- The manager requests an internal draft constrained to those sources, with missing information identified explicitly.
- A designated reviewer checks facts against source identifiers, current policy, and the prohibited-action boundary.
- The reviewer records accepted or rejected, the defect reason, handling time, and review time.
- For a rejected draft, the manager completes the review through the manual fallback and records that additional effort.
- The business owner reviews aggregated quality, control, capacity, and commercial evidence before any expansion decision.
Grounding depends on source quality and currency; it does not guarantee correctness. User permissions also do not establish that every accessible document is appropriate for this process. The Data Steward must remove superseded sources from the approved set, while the Security Lead verifies access boundaries. Keep these responsibilities separate so a source-currency defect is not misdiagnosed as a license or training problem.
Step 3: Check the Fixed Observation Ledger
The following values are synthetic observations supplied by the exercise, not results obtained by running a Microsoft service:
| Observation | Fixed value |
|---|---|
| Attempted drafts | 200 |
| Accepted AI-assisted drafts | 180 |
| Rejected drafts requiring full manual fallback | 20 |
| Rejection breakdown | 12 stale-policy errors and 8 unsupported next-action statements |
| Assisted handling time | 5 minutes for every attempt, including rejected attempts |
| Human review time | 2 minutes for every attempt, including rejected attempts |
| Additional manual fallback | 12 minutes for each of the 20 rejected attempts |
| Confirmed unauthorized disclosure or prohibited action in the supplied record | None reported; absence of a report is not runtime verification |
Apply the gates separately:
| Acceptance check | Calculation or evidence | Case result |
|---|---|---|
| Draft quality | 180 / 200 = 90%; threshold requires at least 190 / 200 | Fail: ten fewer accepted drafts than required |
| Complete workload | 180 accepted drafts plus 20 manually completed fallbacks = 200 completed reviews | Met in the supplied ledger; fallback does not turn a rejected AI draft into an accepted one |
| Review staffing | 200 × 2 = 400 minutes; available capacity is 450 minutes | Fits the current cohort; expansion capacity is not established |
| Access and prohibited actions | Check permitted account, denied account, missing fact, stale source, and attempted external action against expected behavior | Technical evidence is still required before real expansion |
| Commercial fit | Check actual entitlement, service path, attributable consumption, and budget owner | Fictional allowance only; revalidate against current terms before implementation |
For a real pilot, record the identity, allowed source, expected result, observed result, and reviewer for each negative access or action test. Do not mark a technical check passed merely because the policy worksheet describes the right boundary. The quality failure already requires revision; incomplete technical evidence independently prevents production approval.
Step 4: Calculate Capacity With One Labor Boundary
Use the net-time-returned method from the economics guide. All three assisted labor activities are deducted once for the same 200 completed tasks:
| Labor item | Calculation | Weekly minutes |
|---|---|---|
| Manual baseline | 200 × 12 | 2,400 |
| Assisted handling for all attempts | 200 × 5 | 1,000 |
| Review for all attempts | 200 × 2 | 400 |
| Extra fallback for rejected attempts | 20 × 12 | 240 |
| Total assisted workflow labor | 1,000 + 400 + 240 | 1,640 |
| Net time returned | 2,400 − 1,640 | 760 |
The 760 minutes equal 12 hours and 40 minutes of potential weekly capacity. Do not omit the 20 rejected attempts, apply review only to accepted drafts, or subtract reviewer labor again after monetizing this net time.
For this exercise alone, assume the organization values capacity at USD 40 per hour and allocates USD 300 per week for incremental software and service costs. Both figures are hypothetical; the USD 300 excludes labor already counted above and is not a Microsoft price.
- Conditional capacity valuation: 760 / 60 × USD 40 = USD 506.67 per week, rounded once at the end.
- After the assumed nonlabor allocation: USD 506.67 − USD 300 = USD 206.67 per week of remaining conditional capacity value.
This is not cash saving, profit, guaranteed return on investment (ROI), or a payback calculation. The organization has not demonstrated that released time changes expenditure or revenue. One-time preparation, control remediation, rollout costs, and any additional ongoing effort would also need evidence before a full investment decision. The calculation is favorable under its stated boundary, but the quality gate still fails.
Step 5: Diagnose Failure and Define the Retest
The supplied defect breakdown points to two hypotheses, not proven causes:
| Defect | Investigation and owner | Required retest evidence |
|---|---|---|
| 12 stale-policy errors | Data Steward traces the cited source/version and checks whether POL-6 remained reachable; verify that POL-7 is authoritative | Current-policy cases cite POL-7; stale or conflicting evidence is identified and escalated rather than silently reused |
| 8 unsupported next-action statements | Service Owner and reviewer inspect source support, instructions, and review criteria | Missing facts remain missing; the draft requests human judgment instead of inventing a commitment |
| Potential unauthorized access or action | Security Lead checks identity, source permissions, and action boundaries | Allowed and denied identities behave as expected; prohibited sending or record modification cannot occur through the selected workflow |
Limit the next cycle to the original cohort. Correct the approved source set and guidance, retain the original failed examples as regression cases, and add fresh representative cases so the team cannot satisfy the gate by memorizing corrections. Review every retest attempt and record new handling, review, rejection, fallback, and consumption data; do not carry the current 760-minute benefit forward as a promised result.
For the same 200-attempt retest size, the fictional quality gate remains at least 190 accepted drafts, with all attempts reviewed and no unresolved access or prohibited-action failure. A different workload or cohort needs an explicitly comparable test design and newly approved thresholds. Expand only when quality, technical controls, staffing, ownership, and commercial evidence all pass together.
Suspend the pilot immediately if any unauthorized disclosure, unreviewed customer communication, prohibited record change, or inability to enforce required review is confirmed. The incident owner contains the affected path, preserves appropriate evidence, and routes reassessment to the authorized control owners. If mandatory controls cannot be restored or the use case remains unsuitable after remediation, stop it and retain the manual workflow. These severity rules are fictional organizational guardrails, not vendor-defined incident categories.
Step 6: Record the Decision and Close the Exercise
Write a decision record with the following result:
| Decision field | Completed case result |
|---|---|
| Decision | Revise and retest; do not expand |
| Accountable decision owner | Account Operations Lead, with required control-owner signoffs |
| Reason | 90% draft acceptance misses the 95% threshold; real technical and commercial evidence remains necessary |
| Value interpretation | Positive conditional capacity value, with review and fallback counted once; no demonstrated cash return |
| Required actions | Resolve source-currency and unsupported-action defects, complete access/action checks, and repeat a representative bounded evaluation |
| Reassessment trigger | A complete retest record satisfying every release condition; any material change in sources, audience, actions, or entitlement reopens review |
For this document-only exercise, retain the worksheet, labeled synthetic inputs, calculations, failed checks, owners, and decision as a learning artifact. Remove temporary duplicate worksheets or draft snippets you no longer need. No cloud resources, licenses, agents, or permissions were created, so no service teardown is required.
If applying the method to a real pilot, its owners must separately close temporary access, stop unused metered workloads, and retain or delete evidence under organizational policy. Do not assume deleting a local worksheet retires an agent, revokes a permission, or ends billing.
Conclusion
The correct decision for the supplied case is revise and retest. The workflow returns 760 potential minutes per week under consistent accounting, but 180 accepted drafts do not satisfy the required 190. Expansion depends on accepted output, accountable controls, staffing, and verified commercial conditions together. A defensible decision record preserves the failed evidence and specifies what must change before the next approval.
Authoritative Sources
- Grounding data design for AI workloads
- Adopt and secure AI and data
- Data, privacy, and security for Microsoft Copilot
- License options for Microsoft Copilot
- Microsoft 365 Copilot adoption essential guide
- Define roles, responsibilities, and decision rights
- Apply responsible AI
- Understand usage-based billing and cost management for Copilot Credits
- Copilot Studio standard harness licensing
Related Reading
- Prerequisite: AB-731: Generative AI Cost Drivers, Tokens, and ROI
- Prerequisite: AB-731: Data Quality, Reliability, Bias, and Security in Generative AI
- Prerequisite: AB-731: Responsible AI Governance and the AI Council
- Prerequisite: AB-731: Building an AI Adoption Team, Champions Program, and Change Strategy
- Prerequisite: AB-731: Copilot and Foundry Licensing, Cost, Data, and Security Planning
- Related: AB-731: A Decision Review Checklist for AI Transformation Leaders