The short answer
A scenario check gives a person a realistic work situation, asks what they would do, and shows what good looks like. It tests judgment, which a multiple-choice quiz cannot. Score each response on three questions: did they spot the risk, did they take a safe next step, and did they name who decides or reviews.
Why do quizzes fail for AI training?
A quiz tests whether someone can recall a rule. Working with AI is mostly judgment: noticing that an answer looks too tidy, deciding that a document should not go into a tool, or knowing who to ask. People can pass a quiz on a data policy and still paste the wrong thing under deadline pressure.
A scenario check puts the person in the situation. They decide what to do, then see what good looks like. That is closer to the real task, it surfaces misunderstandings a quiz hides, and it works well in a live session because the discussion afterwards teaches as much as the answer.
Our 400-user case study used scenario-based knowledge checks in every session for this reason.
What is the anatomy of a good scenario check?
Every scenario has four parts:
- Situation. Two to four sentences with a realistic role, task, and complication.
- Decision. One clear question: "What do you do next?"
- Good response. The two or three things a strong answer includes.
- Debrief. A sentence on the habit it builds.
Rules that keep them useful:
- Use synthetic details; never real names, figures, or data.
- Include one tempting wrong answer, usually the faster one.
- Write them in your team’s language, with your tools and report names.
- Keep each under two minutes so a session can run several.
How do you score a scenario check?
Score each response on the same three questions, 0 to 2 points each:
| Criterion | 0 | 1 | 2 |
|---|---|---|---|
| Spotted the risk | Missed it | Noticed something is off | Named the specific risk |
| Safe next step | Unsafe or none | Partly safe | Clear, safe action |
| Named who decides or reviews | No one | Vague | Named role or process |
A total out of six per scenario is enough. Track results by scenario, not just by person, because a scenario most people miss usually means a gap in the training or the policy, not in the people. We do not publish a pass threshold here; set one with your sponsor based on the risk of the role.
What are twelve scenario checks you can adapt?
All are illustrative, with invented details. Adapt the tools and terms to your organization.
Finance
1. The tidy summary. *Claude’s variance brief says every overage is a timing difference, but the source rows show one item with no confirmed cause.* Good response: verify each claim against the ledger, keep the unconfirmed item visible with its owner, and do not publish until a person signs off.
2. The raw export. *An analyst wants to paste the full ledger export, which includes vendor bank details.* Good response: stop, request or prepare the approved reconciled extract, and use only that.
HR
3. The sensitive letter. *A manager asks Claude to draft a performance letter and includes the employee’s medical situation for context.* Good response: do not enter personal or health data, use a template with placeholders, and have HR review before anything is sent.
4. The screening shortcut. *A recruiter wants Claude to rank applicants and reject the bottom half.* Good response: AI does not make hiring decisions; use it for drafting or summarizing against stated criteria, with a person accountable for every decision.
Operations
5. The invented date. *Claude summarizes a supplier email and states a delivery date the email never mentions.* Good response: check every figure and date against the source, ask Claude to quote the sentence it relied on, and correct the summary.
Legal and compliance
6. The confident clause summary. *Claude summarizes a customer contract that is under NDA.* Good response: confirm the tool is approved for contract data and that a written exception exists; treat the summary as a draft for counsel, not advice.
Customer success and sales
7. The made-up policy. *Claude’s draft reply promises a refund policy that does not exist.* Good response: check the real policy, remove any commitment not in it, and have a person approve before sending.
IT and security
8. The shiny plugin. *A teammate found a connector online and wants to install it.* Good response: if it is not on the approved list, submit it for review; do not install it.
Managers
9. The fabricated citations. *A team member submits an AI-assisted report and two sources do not exist.* Good response: verify all citations, coach on the verification habit, and set the expectation that the author owns the accuracy.
Anyone
10. The personal account. *Someone uses a personal free account "just this once" for a work document.* Good response: do not do it, report it through the agreed path, and use the approved tool.
11. The marked drawing. *A file arrives marked EAR99 and a colleague wants it summarized.* Good response: EAR99 is an export classification, not permission; ask the named decider before entering it. See training with CUI or ITAR data.
Agents
12. The helpful cleanup. *An AI agent asked to tidy a shared folder proposes deleting files it considers duplicates.* Good response: do not approve deletions blindly, check exactly which files are affected, and keep a backup. Destructive actions need a person’s approval.
How do you run scenario checks in a live session?
A format that works in a 60 to 90 minute session:
- Teach the habit for ten minutes with a short demonstration.
- Hand out two or three scenarios and give people three minutes to decide individually.
- Compare answers in pairs, then as a group.
- Show what good looks like and discuss the tempting wrong answer.
- Record results by scenario for the readout.
Keep the tone free of blame. People should feel safe saying what they would actually do, because that is the data you need. For the measurement side, see how to measure Claude training ROI.
Frequently asked questions
What is a scenario-based assessment?
It presents a realistic work situation and asks what the person would do, then shows what good looks like. It tests judgment rather than recall.
How many scenario checks should a training session include?
Two or three per session is usually enough, each under two minutes, with time for discussion afterwards.
Should scenario checks be graded pass or fail?
A simple 0 to 2 score on three criteria works well. Review results by scenario to find training gaps, and set any pass threshold with your sponsor based on role risk.
Sources and further reading
Product features, plans, and limits change often. This guide reflects the vendor documentation listed above as of November 2026. Check the vendor’s current documentation before you make purchasing or policy decisions. TechStack Consulting is an independent training firm and is not affiliated with Anthropic or OpenAI.