01 · What breaks
Design for the failure modes, not only the happy path.
Escalation fails in predictable ways. A queue exists on paper but nobody works it, or reviewers approve what they cannot check. Each card below is a design choice to settle before go-live.
An escalation queue nobody owns
A shared inbox of cases needing review fills up and ages, and people who asked for help wait for days. Give each queue one accountable owner, a backup, a response time and an overdue alert to a manager. Pause new runs when the backlog passes a set size.
Confidence used as the only risk signal
A model can be confident and wrong, and a high-stakes case can look routine to it. Stop rules should also fire on content and context, whatever the confidence: distress or safety words, suspected fraud, money or rights at stake, record conflicts and requests outside the mission's scope.
Reviewers seeing conclusions without evidence
A packet that says "proposed: not employed" invites a rubber stamp. Show the record on file and the new evidence side by side, highlight what changed, and offer four actions: approve, edit, ask for evidence or reject. If a reviewer cannot check it in a few minutes, the packet is incomplete.
Overrides that disappear from the audit trail
When a reviewer edits a value directly in the record, the mission's proposal and the reason for changing it vanish. Store the proposal, the decision, the reviewer, the time and a mandatory reason as a separate override record, reversible wherever the outcome allows.
02 · Minimum viable model
Give every escalated case its own record, from trigger to lesson learned.
In DozAI missions, autonomy is earned task by task, the way an apprentice earns trust. The ladder below sets what the mission does at each stage and what evidence moves it up. A random sample of cases keeps going to review at every stage.
| Stage | What the mission does | What people do | Move up when |
|---|---|---|---|
| Shadow | Runs on past cases and acts on nothing | Compare its calls with staff decisions | Agreement is consistently high on the case types in scope |
| Draft | Proposes every classification | Approve or edit each one | Edits become rare and their reasons are understood |
| Limited action | Clears simple, named case types | Review unusual cases plus a random sample | Sampled cases hold up and overturns stay low |
| Managed autonomy | Handles defined case types end to end | Audit samples and own the rules and exceptions | Reviewed on a schedule, and can move back down |
03 · Implementation sequence
Move through reviewable gates.
Design the human side as carefully as the automated side. These steps produce a set of rules, a reviewer packet and a staffing plan that a programme head can sign.
List consequential decisions and failure modes
Name every decision in the mission that affects someone's rights, money or opportunity, such as a reported placement, an eligibility result or a referral. For each, ask who is harmed if it is wrong, how they would find out and how it is undone. Those decisions stay with people.
Define stop and escalation rules
Write stop rules in plain language with an example each: graduate mentions harassment, salary differs from the baseline by more than a set band, request falls outside follow-up. Add that sensitive data goes only to approved roles, and an outside expert receives the minimum, not the whole conversation, unless the person consents.
Design the reviewer packet
Put the record on file, the new evidence, the rule that fired, the proposed decision and the mission's history on one screen, with sensitive answers marked. Test the packet by giving two reviewers the same ten cases. If they reach different decisions, the packet or the rule is unclear.
Test capacity and response time
Estimate from the shadow stage how many cases each run will escalate, then size reviewer hours before go-live. Agree a response time per trigger type, with the fastest handling for safety and distress, and plan cover for holidays, month-end and reporting crunches.
Record overrides and downstream effects
Each override records what changed, why, and what it affected next: a report figure, a support call or a re-placement task. Where the effect can be reversed, note how. Turn each override reason into a test case before the related rule is changed.
Review patterns and adjust autonomy
Each month, read overrides by reason, case type and reviewer. A case type with steady agreement can move up a stage. One with rising overturns, a new kind of failure or a policy change moves back down. The programme head approves both kinds of move, and the change is logged.
04 · Evidence
Measure whether the work became clearer.
Watch both sides of the handoff: whether the right cases stop, and whether people act on them in time. Our guide to AI agents for nonprofits shows where this fits in a wider AI plan.
Move a case type back a stage when overturns on sampled cases rise for two review cycles, when a stop rule misses a case that a person later finds, when a funder or policy definition changes, or when review queues run overdue. Reducing autonomy is routine maintenance, not a failure. The log should show who decided it and why.
05 · Responsible automation
Let systems prepare. Keep consequence with people.
Systems can support
- Pausing a case when a stop rule fires
- Assembling the reviewer packet with its evidence
- Alerting owners and managers when reviews run late
- Logging overrides and turning reasons into test cases
People should decide
- The consequential decision on each case
- Accepting residual uncertainty
- Changing mission rules
- Expanding or reducing autonomy
06 · Common questions
Practical questions teams ask.
Questions operations and AI leads ask when they design the point where a mission hands work to a person.
When should an AI workflow hand a case to a person?
Whenever the next step affects someone's rights, money or opportunity and the case is not routine. Typical triggers are distress or safety concerns, suspected fraud, evidence that conflicts with the record, requests outside the mission's scope and low confidence. Write these down as stop rules before go-live, and send a random sample of routine cases to review as well.
What information should be passed during an AI-to-human escalation?
Enough for the reviewer to decide without reopening the system: the record on file, the new evidence such as a transcript or document, the rule that triggered the escalation, the mission's proposed decision and what it has already done. Share only what the reviewer needs. A full conversation should go to an outside expert only with the person's consent.
Which actions can AI pause or prepare, and which require a person?
AI can schedule, retry, switch channels, transcribe, honour opt-outs at once, propose outcomes and draft summaries, and it can always pause itself. A person decides whether conflicting evidence counts, what meets a funder's definition, anything involving distress, safety or suspected fraud, and any change to the definitions or rules the mission follows.
What should an override record contain?
The mission's original proposal, the reviewer's decision, who made it and when, and a reason chosen from a list with an optional note. It should also capture what the decision changed downstream, such as a report figure or a support call, and whether it can be reversed. Overrides kept this way become test cases for the next rule change.
How do you decide when an AI mission can work with less review?
Promote one case type at a time, on evidence. Run the mission in shadow on past cases, then in draft with every decision checked, then let it clear simple cases while unusual ones and a random sample still go to review. Move a case type back down if overturns rise, a stop rule misses a case or the rules change.