Escalation on Ambiguity

Escalate to a human only when a decision is high-stakes, hard to reverse, and confidence is genuinely low.

Context & Reliability·Lesson 5 of 6·7 min

An agent operating autonomously will regularly encounter situations where it isn't fully certain what the right action is. The question this domain tests is not whether agents should ever ask for help, but under what specific conditions escalating to a human is the correct call versus simply making the best available judgment and proceeding.

The three-part test

Escalation is warranted when three conditions hold together: the decision is high-stakes, the action is hard to reverse, and the agent's confidence is low. All three matter, and the exam specifically tests whether you'll escalate on just one or two of these being true. A decision can be high-stakes and irreversible but still not warrant escalation if the agent is highly confident in the right action based on clear instructions or unambiguous data. A decision can involve real uncertainty but not warrant escalation if the stakes are low or the action is trivially reversible — in that case, the right move is usually to make a reasonable choice, proceed, and let a subsequent step or a human correct course later if needed.

This is a deliberate design constraint against the two easy failure modes. An agent that escalates on every uncertain judgment call becomes useless — it turns into a chain of human-approval requests that defeats the purpose of automation and produces "escalation fatigue," where humans stop reading the requests carefully because there are too many of them, including trivial ones. An agent that never escalates, on the other hand, will eventually take an irreversible, high-stakes action on a low-confidence guess, which is the exact scenario escalation exists to prevent.

Routine judgment calls are not escalation triggers

Most ambiguity an agent encounters is routine: which of two reasonably similar phrasings to use, which of several valid file locations to put a new module in, how to interpret a slightly underspecified but low-consequence instruction. These should be resolved by the agent making its best call, ideally stating its assumption explicitly in its output so a human reviewing the result later can catch and correct it if needed. Reserving escalation for the narrow, genuinely high-stakes-and-irreversible-and-uncertain intersection is what keeps the mechanism useful.

Exam trap A question describes an agent that pauses and asks for human confirmation before renaming a local variable in a private helper function, because "it wasn't 100% sure the new name was better." This is framed as admirably cautious. It is not — a variable rename in a private function is low-stakes and trivially reversible (a code review or a follow-up commit fixes it instantly), so this is escalation fatigue in miniature, not good judgment. Contrast with a question where an agent proceeds to delete a production database table without confirmation because it was "reasonably confident" the table was unused — that's the opposite failure, proceeding on insufficient confidence when the action is high-stakes and irreversible.

Scenario: An agent managing infrastructure-as-code is asked to "clean up unused resources" and finds a storage bucket with no read/write activity in the logs for 90 days, but the bucket's name suggests it might be a disaster-recovery backup target that legitimately sees no routine traffic. Should it escalate? Deleting a storage bucket is hard to reverse (data loss, assuming no separate backup), the stakes are high if it turns out to be load-bearing, and the agent's confidence about its actual purpose is genuinely low given the ambiguous signal from the name. All three conditions hold, so this should escalate with a clear summary: what the agent found, why the name suggests uncertainty, and what evidence would resolve it, rather than either deleting the bucket or silently skipping it without flagging the ambiguity to anyone.

Try it

List five decisions a coding agent working in your codebase might face in a normal week (examples: renaming a function, deleting a seemingly-unused file, changing a default config value, modifying a database migration, updating a dependency version). For each, score it on the three axes — stakes, reversibility, and your confidence in the right call — and decide which ones genuinely clear the bar for escalation versus which are routine judgment calls the agent should just make and note.