The Approval Button Is Not a Safety Feature

Adding a human to the loop is now a one-liner. The OpenAI Agents SDK lets you mark a tool needsApproval and pauses the run with an interruption until someone approves or rejects. Anthropic's Claude Agent SDK exposes a can_use_tool / canUseTool permission callback (Python / TypeScript). LangGraph gives you interrupt(). All three document it as a first-class safety feature, and the pitch writes itself: the agent will not take a consequential action until a person signs off.

For a large and important class of actions, that sign-off is ceremony. Not because the feature is badly built, but because the human on the other end of it cannot do the thing the design assumes they are doing. When an agent proposes a fifteen-step plan that reaches across three systems, or asks you to approve "send" on a memo synthesized from a dozen retrieved documents, the reviewer is being asked to certify a chain while the interface shows them an action. They click approve. The system records that a human was in the loop. Nothing was actually checked.

Make it concrete. An agent proposes to send a customer their remediation plan. The visible action is one email. The chain behind it retrieved prior tickets, summarized the contractual obligations, chose which incidents to disclose, decided which uncertain findings to omit, and drafted legal-adjacent language. Approving "send" certifies all of that, and none of it is on the screen.

This article makes a precise version of that claim, defends it with three independent mechanisms, and then does the harder and more useful thing: it says exactly when human oversight is meaningful, and what four functions should replace per-action approval for the cases where approval is theater. It is a framework with falsifiable predictions, not a measurement paper; the two experiments that would test it are named at the end.

The claim, stated precisely

The uncomfortable sentence is: human-in-the-loop approval is often security theater, because the human cannot meaningfully evaluate the model's action chain. That sentence is substantially correct as a description of dominant current practice, and wrong only if read as a general impossibility claim. Keeping those two readings apart is the entire payoff. Human oversight of agents is not hopeless. Per-action approval of illegible, multi-step action chains is, and conflating "this specific pattern fails" with "oversight is impossible" is how the conversation usually goes wrong in both directions.

The way to make the claim testable is to define meaningfulness against the right baseline. Human approval is meaningful only when it measurably lowers the probability of harm compared to the controls you should already have:

Approval is meaningful when P(harm | strong deterministic controls + human approval) is materially lower than P(harm | strong deterministic controls alone), at an acceptable review cost and false-positive rate.

The baseline is not "no controls." It is least privilege, policy-as-code, egress limits, spend caps, static analysis, and enforced rollback. If the human in the loop is doing work a policy engine already does deterministically, the approval step is theater by definition, no matter how earnest the reviewer. The interesting question is never "did a human approve it?" It is "did the human approval change the outcome distribution, over and above the deterministic floor?"

Why per-action approval of illegible chains fails

Three mechanisms make this a structural failure rather than an implementation bug. They are independent, which is why fixing any one of them does not rescue the pattern.

Attentional decomposition. A reviewer evaluating an action in context has to hold three things at once: the task's intent, the execution trace that got here, and the semantics of the specific artifact in front of them. Human working memory does not stretch across long chains while holding all three. The likely anchor is the working-memory-capacity literature, where the usable capacity is on the order of four chunks (Cowan, 2001); the application to reviewing agent action chains is an extrapolation from that, not an established result, and it is exactly the kind of thing a cheap experiment could confirm or refute. The predicted consequence is sharp: past a handful of steps, per-action review silently degrades into local evaluation. The reviewer checks whether this step looks reasonable on its own and loses the ability to see harm that is emergent across the chain. This is the mechanism behind the widely-shared intuition that detection collapses for complex actions, and the prediction that follows (that detection of illegible high-risk actions stays low regardless of how nice the approval UI is, because the bottleneck is working memory, not information availability) is a falsifiable one worth running before anyone ships "better approval screens" as a fix.

Harm decomposition against static thresholds. Per-action approval usually gates on a per-action risk threshold. An adaptive adversary, or simply an optimizer under pressure, fragments a harmful outcome into individually sub-threshold pieces, none of which trips the gate, and reassembles the harm across the trajectory. The defense is not a better per-action threshold; it is cumulative, trajectory-level risk accounting and capability-scoped execution, so that the budget is spent across the whole session rather than reset at every step. Static per-action gates are the wrong instrument against an opponent who can see the threshold.

The fatigue-sensitivity coupling. The naive rebuttal to all of this is "then review more carefully, and escalate more." That backfires, and there is a clean model of why. Treat the reviewer's vigilance as decaying with escalation load, and false positives as consuming that vigilance. Raising a monitor's sensitivity adds coverage but also adds false alarms, and past an optimum the false alarms destroy vigilance faster than the added coverage helps, so harm reduction actually falls as you escalate more. The specific numbers in any such model are illustrative parameter choices, not measurements, but the qualitative conclusion (an optimal escalation rate exists, and pushing past it is counterproductive) is robust to the parameters and rests on a well-supported premise: the cry-wolf and automation-complacency literature has documented for decades that false alarms erode operator response (Parasuraman and Riley, 1997; Parasuraman and Manzey, 2010). "Escalate more" is not a safe default; there is an optimum, and it should be estimated empirically rather than assumed.

The two axes that actually decide

The reason approval sometimes works and sometimes is theater is that two different properties of an action get collapsed into one word, "risk." Separate them and the picture resolves.

• Risk: reversibility, scope of harm, and authorization sensitivity.
• Legibility: can a domain expert evaluate this action, with the evidence available, in roughly two minutes?

These are independent. A fifty-thousand-dollar wire transfer is high-risk but highly legible: the amount, the recipient, and the reversibility are all right there. A memo synthesized from fifteen retrieved documents may look lower-risk than a wire transfer, but it can be deeply illegible: verifying it means checking the retrieval set, source interpretation, omissions, and policy consequences. Plotting risk against legibility gives four quadrants with genuinely different answers:

The load-bearing cell is the bottom right. Per-action approval of illegible high-risk actions is not merely insufficient; it is contraindicated, because it actively manufactures false confidence. It becomes admissible again only if the system can compress the action into a legible evidence packet that a qualified reviewer can evaluate within the review budget, which is exactly the move the illegible-high-risk cell has not made. And it does something worse than waste effort. It launders accountability: it assigns blame to a human for an outcome they had no capacity to evaluate. When an illegible action goes wrong after a human clicked approve, the org points at the human, when the honest allocation is that the system designer chose to route an unevaluable decision through a human rubber stamp. "A person approved it" should never be the safety story for an action no person could assess. (The quadrant boundaries are a decision aid, not constants; where a given action sits, and the two-minute legibility bar, should be tuned to the domain.)

Show the execution trace, not the reasoning

A common instinct for the illegible cases is to give the human more of the model's thinking: show the chain-of-thought. This is the wrong evidence. A model's stated reasoning is self-narration, and self-narration is not a reliable account of what actually drove the output. The evidence for that is direct: insert a biasing feature into a prompt and models will change their answer while their chain-of-thought never mentions the bias (Turpin et al., 2023). Mechanistic work points the same way, finding cases where a model's articulated reasoning diverges from the computation actually producing the answer (Lindsey et al., Transformer Circuits, 2025).

What a reviewer needs is the execution trace, not the narration: the concrete tool calls, the data lineage, the state and risk diffs, and the rollback path. Layer it, so the default view is compact (the exact action and parameters, the target system, irreversible side effects, the risk classification, the policy basis, and the rollback plan) with raw tool logs, retrieved documents, and full provenance available on drill-down. A summarized rationale is acceptable as the sole basis for a decision only when the action is low-risk, reversible, and directly inspectable. For anything else, the rationale is an index into evidence, not a substitute for it.

The four functions that replace approval

If per-action chain approval is the wrong human operation for the hard quadrant, the answer is not "remove the human." It is to give the human operations they can actually perform. Four survive scrutiny, and they share a structural signature: compact evidence, an explicit semantic contrast to judge against, domain authority or genuinely exogenous context, legibility, and low review volume. Where all of those hold, expert detection is plausibly high; where none hold (which is exactly the case for per-action chain approval) it is not.

1. Constraint authoring (upstream, primary). Humans define the capability-scoped operating envelope before execution, rather than judging actions one at a time inside it. This moves the human from gatekeeper to policy author.
2. Artifact review (single-point). A domain expert reviews one complete output (a letter, a diff, an email) against the task specification. This is a fundamentally different operation from approving "send this next step," because it does not require holding a long chain in working memory. It is the operation humans are actually good at, but only when the artifact comes paired with a compact evidence packet (source excerpts, provenance, diffs from the user's intent, flagged policy conflicts, irreversible consequences). A bare artifact with no evidence is subject to the same illegibility test as an action, and can fail it.
3. Conflict resolution. The system automatically detects a policy conflict it cannot itself adjudicate (a privacy-deletion request colliding with a legal hold, say) and surfaces it. The human supplies the normative resolution. Detection is mechanized; the judgment is the human's contribution.
4. Exogenous-context injection and deep-audit sampling. Humans supply world-state the system provably lacks (the account manager who recognizes a customer from a meeting the agent has no record of), and provide statistical coverage of novel failure modes through asynchronous, random deep audits rather than per-action gates.

All four sit on top of a deterministic-first control base: least privilege, policy-as-code, egress controls, spend caps, static analysis, enforced rollback, and, for adaptive adversaries, cumulative trajectory-level risk accounting and capability-scoped execution. The humans are the narrow residual on top of that base, not the base itself.

What is genuinely unsolved

This framework has real soft spots, and naming them is part of taking it seriously.

• Constraint authoring might itself be partly theater. The framework applies a sharp marginal-value test to per-action approval and then quietly exempts its own preferred residual. Human-authored constraints are also policy-as-code, and they face the same specification-gaming that indicts action approval. Whether expert-authored constraints actually beat a competent security engineer's default deny-list is untested, and it should be tested before "constraint authoring" is enshrined as the primary human function.
• The legible-to-illegible ratio in real deployments is unmeasured. That single number decides whether this framework is a tuning note (in a mostly-legible world, per-action approval is usually fine) or a sweeping indictment (in a mostly-illegible world, it is usually theater). Nobody has measured it, and it is deployment-specific. Measure it in your own system before deciding how much of this applies to you.
• Artifact-review detection may not transfer to adversarial content. The optimism about single-artifact review comes from known-error-type review, where the reviewer knows what class of mistake to hunt for. Adversarial or novel content, where the reviewer does not know what they are looking for, could be substantially harder, and the honest expectation is a meaningful drop.
• The expertise-staffing gap is a live risk. Every claim that human review works well assumes an actual domain expert. Production HITL is frequently staffed by generalists, and a generalist reviewing an expert-domain artifact may be back in theater regardless of the framework. Either route review to real experts or acknowledge the gap honestly.

A word on the numbers, because it matters for how this piece should be read. The detection and approval rates that appear in the underlying analysis (predictions that single-artifact review clears some threshold under specified conditions, that illegible-high-risk detection stays low, that stratified approval rates differ by how obvious the harm is) are falsifiable predictions, not measurements. Every one of them is a designed experiment nobody has run. The cheapest and most decisive is a two-hour study of the core attentional-decomposition claim: does single-artifact detection hold up while chain-review detection collapses as the chain lengthens, invariant to UI quality? Publish the framework as reasoning; run that experiment to turn its central prediction into a result. Where a real number does exist, it is sobering rather than reassuring: the landmark legal-document-review study found exhaustive manual human review averaged about 59 percent recall (Grossman and Cormack, 2011), which makes optimistic detection predictions look optimistic, and sharpens rather than softens the case for measuring before claiming.

Bottom line

Stop shipping, and stop marketing as safety, per-action human approval of illegible, multi-step agent chains. It is theater for the hard quadrant, it manufactures false confidence, and it launders accountability onto reviewers who could not have caught the failure. Build the deterministic-first control base first, add cumulative trajectory-level monitoring, classify every high-risk action on risk against legibility, and contraindicate approval in the illegible-high-risk cell. Reposition humans into the four operations they can actually perform (constraint authoring, artifact review, conflict resolution, and deep audit), staff them with real domain experts, show them execution traces rather than model narration, and keep the escalation rate below the point where false alarms eat vigilance. Then run the two tests the framework itself cannot skip: whether human-authored constraints beat a good default deny-list, and what the legible-to-illegible ratio actually is in your deployment.

The feature is a one-liner. Knowing when it is a safety control and when it is a rubber stamp is the actual engineering, and no SDK ships that.

References

The load-bearing citations

1. Miles Turpin, Julian Michael, Ethan Perez, Samuel R. Bowman. Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting. NeurIPS 2023. arXiv:2305.04388. arxiv.org/abs/2305.04388
2. Jack Lindsey, et al. (Anthropic). On the Biology of a Large Language Model. Transformer Circuits Thread, 2025. transformer-circuits.pub/2025/attribution-graphs/biology. Cite for the divergence between stated reasoning and the mechanism producing the output.
3. Nelson Cowan. The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences 24(1):87-114 (2001). pubmed.ncbi.nlm.nih.gov/11515286. The working-memory-capacity anchor for the attentional-decomposition hypothesis (the HITL application is an extrapolation).
4. Raja Parasuraman, Victor Riley. Humans and Automation: Use, Misuse, Disuse, Abuse. Human Factors 39(2):230-253 (1997). sagepub. The cry-wolf / false-alarm effect.
5. Raja Parasuraman, Dietrich H. Manzey. Complacency and Bias in Human Use of Automation: An Attentional Integration. Human Factors 52(3):381-410 (2010). sagepub
6. Maura R. Grossman, Gordon V. Cormack. Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review. Richmond Journal of Law & Technology 17(3), Art. 11 (2011). scholarship.richmond.edu. The ~59% manual-review recall datum.

Human-factors grounding (situation awareness, automation surprises)

7. Mica R. Endsley. Toward a Theory of Situation Awareness in Dynamic Systems. Human Factors 37(1):32-64 (1995).
8. Nadine B. Sarter, David D. Woods, Charles E. Billings. Automation Surprises. In Handbook of Human Factors and Ergonomics, 2nd ed. (1997).
9. M. L. Cummings. Automation Bias in Intelligent Time Critical Decision Support Systems. AIAA 1st Intelligent Systems Technical Conference (2004). arc.aiaa.org

Agentic-AI context (threat model, evaluation, capabilities)

10. Kai Greshake, et al. Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. AISec @ CCS 2023. arXiv:2302.12173. arxiv.org/abs/2302.12173. The trace-injection / compromised-auditor threat.
11. Shunyu Yao, et al. ReAct: Synergizing Reasoning and Acting in Language Models. ICLR 2023. arXiv:2210.03629.
12. Xiao Liu, et al. AgentBench: Evaluating LLMs as Agents. ICLR 2024. arXiv:2308.03688.
13. Shuyan Zhou, et al. WebArena: A Realistic Web Environment for Building Autonomous Agents. ICLR 2024. arXiv:2307.13854.
14. Laura Weidinger, et al. (DeepMind). Taxonomy of Risks posed by Language Models. ACM FAccT 2022.

Product hook (per-action approval as a shipped primitive)

15. OpenAI Agents SDK, human-in-the-loop (needsApproval + run interruptions). JS docs · Python docs
16. Anthropic Claude Agent SDK, can_use_tool / canUseTool permission callback (Python / TypeScript).
17. LangGraph / LangChain deep agents, human-in-the-loop via interrupt. docs.langchain.com

Author's note on method

This draft was produced through a structured multi-model deliberation and then citation-verified against the primary sources above. The deliberation is how the argument was drafted and stress-tested, not evidence for it. One episode is worth reporting because it is a clean live instance of the article's own rule. During the deliberation, one participant introduced three fabricated citations: a nonexistent paper attributed to a real researcher, an invented technique, and a made-up statistic dressed as a finding. Their falseness is a mechanizable property, so a deterministic citation-checker would have caught all three instantly; the human-analog reviewer overseeing the deliberation could not verify them in real time and added value only on the non-mechanizable dimension, the quality of the argument. That is the framework's organizing rule playing out live: mechanizable risk to deterministic controls, scarce human judgment to the non-mechanizable residual. One correction for the record, because a companion verification pass caught it: a separate citation the deliberation later flagged as invented, Callaway et al. 2022 on resource-rational planning, is a real Nature Human Behaviour paper; what was wrong was a specific claim hung on it, not the citation's existence. Consistent with the rule, every number the deliberation produced was treated as false until verified: the three fabrications and an unsourced "70-85%" detection figure were removed, and real published rates were substituted where a number is load-bearing.

Original contributions (relative to the HITL literature)

• A meaningfulness criterion for human approval, defined against a strong deterministic baseline rather than against no controls, which makes "is this HITL real or theater?" a testable question.
• The risk-by-legibility classification, with legibility as an axis independent of risk, and the contraindication rule for the illegible-high-risk quadrant.
• "Accountability laundering" as the normative failure mode of approving unevaluable actions.
• The four-function decomposition (constraint authoring, artifact review, conflict resolution, deep audit) that replaces per-action chain approval, unified by a single legibility-and-volume signature.