Triage false success with a cheap trained detector, not an LLM judge

Train a lightweight text classifier (TF-IDF with logistic regression) on 50-100 labeled runs to flag likely false successes for review. LLM judges, including reasoning models and tuned judge prompts, peaked at AUROC 0.65 and were anti-correlated on false success against honest failure; the detector reached 0.83-0.95 and caught 72% at a 10% review budget.

Tier 2 because detects a silent failure mode; it does not change the agent.

When it applies

• In traces: runs that end with a success claim; the closing message text and the tool-call sequence are the detector's inputs
• In eval results: an LLM judge currently used to decide whether a run completed; a labeled sample with a false-success rate above a few percent

How

1. Label 50-100 runs as true success, false success or honest failure from the environment (9.b.iii).
2. Train the classifier on the run's closing text and tool sequence.
3. Route the top 10% by score to review; track catch rate against the labels.

How to measure it

• monitor: catch rate at a fixed review budget against a labeled sample

To compare the change against your agent's current version, with the same cases and enough repeats to see past run-to-run noise, use aot-evals.

Expected impact

• Quality: improves, medium: as a review triage signal: 72% of false successes caught at a 10% review budget, against 13% for the best judge (Jun 2026)
• Cost: improves, large: compared with an LLM judge
• Performance: improves, large: 3,300x lower latency than a judge call

Evidence

1. Triage false success with a lightweight trained detector

   Task-disjoint AUROC 0.849 (LR) / 0.825 (XGB) / 0.827 (DeBERTa-v3) on tau2-bench and 0.953 (XGB) on AppWorld API-call sequences, versus best LLM judge 0.640 and 0.537. At a 10% flag rate the detector recovers 72% of false successes [95% CI 63-80] vs 13% [8-17] for the best judge (Sonnet 4.5 no_closing); at 5%, 51% vs 6%. Precision at 10% flag rate is only 50%. 1.19 ms per CPU inference vs ~4,000 ms per judge call (~3,300x). Cross-domain zero-shot transfer drops to 0.69 (airline<->retail); cross-generation (tau-v1 -> tau2) 0.73; rewriting honest-failure closings in confident style flips the detector on 10/50 (20%).

   Laksh Advani, "From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents" · Jun 2026 · §4.2, §4.4, Tables 4 and 6, Appendix H

2. Calibrate the detector on 50-100 in-domain examples

   tau2-bench airline<->retail LODO: TF-IDF+LR 0.693 zero-shot -> 0.736 (K=10 per class) -> 0.788 (K=50) -> 0.827 (K=100). TF-IDF+XGB degrades 0.657 -> 0.538 at K=100. Entity stripping does not help (LR 0.690, XGB 0.684).

   Laksh Advani, "From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents" · Jun 2026 · Appendix I, Table 13; §5 Limitations

3. Ensemble detector with an LLM judge

   Detector and judge scores correlate at rho=0.41; a 75/25 weighted ensemble raises tau2-bench AUROC from 0.834 to 0.855. Author calls the gain modest.

   Laksh Advani, "From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents" · Jun 2026 · §4.4, Figure 7

4. Do not rely on an LLM judge to detect false success

   tau2-bench, 295 trajectories, 5 judges (GPT-4o, Sonnet 4.5, Llama 3.3-70B, DeepSeek-R1, o3-mini) x 5 prompt conditions x 3 phrasings: no configuration exceeds AUROC 0.65 on false success vs true success (best 0.640). On false success vs honest failure every cell is 0.18-0.30, i.e. anti-correlated with truth; assertion-vocabulary trajectories score 0.27-0.36 more "completed" regardless of outcome. AppWorld, 200 API-call sequences, 3 judges: best 0.537.

   Laksh Advani, "From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents" · Jun 2026 · §4.3, Table 5, Figures 4-5

5. Judge prompt tweaks do not fix completion judging

   tau2-bench: Sonnet 4.5 blind 0.576, checklist 0.576, stepwise 0.590, tool_only 0.605, no_closing 0.640; adding the full task spec (strong-real) moves 0.640 -> 0.632 ("judges do not lack information, they fail to use it"). AppWorld: telling the judge GET-only sequences cannot complete write tasks raises GPT-4o 0.394 -> 0.537 but lowers Sonnet 4.5 (blind 0.368 -> with_task 0.289 -> checklist 0.274) and Llama 3.3-70B (0.505 -> 0.384).

   Laksh Advani, "From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents" · Jun 2026 · §3.5, §4.3, Table 5, Appendix D

6. Reasoning models do not prevent false success

   Qwen3-Max-Thinking has the highest tau2-bench false-success rate (79% of failures; range across 8 families 13% GPT-5.2 to 79%); its <think> traces average 1,274 chars in false-success cases vs 486 for other models. As judges, DeepSeek-R1 peaks at AUROC 0.573 and o3-mini at 0.554.

   Laksh Advani, "From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents" · Jun 2026 · §4.1, §4.3, Figure 1, Table 5

Related

• 9.b.iiiVerify completion against the environment before the agent stops
• 9.c.iiSpend extra samples before extra verification: parallel attempts with a vote or judge; verify only where it pays
• 7.c.iiSelf-critique evaluator loop

In Claude Code

This optimization is a skill in aot-kit, our open-source Claude Code plugin: /aot-kit:false-success-detector. Its library entry holds every source. Have new evidence, or a correction? Suggest a change.