Compact conversation history in stages: elide stale tool outputs, then keep an anchored summary

Manage the history before it overflows. Past a soft threshold (about 60% of the window) replace old tool outputs with short stubs; only past a hard threshold (about 85%) summarize the oldest turns into a summary with fixed sections (goal, files touched, done, pending, errors, next step) that is updated incrementally, not regenerated. Keep the recent tail verbatim. The value is large at small windows and near zero at 128k; compaction alone does not carry multi-window work (see 4.b.iii for state kept in files).

Tier 2 because stops runs dying on context overflow, a hard failure.

When it applies

• In the code: no compaction, or compaction that summarizes the whole history at once; a context window under about 64k
• In traces: context-overflow outcomes; peak context over about 85% of the window; summaries that drop file changes or decisions; failure rows labeled context-rationale-erosion
• In eval results: failures concentrated in the longest trajectories

How

1. Measure peak context against the window on the longest runs.
2. Implement the two thresholds and the fixed-section summary.
3. Evaluate with post-compaction probes (recall, artifacts, next step) and tokens per task, not compression ratio.

How to measure it

• before after (tier 2): context-overflow rate on the same cases
• experiment (tier 3): paired eval, tokens per task, post-compaction probes

To compare the change against your agent's current version, with the same cases and enough repeats to see past run-to-run noise, use aot-evals.

Expected impact

• Quality: improves, large: +35.7 points at 32k, +2.7 at 128k (Fan, Sep 2026); staged elision matches other strategies on success
• Cost: improves, medium: staged elision lowest cost in 7 of 8 panels, e.g. $1.45 against $2.57

Evidence

Evidence comes from: Code generation.

1. five follow-up benchmarks

   MAX_HISTORY_CHARS lowered twice, never raised.

   Scale AI, VeRO code release · figure_04

2. five follow-up benchmarks

   Context-management edits in 6 to 10 of 20 runs per benchmark.

   Scale AI, VeRO code release · figure_01

3. Add context management when the window is tight

   Managed tiers (T1-T4) vs no management (T0), averaged over 4 models (Nemotron-3 30B/120B/550B, Mistral-Medium-3.5-128B): the gap is 35.7, 15.9, 5.5 and 2.7 pp on SWE-Bench Verified (n=500) and 9.5, 7.5, 4.8 and 2.8 pp on Terminal-Bench 2.1 (n=89) at 32k/64k/96k/128k. T0 overflow rate falls from 78.7% to 8.7% (SWE) and 61.0% to 12.1% (TB) over the same budgets. Managed tiers have zero overflow at every budget. Example at 32k SWE: T0 9.4/11.4/6.4/12.6% vs T1 20.6/43.0/51.4/64.4%. At 128k, managed tiers change trajectory shape very little (median turns 39-42 for 30B, 70-74 for 550B).

   Fan et al., "An Empirical Study of Harness Design for Coding Agents" · Sep 2026 · §3.2 'The value of context management grows as the context-window budget shrinks', Fig 3, Tables 3-4; §4, App 9.5

4. Skip recoverable recall of elided outputs

   T2 (elision+recall) vs T1 (elision) over 32 model-benchmark-window comparisons: 15 wins, 14 losses, 3 ties. Mean -0.36 pp (+0.40 SWE, -1.12 TB). 36 of 64 T2/T4 settings never call recall_event and the median rate is zero. Use is concentrated at 32k in Nemotron-3 30B (TB T2 4.326 calls/task), and even that setting scores below T1. The 550B and Mistral models almost never recall.

   Fan et al., "An Empirical Study of Harness Design for Coding Agents" · Sep 2026 · §3.2 'Recall (M2) is rarely used...'; App 9.6 Table 13

5. Compact at a fixed buffer, keeping a verbatim recent tailNot measured

   Claude Code "AUTOCOMPACT_BUFFER_TOKENS = 13 000"; post-compact restores "up to 5 files (POST_COMPACT_TOKEN_BUDGET = 50 000)"; Gemini "fires when history exceeds 50% ... preserves the most recent 30%"; Mistral Vibe envelope "re-injects prior user messages ... so original task goals survive resets"; 7 of 11 systems use threshold compaction.

   Barbaste et al., "Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems" · Jul 2026 · Sec 9.5; Recommendation 7; Figure 4

6. Compact reactively on context-overflow errorsNot measured

   Mistral Vibe "the same routine fires reactively when a turn overflows the context window mid-flight"; OpenHands "context-window overflows and malformed-history errors trigger a condensation rather than a crash"; Pi "one-shot compact-and-retry on context overflow."

   Barbaste et al., "Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems" · Jul 2026 · Sec 9.4; Sec 9.5; Recommendation 7

7. Soft-prune old tool outputsNot measured

   OpenCode "opt-in pruning soft-deletes old tool outputs, protecting the newest 40K tokens and firing only if more than 20K are reclaimable."

   Barbaste et al., "Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems" · Jul 2026 · Sec 9.5 (OpenCode)

8. Compact by rotating to a linked child sessionNot measured

   Hermes "compaction does not rewrite the transcript—it ends the session ... rotates to a child chained by parent_session_id ... no history is ever destroyed." Trigger at 50% of context minus max-tokens (64K floor), summary capped at min(5% of context, 12K).

   Barbaste et al., "Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems" · Jul 2026 · Sec 9.5 (Hermes); Table 12 "Lineage Compaction"

9. Compress history with a dedicated summarization modelNot measured

   "we introduce a new LLM model whose key purpose is to compress a history of actions & conversation into key details, events, and decisions. This is hard to get right ... you might even consider fine-tuning a smaller model (this is in fact something we've done at Cognition)." Still "eventually hit a limit".

   Walden Yan (Cognition), "Don't Build Multi-Agents" · Jun 2025Older evidence · A Theory of Building Long-running Agents (compression diagram)

10. Do not rely on compaction alone for multi-window workMixedNot measured

    "compaction isn't sufficient. Out of the box, even a frontier coding model like Opus 4.5 running on the Claude Agent SDK in a loop across multiple context windows will fall short of building a production-quality web app if it's only given a high-level prompt". Compaction "doesn't always pass perfectly clear instructions to the next agent."

    Justin Young (Anthropic), "Effective harnesses for long-running agents" · Nov 2025 · The long-running agent problem

11. Keep context structured so sections prune independentlyNot measured

    Design suggestion: "ensure the main instructions and goals are preserved while the document or history sections can be pruned or summarized".

    Drew Breunig, "How Long Contexts Fail / How to Fix Your Context", https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-how-to-fix-them.html ; https://www.dbreunig.com/2025/06/26/how-to-fix-your-context.html (2025-06-26) · Jun 2025Older evidence · 06-26 post, "Context Pruning" (closing paragraph)

12. Summarize accrued context before the distraction thresholdNot measured

    Gemini Pokemon agent: "anything beyond 100,000 tokens triggered this behavior" (repeating past actions) despite a 1M+ window. No effect size for summarizing.

    Drew Breunig, "How Long Contexts Fail / How to Fix Your Context", https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-how-to-fix-them.html ; https://www.dbreunig.com/2025/06/26/how-to-fix-your-context.html (2025-06-26) · Jun 2025Older evidence · 06-26 post, "Context Summarization"

13. Offload old tool results to files before summarizingNot measured

    "Manus writes old tool results to files and only applies summarization once there is diminishing returns from offloading." Addresses the concern that compaction "can result in loss of useful information."

    Lance Martin, "Agent Design Patterns" · Jan 2026 · Offload Context

14. Give summarized agent a searchable chat-history fileNot measured

    "If the agent knows that it needs more details that are missing from the summary, it can search through the history to recover them." No numbers.

    Jediah Katz (Cursor), "Dynamic context discovery" · Jan 2026 · 2. Referencing chat history during summarization

15. Elide stale tool outputs to stubs before summarizing

    T4 (staged elision, then summarization) matches T1-T3 on success and has the lowest cost in 7 of 8 model-benchmark panels. It has the lowest mean cost per task and the lowest average peak-context/budget ratio at every budget, and it calls summarization less than T3 at every budget. At 32k, T1/T2 (elision only) still reach about the full window. Example SWE 32k, Nemotron-3 550B: T4 55.6% at $1.45 vs T1 51.4% at $2.57.

    Fan et al., "An Empirical Study of Harness Design for Coding Agents" · Sep 2026 · §2.3 Algorithm 1; §3.1 Implementation Details (thresholds); §3.2 'T4 offers the best accuracy-cost trade-off', Figs 4-5

16. Summarize old history into fixed-heading running summary

    Summarization alone (T3) vs T0 at 32k SWE: 23.8/39.6/58.4/63.2% vs 9.4/11.4/6.4/12.6% (30B/120B/550B/Mistral). T3 is cheaper than elision-only at 32k, e.g. 550B $1.25 vs T1 $2.57, and keeps peak context well below the window where T1/T2 do not. The prompt is reproduced in Fig 18.

    Fan et al., "An Empirical Study of Harness Design for Coding Agents" · Sep 2026 · §2.3; App 7.3 Fig 18-19; Tables 3-4; Fig 5(a)

17. Anchored iterative summarization with fixed sections

    36,611 messages from production SWE sessions (hundreds of compression points, 4 probes each, GPT-5.2 judge, 0-5 scale): overall Factory 3.70, Anthropic SDK (full regeneration) 3.44, OpenAI /responses/compact 3.35. Accuracy 4.04 / 3.74 / 3.43; context awareness 4.01 / 3.56 / 3.64. Compression 98.6% / 98.7% / 99.3%. Differences "consistent across task types and session lengths." Factory summaries came from production; the others were generated from the same prefixes. Agent model not stated. Vendor evaluating its own method.

    Factory Research, "Evaluating Context Compression for AI Agents" · Dec 2025 · Three approaches to compression; Results table

18. Track file artifacts in a separate indexNot measured

    Proposed, not tested: artifact-trail score was 2.19-2.45/5 for all methods; it "probably requires specialized handling beyond summarization: a separate artifact index, or explicit file-state tracking in the agent scaffolding."

    Factory Research, "Evaluating Context Compression for AI Agents" · Dec 2025 · What we learned

19. Merge summaries incrementally into anchored sectionsNot measured

    OpenCode <previous-summary> merge into "Objective / Important Details / Work State / Next Move / Relevant Files"; Pi iterative merge; Rec 7: incremental merging "preserves early decisions that from-scratch re-summarization loses."

    Barbaste et al., "Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems" · Jul 2026 · Sec 9.5 (OpenCode, Pi); Recommendation 7

Also known as

• Context compaction

Related

• 10.c.iContain sensitive data and untrusted input
• 2.c.iBound tool output: cap it, or write it to a file the agent can read
• 7.b.ivTrack explicit progress: a task list with a status per item, re-injected each turn
• 7.b.iDetect stalls and loops: warn, then stop early
• 5.c.iDiscrete phase separation
• 5.c.iiSub-agent architectures with clean contexts
• 7.b.iiKeep errors in context
• 4.b.iiiKeep plan, progress and results in files when work outlives one context window
• 7.b.iiiSize run budgets so productive runs aren't cut off: turn caps, deadlines, command timeouts

In Claude Code

This optimization is a skill in aot-kit, our open-source Claude Code plugin: /aot-kit:staged-compaction. Its library entry holds every source. Have new evidence, or a correction? Suggest a change.