Track explicit progress: a task list with a status per item, re-injected each turn

Give the agent a task-list tool (each item with a status, one in progress at a time), ask for the list on the first turn of non-trivial tasks, and re-inject the current list after the system prompt each turn instead of keeping old copies in history. The effect depends on the model: the one controlled study found a large success gain for a weaker model, none for a mid-size one, and a small success loss with a large cost saving for strong ones, whose removed turns were mostly post-edit verification.

Tier 3 because it changes what the model sees each turn.

When it applies

• In the code: no task list or todo mechanism in the harness
• In traces: runs ending with no edit or output, and short aimless trajectories (weaker models); long post-edit loops (stronger models); failure rows labeled goal-drift or state-tracking-failure
• In eval results: success by trajectory length, per model

How

1. Add the task-list tool and the first-turn requirement.
2. Inject the current list after the system prompt each turn; strip old copies from history.
3. Paired eval per model you run: success, cost and turns. Re-run it when the model changes.

How to measure it

• experiment (tier 3): paired eval with and without the task list, separately for each model in use

To compare the change against your agent's current version, with the same cases and enough repeats to see past run-to-run noise, use aot-evals.

Expected impact

• Quality: mixed, medium: weaker model +11.6 points on SWE tasks and +4.5 on TerminalBench (a 30B model); no consistent gain at 120B; strong models about -2 points (Fan, Sep 2026)
• Cost: mixed, medium: strong models about 30% cheaper (fewer post-edit turns); the weaker model up to 75% dearer, because it now works the task instead of stopping early
• Performance: mixed: follows turns: fewer for strong models, more for the weaker one

Caveats

Model strength decides the sign. On weaker models the list keeps the agent working the task; on strong models it mainly trades a little success for a lot of cost, by cutting verification turns. Measure it per model, and again after a model upgrade. Keeping the plan in a file only matters when work outlives one context window; that is 4.b.iii.

Evidence

Evidence comes from: Code generation.

1. Add a persistent plan tool re-injected each turn

   T4/128k, full tools. Nemotron-3 30B: planning +11.6 pp SWE and +4.5 pp TB at higher cost (TB cost +75%). Without planning its median SWE trajectory is 5 turns (vs 40), and 68.6% of runs end without an edit (vs 27.8%). Nemotron-3 120B: no consistent SR gain; cost up on SWE and down about 26% on TB. Nemotron-3 550B / Mistral: SWE cost down about 30% / 32% with SR -2.0 / -0.4 pp. Median turns 108->74 and 68->53, the removed turns being mostly post-edit verification. Mistral TB cost down about 40%.

   Fan et al., "An Empirical Study of Harness Design for Coding Agents" · Sep 2026 · §2.1; §3.2 'Planning improves success rate for the weaker model...', Fig 6, Table 5; §4 Table 6, Figs 8-9; App 7.2 Figs 15-17, App 8.4 Fig 30

Also known as

• Attention recitation

Related

• 4.b.iiiKeep plan, progress and results in files when work outlives one context window
• 5.a.iCompact conversation history in stages: elide stale tool outputs, then keep an anchored summary
• 7.d.iContinuous autonomous task loop
• 5.c.iDiscrete phase separation
• 4.b.iiInitializer-maintainer dual agent
• 3.a.iiAvoid few-shot ruts

In Claude Code

This optimization is a skill in aot-kit, our open-source Claude Code plugin: /aot-kit:track-task-progress. Its library entry holds every source. Have new evidence, or a correction? Suggest a change.