3.b.i Keep the prompt prefix cache-stable

Put static content (identity, rules, tool definitions) before a boundary and per-turn content (date, state, memory) after it; keep the prefix bit-for-bit identical across turns; strip thinking text from persisted history; and track per-turn cache-miss cost. Check the provider's caching minimum: a prefix below it never caches.

Tier 1 because a reordering that leaves what the model sees unchanged, verified on the hand-labeled set.

When it applies

In the code: a timestamp, session id or per-turn state before the static block; non-deterministic serialization; a marked-for-caching prefix shorter than the provider's minimum (in one image-labeling agent, 504 tokens against Haiku 4.5's 4,096)
In traces: cache_read tokens near zero across calls that share a prefix; cache-hit rate below about 50% on multi-turn runs

How

Read the prompt assembly code: find timestamps, per-turn state, or non-deterministic serialization (dict order, ids) early in the prompt.
Move them after the static block; sort and fix serialization.
Confirm the static prefix exceeds the provider's minimum cacheable length.
Replay the hand-labeled set; compare cache-hit rate before and after.

How to measure it

verify fix (tier 1): replay the hand-labeled set; compare cache-hit rate
monitor: cache-hit rate and cache-miss cost per turn

To compare the change against your agent's current version, with the same cases and enough repeats to see past run-to-run noise, use aot-evals.

Expected impact

Quality: none
Cost: improves, large: cached input is billed at a fraction of full price; RETRACE's cache hits rose 45.6% to 90.8% as a side effect (Aug 2026)
Performance: improves, small

Evidence

Evidence comes from: Code generation, Data analysis.

Split the system prompt at a static/dynamic cache boundaryNot measured

Claude Code SYSTEM_PROMPT_DYNAMIC_BOUNDARY; the static prefix is "cached via Blake2b hashing with scope 'global'" and "achieves high cache hit rates across turns and even across sub-agents"; OpenHands CacheTier.STATIC/DYNAMIC arrived at the same design independently; Codex prompt_cache_key derived from ThreadId. No hit-rate numbers.

Barbaste et al., "Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems" · Jul 2026 · Sec 7.2; Observation 2; Table 11 "Prompt Caching"
Normalize the prompt prefix bit-for-bit for cache hitsNot measured

Hermes "prompt prefix is bit-perfect normalized (sorted compact re-serialization of tool-call JSON, a date-only timestamp, a frozen memory snapshot) so that even local llama.cpp and vLLM KV caches hit across turns"; places four cache_control breakpoints ("system_and_3").

Barbaste et al., "Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems" · Jul 2026 · Sec 7.2 (Hermes); Observation 2
Emit every provider's cache dialect at onceNot measured

OpenCode "emits six providers' cache dialects simultaneously and shapes its system prompt to at most two messages (one by default) to match the cache slots."

Barbaste et al., "Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems" · Jul 2026 · Sec 7.2; Observation 2; Table 12 "Cache-Dialect Fanout"
Strip extended-thinking text from persisted historyNot measured

Gemini CLI "Keeps extended-thinking parts out of persisted history at response-recording time so they don't pollute future cache lookups"; cited as a mitigation for "Multi-agent systems use ~15x more tokens than chat" [17].

Barbaste et al., "Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems" · Jul 2026 · Sec 7.4; Sec 15.1
Audit per-turn cache-miss cost as a metricNot measured

Pi "audits per-turn cache-miss dollar waste as a first-class metric"; "a per-turn cache-miss dollar-waste audit that prices every avoidable cache invalidation."

Barbaste et al., "Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems" · Jul 2026 · Sec 7.5; Observation 2
Maximize prompt-cache hit rate; track it as metricNot measured

Manus called cache hit rate "the most important metric for production agents": "a higher capacity ($/token) model with caching can actually be cheaper than a lower cost model without it." Coding agents "would be cost-prohibitive without caching." No numbers.

Lance Martin, "Agent Design Patterns" · Jan 2026 · Cache Context
Share issue and trajectory prefix across verifier calls

Table 4, average per issue, SWE-bench Verified: GPT-5 mini baseline 26.3K input, 45.6% cache hit, $0.07 vs RETRACE 60.6K input, 90.8% cache hit, $0.06; output tokens 4.2K -> 1.2K. MiniMax M2.5 baseline 123.3K input, 94.6% cache hit, 0.07 vs RETRACE 78.0K, 96.7%, 0.05. Cost ratio below 1.0x on both backbones.

Chenglin Li, Yisen Xu, Zehao Wang, Shin Hwei Tan, Tse-Hsun (Peter) Chen, "Independent Patch Verification for Coding Agents with a Bidirectional Reconstruct-and-Verify Framework" · Aug 2026 · Cost analysis; Table 4
Cache the schema and semantic-layer prefixNot measured

Argued, not measured: the document adds ~2-4K input tokens per query (Table 1); with caching "the marginal cost ... is effectively zero".

Michael Rumiantsau, Ivan Fokeev (Cube), "Semantic Layers for Reliable LLM-Powered Data Analytics: A Paired Benchmark of Accuracy and Hallucination Across Three Frontier Models" · Apr 2026 · Section 5.3 Cost and Latency

Also known as

KV-cache-friendly prompt layout

In Claude Code

This optimization is a skill in aot-kit, our open-source Claude Code plugin: /aot-kit:cache-stable-prefix. Its library entry holds every source. Have new evidence, or a correction? Suggest a change.

Keep the prompt prefix cache-stable