Mahal Systems AOT Library
Our AOT Library is the most comprehensive collection of agent optimization strategies. Updated daily, researched, and vetted against our benchmarks.
Browse the library here, search for strategies to implement below, or plug it into your coding agent with the prompt in the left rail.
- optimizations
- 60
- 60 complete, 0 in development
- research papers
- 217
- of 279 sources cited
- benchmarks
- 57
- named in the evidence
- measured results
- 386
- of 662 cited
- results against
- 166
- mixed or contradicting, kept in
- last updated
- Oct 26
- refined daily
1Models
b.Finding specialists
- i.
Match the model and its effort to each call site and query class
Pick the model per role or call site rather than one model everywhere, and set its reasoning effort and sampling parameters per site; tune each role's prompt to its model (GEPA raised every model tested at lower cost).
2Tools
a.Refactoring
- i.
Reduce the tool surface: remove and consolidate tools, then defer the rest behind search
Offer the agent only the tools the task needs.
- ii.
Rewrite the tool surface: names, schemas, descriptions
Change what the model sees of each tool without touching its implementation: a distinct name, a strict schema (enums, required fields), and a description with fixed sections (what it is for and when to use it versus sibling tools, defaults and limits, error modes, side effects), plus a usage example for complex parameters.
- iii.
Choose how the agent acts: typed tools, code execution, or shell, matched to the model and task
Decide the action interface deliberately, by model and by task, and provision what it needs.
b.Avoiding misuse
- i.
Make tool calls and tool results fail soft, and retry transient failures
Keep a failed call from ending the run: validate tool calls and model responses, return tool errors to the agent as results, and retry only failures that are transient and safe to repeat.
- ii.
Tell the agent when to use each tool, in context it always sees
For each tool or skill that is under-used or misused, add an explicit trigger rule where the model always sees it (system prompt, AGENTS.md): the situation that calls for the tool, and what to use it instead of (e.g.
c.Trimming results
- i.
Bound tool output: cap it, or write it to a file the agent can read
Cap the characters of tool output kept per call (VeRO settled near 15-20 KB on TerminalBench), or, better, write long output to a file and return its path with a short head, so the agent can read more if it needs to.
3Prompts
a.Structuring
- i.
Add worked examples to the prompt (few-shot), or curated example queries for data agents
Put a few solved examples of the task in the prompt (the input and the exact output wanted), or give a data agent commented example queries for its custom metrics and an index of past queries it retrieves per question.
- ii.
Avoid few-shot ruts
An agent's own history, and any examples in its prompt, act as few-shot demonstrations: when the context fills with near-identical action-observation pairs (twenty similar items in a row, a retried call), the model tends to copy their surface form instead of judging the current item.
- iii.
Dynamic context injection
Put the runtime facts the model cannot know into its context each run: today's date and timezone, the working directory, OS and git state, the tools available, the user and locale.
- iv.
Restructure the system prompt: every requirement once, up front, in named sections; nothing the model doesn't need
Rewrite the system prompt so that each requirement the task has (constraints, the answer format, the completion check, how to work efficiently, the domain procedure) is stated once, up front, in a named section, and remove instructions the model does not need.
b.Caching
- i.
Keep the prompt prefix cache-stable
Put static content (identity, rules, tool definitions) before a boundary and per-turn content (date, state, memory) after it; keep the prefix bit-for-bit identical across turns; strip thinking text from persisted history; and track per-turn cache-miss cost.
- ii.
Mask tools instead of removing them
When the tools an agent may use change mid-task (by mode, phase or permission), declare every tool once, keep the definitions byte-stable, and restrict the choice another way: OpenAI's allowed_tools or tool_choice "none", Anthropic's appended tool_removal and tool_addition blocks, a decode-time logit mask or grammar on self-hosted models, and a harness check that rejects calls outside the allowed set.
c.Sizing reasoning effort
- i.
Self-Discover reasoning structures
Self-Discover has the model compose a reasoning structure once per task type (select useful modules from 39 seed heuristics, adapt them to the task, write them as a step-by-step plan), then follow that cached structure on every instance, so each instance costs one call.
4Memory
b.Resuming
- i.
Filesystem-based agent state and checkpoints
Save the agent's state at each step boundary (the conversation, the workspace files, process state when it matters, and a ledger of external side effects) so a crashed or stuck run resumes or rolls back instead of starting over.
- ii.
Initializer-maintainer dual agent
Give the first session of a multi-session build a setup-only prompt (expand the spec into a JSON list of end-to-end tests, all failing, and write an init script, a progress file and a first commit) and every later session a maintainer prompt that reads that state, re-tests what passed, builds one failing item, verifies it end to end and commits.
- iii.
Keep plan, progress and results in files when work outlives one context window
When a task spans context resets (compaction, a truncation limit, a new session), have the agent keep what the next window needs in files and read them back at the start of each window: the plan and a progress list, decisions and open items, intermediate results too large to hold in context, and for coding a structured test list and an init script.
c.Reusing work
- i.
Episodic memory retrieval
Log every agent session as a raw trace, distill a few verified lessons from them (user corrections, definitions, workflows that worked, mistakes) into short memory files, show the agent an index of those files at the start of each task, and let it search both.
- ii.
Skill library evolution
Have the agent turn runs that passed a verifier into reusable skills (a code function or a SKILL.md procedure), store them in a library, and on later tasks show it only the few a separate selection step picks, reviewing what goes in and pruning what goes unused.
5Context
a.Compacting
- i.
Compact conversation history in stages: elide stale tool outputs, then keep an anchored summary
Manage the history before it overflows.
b.Filtering input
- i.
Give the model the schema it queries, in context: descriptions, keys, codes, sample values
When the model writes SQL or structured filters, put the schema it is querying in its context: table and column descriptions, key relationships, what enum codes and nulls mean, sample values.
c.Isolating subtasks
- i.
Discrete phase separation
Run research, planning and implementation as separate agent sessions, each starting from a fresh context that holds only the previous phase's written output (a research document, then a plan file), and have a person review that document before the next phase starts.
- ii.
Sub-agent architectures with clean contexts
Hand a context-heavy side task (code search, reading many files, documents or logs) to a sub-agent that starts with a fresh context and only the tools it needs, and have it return a short result or file references so the parent's context stays small.
6Topologies
a.Escalating
- i.
Oracle-worker multi-model
Let a cheap model do the work and call an expensive one only when a trigger fires: a cascade escalates when the cheap answer looks unreliable or fails a check, a router picks the model before the call, and an advisor tool lets a cheap agent consult a stronger model mid-task.
b.Parallelizing
- i.
Parallel tool execution for read-only calls
Let the model put several independent read-only tool calls (searches, fetches, file reads) in one turn and have the harness run them at once, while calls that write, share state or need another call's result run one at a time in the order the model emitted them.
- ii.
LLM map-reduce
Give each piece of a large or untrusted input its own tool-less model call that sees only that piece and must answer in a fixed schema, then combine the answers in code or in a reducer that never sees the raw text.
- iii.
Use multi-agent orchestration only for breadth-first, independent sub-tasks
Spawn parallel sub-agents when the work splits into independent, breadth-first parts with little shared context; expect about 15x the tokens of a chat and up to 90% less wall-clock.
c.Separating privileges
- i.
Code-then-execute (CaMeL)
Have a privileged model turn the trusted request into a small program before any untrusted data is read, then run it in an interpreter that labels every value with where it came from and checks a per-tool policy before each call, so injected text can neither add actions nor send private data to the wrong place.
- ii.
Plan-then-execute
Have the agent fix its tool calls (an ordered list or a dependency graph) from the trusted request before it reads any tool output, and let a controller execute only those calls, so injected text cannot add an action.
7Orchestration
a.Routing
- i.
Route metric questions through a deterministic semantic layer; fall back to text-to-SQL
Where a semantic layer exists, have the agent answer metric questions by choosing metrics and dimensions that the layer compiles to SQL, and write SQL only when the layer cannot answer.
- ii.
Prompt chaining with gate checks
Split a task that decomposes cleanly into fixed steps into a chain of model calls, one goal per call, with a gate between steps: a schema check, an assertion, or a run against the real system, and an LLM check only for subjective criteria.
b.Bounding execution
- i.
Detect stalls and loops: warn, then stop early
Track repeated identical actions (same tool, same arguments; same search query) and turns with no new information.
- ii.
Keep errors in context
Return every failed action to the model as an error it can read (the call, the cause, a capped trace) instead of swallowing the exception, retrying silently, returning an empty result or deleting the failed attempt, and collapse each error once it is resolved.
- iii.
Size run budgets so productive runs aren't cut off: turn caps, deadlines, command timeouts
Set the turn or step cap, the wall-clock deadline and per-command timeouts from what runs that succeeded actually needed, not from framework defaults, so a run that is still making progress is not stopped short.
- iv.
Track explicit progress: a task list with a status per item, re-injected each turn
Give the agent a task-list tool (each item with a status, one in progress at a time), ask for the list on the first turn of non-trivial tasks, and re-inject the current list after the system prompt each turn instead of keeping old copies in history.
c.Scaling compute
- i.
Reflection loop
The same model critiques and revises its own output for a few rounds (Self-Refine), or after a failed attempt writes a short lesson it carries into the next try (Reflexion).
- ii.
Self-critique evaluator loop
Train or tune the system's own judge from its own outputs and synthetic judgments (contrastive pairs with one side deliberately degraded, the judge's own verdicts kept only where they match the construction label) instead of human labels, then use it to score, select or train in the loop.
- iii.
Evaluator-optimizer loop
A separate evaluator (an LLM judge with written criteria, a different model, or an agent that tests the running app) scores the generator's output and returns specific findings; the generator revises until the output passes or a round cap is hit.
- iv.
Graph-of-Thought
Graph-of-Thought runs one task as a graph of model calls: split the input into parts, generate and score several candidates per part, keep the best, then merge the partial results and refine the merged answer.
- v.
Language Agent Tree Search (LATS)
Language Agent Tree Search runs the agent as a Monte Carlo tree search over its own actions: sample several next actions, execute each in the environment, score the resulting states with the model, reflect on failed endings, and back the scores up the tree until a branch succeeds or the budget runs out.
- vi.
Multi-agent debate
Several model instances answer the same question, read each other's answers and revise over one or more rounds, and a vote or judge settles the final answer.
- vii.
Tree-of-Thought
Tree-of-Thought replaces one reasoning chain with a search: the model proposes several next steps, an evaluator (the model itself, a trained verifier or a rule check) scores the partial solutions, and breadth-first or depth-first search keeps the best or backtracks.
d.Shaping the loop
- i.
Continuous autonomous task loop
Run the coding agent from a shell or harness loop that starts a fresh headless session every iteration with the same short prompt; each session reads the plan, progress notes and git history, does one item, runs its tests, commits and exits, until an iteration cap, a budget or a stop rule ends the run.
- ii.
Inversion of control
Give the agent the goal, the tools and the acceptance checks and let it choose the sequence of steps, instead of coding a fixed workflow; the goal, permissions, budget and evaluation stay outside the agent.
- iii.
ReAct loop
Run the agent as a loop: the model reasons, calls a tool, reads the result and picks the next step, until it answers or hits a cap.
8Retrieval
a.Expanding sources
- i.
Choose the retriever by corpus size and question type
On a bounded corpus (up to about 100K documents), grep and file reads over the raw text beat an embedding retriever: the agent narrows into documents it has found.
- ii.
Surface information the source holds but the agent cannot read
Find information a connected source holds that the agent never reads, and expose it.
b.Querying precisely
- i.
Tune the search tool: snippets, enough results, retries, reformulation, filters, source quality
Make the search tool return what the model needs to choose what to open (a snippet per result, not just titles; five or more results; a full-document reader beside truncated search), retry failed searches, and reformulate when a query returns little (start broad, then narrow; use the corpus's own terms) rather than retrying it verbatim.
- ii.
Link large schemas by exploration: start broad, grow the linked set with probing queries, favor recall
When a schema is too large to load whole, link the tables and columns a question needs by exploration rather than by one up-front retrieval or by loading everything: start from all table names plus the top retrieved columns, give the agent a column-level vector index, metadata probes and trial queries to grow the linked set, favor recall over minimizing noise, and cap the exploration turns low.
9Verification
a.Checking externally
- i.
Feed back a sound external verifier's verdict; never the model's own critique
Run a check the model cannot fool on its output (compiler, tests, schema validation, execution, EXPLAIN on generated SQL, a solver) and feed the verdict back; a pass/fail verdict is enough, and resampling until the check accepts works about as well as detailed feedback.
- ii.
Verify citations claim by claim for factual support
Extract claim-citation pairs deterministically, then check each: the link resolves, the source is relevant, and the source actually supports the claim.
b.Confirming completion
- i.
Make the final answer reliably extractable: state the format, extract robustly, force an answer at the limit
State the required answer format in the prompt, and make the harness extract it robustly: strip markers, normalize numbers, detect refusals, and force a best-effort answer when the loop ends on a budget rather than returning nothing.
- ii.
Triage false success with a cheap trained detector, not an LLM judge
Train a lightweight text classifier (TF-IDF with logistic regression) on 50-100 labeled runs to flag likely false successes for review.
- iii.
Verify completion against the environment before the agent stops
Never take the agent's own report that it finished.
c.Sizing verification
- i.
Don't prompt agents to write their own tests; suppress it for test-heavy models
Agent-written tests are mostly print probes and do not change resolve rate.
- ii.
Spend extra samples before extra verification: parallel attempts with a vote or judge; verify only where it pays
Running several independent attempts and picking by majority vote or a judge raised accuracy (Genie 73.8% to 81.4% with 8 runs, at up to 2.1x cost).
- iii.
Audit a verifier-gated cascade against a gold sample the verifier never sees
When a cheap model's outputs are accepted or escalated by a verifier, measure true error on a small gold-labeled sample outside the verifier: metrics measured through the verifier showed about 3% error while true error was 14-32%.
10Safety
a.Sandboxing
- i.
Constrain what the agent can touch
Run the agent in an OS-level sandbox with a workspace path guard that resolves symlinks, scrub credentials from the command environment, run agent-written SQL read-only with timeouts and row caps, and set resource limits on generated code.
b.Gating risky actions
- i.
Gate irreversible actions: allow, ask or deny by side effect
Classify every action the agent can take by its side effect (read, reversible write, irreversible or externally visible: sending email, issuing a refund, deleting a record, posting publicly, pushing code) and gate by class: allow, ask a human or a policy, or deny.
c.Containing data
- i.
Contain sensitive data and untrusted input
Tokenize PII in the client so it never reaches the model, wrap untrusted tool and web results in taint delimiters, scan context files and skills for prompt injection, and state an explicit instruction-precedence order (user, repo, defaults, external data as data).
Nothing matches that sentence yet. Change a blank, or .