Mahal Systems Overview › Agent type
Reasoning
Math, proofs, science, abstract puzzles.
Specific to reasoning
- 3.c.i
Self-Discover reasoning structures
Self-Discover has the model compose a reasoning structure once per task type (select useful modules from 39 seed heuristics, adapt them to the task, write them as a step-by-step plan), then follow that cached structure on every instance, so each instance costs one call.
- 7.c.iv
Graph-of-Thought
Graph-of-Thought runs one task as a graph of model calls: split the input into parts, generate and score several candidates per part, keep the best, then merge the partial results and refine the merged answer.
- 7.c.v
Language Agent Tree Search (LATS)
Language Agent Tree Search runs the agent as a Monte Carlo tree search over its own actions: sample several next actions, execute each in the environment, score the resulting states with the model, reflect on failed endings, and back the scores up the tree until a branch succeeds or the budget runs out.
- 7.c.vi
Multi-agent debate
Several model instances answer the same question, read each other's answers and revise over one or more rounds, and a vote or judge settles the final answer.
- 7.c.vii
Tree-of-Thought
Tree-of-Thought replaces one reasoning chain with a search: the model proposes several next steps, an evaluator (the model itself, a trained verifier or a rule check) scores the partial solutions, and breadth-first or depth-first search keeps the best or backtracks.
General optimizations with evidence from reasoning
- 1.b.i
Match the model and its effort to each call site and query class
Pick the model per role or call site rather than one model everywhere, and set its reasoning effort and sampling parameters per site; tune each role's prompt to its model (GEPA raised every model tested at lower cost).
- 3.a.i
Add worked examples to the prompt (few-shot), or curated example queries for data agents
Put a few solved examples of the task in the prompt (the input and the exact output wanted), or give a data agent commented example queries for its custom metrics and an index of past queries it retrieves per question.
- 3.a.iii
Dynamic context injection
Put the runtime facts the model cannot know into its context each run: today's date and timezone, the working directory, OS and git state, the tools available, the user and locale.
- 3.a.iv
Restructure the system prompt: every requirement once, up front, in named sections; nothing the model doesn't need
Rewrite the system prompt so that each requirement the task has (constraints, the answer format, the completion check, how to work efficiently, the domain procedure) is stated once, up front, in a named section, and remove instructions the model does not need.
- 6.a.i
Oracle-worker multi-model
Let a cheap model do the work and call an expensive one only when a trigger fires: a cascade escalates when the cheap answer looks unreliable or fails a check, a router picks the model before the call, and an advisor tool lets a cheap agent consult a stronger model mid-task.
- 7.a.ii
Prompt chaining with gate checks
Split a task that decomposes cleanly into fixed steps into a chain of model calls, one goal per call, with a gate between steps: a schema check, an assertion, or a run against the real system, and an LLM check only for subjective criteria.
- 7.c.i
Reflection loop
The same model critiques and revises its own output for a few rounds (Self-Refine), or after a failed attempt writes a short lesson it carries into the next try (Reflexion).
- 7.c.ii
Self-critique evaluator loop
Train or tune the system's own judge from its own outputs and synthetic judgments (contrastive pairs with one side deliberately degraded, the judge's own verdicts kept only where they match the construction label) instead of human labels, then use it to score, select or train in the loop.
- 7.c.iii
Evaluator-optimizer loop
A separate evaluator (an LLM judge with written criteria, a different model, or an agent that tests the running app) scores the generator's output and returns specific findings; the generator revises until the output passes or a round cap is hit.
- 7.d.ii
Inversion of control
Give the agent the goal, the tools and the acceptance checks and let it choose the sequence of steps, instead of coding a fixed workflow; the goal, permissions, budget and evaluation stay outside the agent.
- 9.a.i
Feed back a sound external verifier's verdict; never the model's own critique
Run a check the model cannot fool on its output (compiler, tests, schema validation, execution, EXPLAIN on generated SQL, a solver) and feed the verdict back; a pass/fail verdict is enough, and resampling until the check accepts works about as well as detailed feedback.
- 9.c.ii
Spend extra samples before extra verification: parallel attempts with a vote or judge; verify only where it pays
Running several independent attempts and picking by majority vote or a judge raised accuracy (Genie 73.8% to 81.4% with 8 runs, at up to 2.1x cost).
- 9.c.iii
Audit a verifier-gated cascade against a gold sample the verifier never sees
When a cheap model's outputs are accepted or escalated by a verifier, measure true error on a small gold-labeled sample outside the verifier: metrics measured through the verifier showed about 3% error while true error was 14-32%.