Overview › 7Orchestration › c.Scaling compute

Scaling compute

Spend extra compute only where it pays. 7 optimizations.

1. i.

   Reflection loop

   The same model critiques and revises its own output for a few rounds (Self-Refine), or after a failed attempt writes a short lesson it carries into the next try (Reflexion).

   Better for brandMixed qualityHigher costSlower

   Tier 3 · tuning · changes control flow · any agent · 11 sources, 10 measured

2. ii.

   Self-critique evaluator loop

   Train or tune the system's own judge from its own outputs and synthetic judgments (contrastive pairs with one side deliberately degraded, the judge's own verdicts kept only where they match the construction label) instead of human labels, then use it to score, select or train in the loop.

   Mixed qualityHigher cost

   Tier 3 · tuning · changes control flow · any agent · 11 sources, 10 measured

3. iii.

   Evaluator-optimizer loop

   A separate evaluator (an LLM judge with written criteria, a different model, or an agent that tests the running app) scores the generator's output and returns specific findings; the generator revises until the output passes or a round cap is hit.

   Better governanceBetter for brandMixed qualityHigher costSlower

   Tier 4 · restructuring · changes control flow · any agent · 15 sources, 13 measured

4. iv.

   Graph-of-Thought

   Graph-of-Thought runs one task as a graph of model calls: split the input into parts, generate and score several candidates per part, keep the best, then merge the partial results and refine the merged answer.

   Mixed qualityHigher costSlower

   Tier 4 · restructuring · changes control flow · reasoning agents · 9 sources, 7 measured

5. v.

   Language Agent Tree Search (LATS)

   Language Agent Tree Search runs the agent as a Monte Carlo tree search over its own actions: sample several next actions, execute each in the environment, score the resulting states with the model, reflect on failed endings, and back the scores up the tree until a branch succeeds or the budget runs out.

   Mixed qualityHigher costSlower

   Tier 4 · restructuring · changes control flow · reasoning, planning agents · 14 sources, 13 measured

6. vi.

   Multi-agent debate

   Several model instances answer the same question, read each other's answers and revise over one or more rounds, and a vote or judge settles the final answer.

   Better governanceMixed qualityHigher costSlower

   Tier 4 · restructuring · changes control flow · reasoning agents · 6 sources, 4 measured

7. vii.

   Tree-of-Thought

   Tree-of-Thought replaces one reasoning chain with a search: the model proposes several next steps, an evaluator (the model itself, a trained verifier or a rule check) scores the partial solutions, and breadth-first or depth-first search keeps the best or backtracks.

   Mixed qualityHigher costSlower

   Tier 4 · restructuring · changes control flow · reasoning agents · 11 sources, 9 measured