2.a.i Reduce the tool surface: remove and consolidate tools, then defer the rest behind search

Offer the agent only the tools the task needs. First remove tools no task uses; then consolidate tools with overlapping scopes (read and write tools with a method parameter, tools grouped around jobs), because rewording cannot separate tools that genuinely overlap. Then keep a small core always loaded and defer the rest: names and one-line purposes in the prompt, full definitions loaded on demand through tool search, per-server files or expandable groups. Defer once definitions exceed about 10k tokens or 10 tools; keep deferred groups under about 10 tools each. Every tool offered has a cost even when not called: one irrelevant tool cut accuracy from 98.2% to 63.5%. Deferral costs extra calls (2-3x) and time (about 50%), so consolidate first.

Tier 1 because removing tools that are never called, verified unchanged on the hand-labeled set.

When it applies

In the code: more than about 10 tools, or more than about 10k tokens of tool definitions, in every prompt; duplicate or near-duplicate tool names or descriptions (73% of audited MCP servers have repeated names); tools the agent's credentials cannot use
In traces: tools never called across the eval runs; tool-use coverage below the eval's needs (share of needing cases where the right tool was loaded and chosen; GitHub measured 69% for a static list); calls to a tool where none was needed; failure rows labeled incorrect-tool-selection concentrated on one pair of tools
In eval results: accuracy falling as tools are added (an ablation over catalog size); paired eval, current tool set against a reduced one

How

From traces over the eval set, list each tool's call count and the cases where it was needed.
Remove tools with no calls and no needing case.
Find overlapping pairs (similar names or descriptions, or cases routed to the wrong one of the pair) and merge them; a method parameter often replaces several tools.
If what remains exceeds about 10 tools or 10k tokens, keep a core loaded and defer the rest behind search or expandable groups.
Run the paired eval, old set against new, and compare coverage, success, calls and latency.

How to measure it

verify fix (tier 1): replay the hand-labeled set after removing unused tools
experiment (tier 3): paired eval, old tool set against new: success, coverage, calls, latency

To compare the change against your agent's current version, with the same cases and enough repeats to see past run-to-run noise, use aot-evals.

Expected impact

Quality: improves, medium: GitHub +2-5 points on SWE-bench with 40 to 13 tools (Nov 2025); Anthropic tool search 49% to 74% (Opus 4); one irrelevant tool cost 34.7 points on questions answerable without tools (Sep 2026)
Cost: improves, large: definitions -85% (Anthropic); -46.9% tokens (Cursor A/B); deferral alone raises call count
Performance: mixed, medium: consolidation faster (GitHub -400 ms); deferral about 50% slower with 2-3x calls (Speakeasy)

Evidence

Evidence comes from: Tool use, Code generation.

Sync MCP tool descriptions to per-server folders

"In an A/B test, we found that in runs that called an MCP tool, this strategy reduced total agent tokens by 46.9% (statistically significant, with high variance based on the number of MCPs installed)." No n, model or quality metric given.

Jediah Katz (Cursor), "Dynamic context discovery" · Jan 2026 · 4. Efficiently loading only the MCP tools needed; footnote 1
Expose MCP tools as a code-API filesystem

"This lets the agent load only the definitions it needs for the current task. This reduces the token usage from 150,000 tokens to 2,000 tokens, a time and cost saving of 98.7%." (single Google Drive to Salesforce example; model not stated.) Cloudflare reports similar findings ("Code Mode").

Adam Jones, Conor Kelly (Anthropic), "Code execution with MCP: Building more efficient agents" · Nov 2025 · Code execution with MCP improves context efficiency
Add a search_tools tool with detail levelsNot measured

"Including a detail level parameter in the search_tools tool that allows the agent to select the level of detail required ... also helps the agent conserve context and find tools efficiently."

Adam Jones, Conor Kelly (Anthropic), "Code execution with MCP: Building more efficient agents" · Nov 2025 · Benefits: Progressive disclosure
Select a per-task tool loadout via retrieval

RAG-MCP (Gan & Sun), DeepSeek-v3: selection "becomes critical when you have more than 30 tools"; "Beyond 100 tools, the model was virtually guaranteed to fail"; RAG-selecting under 30 tools gave "as much as 3x better tool selection accuracy" and shorter prompts. "Less is More": Llama 3.1 8b fails with 46 tools, succeeds with 19; dynamic tool selection improved Llama 3.1 8b by 44% on BFCL; where accuracy did not improve, it saved 18% power and 77% time.

Drew Breunig, "How Long Contexts Fail / How to Fix Your Context", https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-how-to-fix-them.html ; https://www.dbreunig.com/2025/06/26/how-to-fix-your-context.html (2025-06-26) · Jun 2025Older evidence · 06-26 post, "Tool Loadout"
Retrieve top-k tools instead of listing all

TinyAgent (DeBERTa-v3 small multi-label selector, >50% threshold) 'reduces nearly half of the prompt size'. The other methods give no numbers. Trade-off: aggressive selection 'may filter out necessary tools'.

Xiaofang Yang, Lijun Li, Heng Zhou, Tong Zhu, Xiaoye Qu, Yuchen Fan, et al. (Shanghai AI Lab et al.), "Toward Efficient Agents: A Survey of Memory, Tool Use, and Planning" · Jan 2026 · §4.1 Tool Selection (External Retriever, MLC); §4.4
Defer tool loading behind a tool-search tool

"Claude Code's shouldDefer flag + ToolSearchTool reduces the initial prompt by 40%" (43 tools; no further conditions given); Hermes collapses catalogs "once schemas would exceed 10% of the context window"; "Below 15 tools the indirection is not worth the added complexity."

Barbaste et al., "Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems" · Jul 2026 · Sec 8.3; Recommendation 4; Table 11 "Deferred Loading"; Appendix table (Claude Code row)
Front a large MCP catalog with a code-mode execute toolNot measured

OpenCode "treats its experimental code mode as MCP-catalog deferral: instead of exposing every MCP tool, one execute tool runs model-written scripts against the catalog."

Barbaste et al., "Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems" · Jul 2026 · Sec 8.3
Index tool definitions and retrieve them on demandNot measured

"some agents index tool definitions and retrieve tools on demand, potentially using a search tool to encapsulate this process."

Lance Martin, "Agent Design Patterns" · Jan 2026 · Progressive Disclosure
List CLI utilities in instructions; agent reads --helpNot measured

Manus "just provides a list of available utilities in the agent instructions. Then agent uses its bash tool to call the --help flag ... to learn about any one of them if needed."

Lance Martin, "Agent Design Patterns" · Jan 2026 · Progressive Disclosure
Package skills as SKILL.md folders, frontmatter onlyNot measured

"The YAML frontmatter of each file is loaded into agent instructions, letting the agent decide to read SKILL.md only if needed."

Lance Martin, "Agent Design Patterns" · Jan 2026 · Progressive Disclosure
Give a bash-only interface rather than typed tools for enterprise digital-worker tasks

Opus-4.8 and GPT-5.5 on TheAgentCompany and APEX-Agents, five interfaces compared: bash-only beats typed tools by 21.8-24.5 pp on TheAgentCompany and 4.8-7.4 pp on APEX-Agents, with 19-72% fewer tokens. Adding typed or agent-synthesized tools on top of bash gives no measurable gain; programmatic tool calling saves tokens but scores below bash.

Mak et al., "Is Bash All You Need? An Empirical Study of Tool Interfaces for Enterprise Digital Worker Agents" · Sep 2026
Show an adaptive shortlist of tools rather than a fixed top-k

Registries of 20-3,251 tools, chance-corrected coverage metric, RL-trained adaptive-depth selector: BFCL (370 tools) 90.3% coverage while showing about 7 tools, against 90.8% at fixed depth; hard ToolBench queries (3,251 tools) 16.7% where fixed top-5 gets 0. With Claude Sonnet 4.6 choosing: 93.1% vs 87.1% selection accuracy, 76.8% vs 60.9% on medium queries.

Repantis et al., "How Many Tools Should an LLM Agent See? A Chance-Corrected Answer" · May 2026
Do not offer tools the task does not need

500 query pairs, 10 domains, 6 LLMs, every query answerable from the model's own knowledge: accuracy falls from 98.2% to 63.5% once an irrelevant tool is offered, whether or not the model calls it.

Paturi et al., "When Tools Get in the Way: The Effect of Unnecessary Tool Availability on LLM Answering" · Sep 2026
Fix description defects: purpose, functionality and accuracy

18 description smells audited over 10,831 MCP servers; 73% have repeated tool names. Controlled mutations: functionality and accuracy defects shift selection most (+11.6%, +8.8%, p<0.001). Among functionally equivalent servers, a standard-compliant description is picked 72% of the time against a 20% baseline.

Wang et al., "From Docs to Descriptions: Smell-Aware Evaluation of MCP Server Descriptions" · Feb 2026
Consolidate tools whose scopes genuinely overlap

Same study: where two skills genuinely overlap in scope, rewording cannot separate them; the paper gives a signal for detecting such pairs, which need consolidation instead.

Zhou et al., "A Single Rewrite Suffices: Empirical Lessons from Production Skill Description Optimization" · Jun 2026
Do not assume pruning the input fixes wrong-tool selectionMixed

3B-32B open models: attention lands most on the correct tool 80% of the time (21% by chance), so failures happen at the decision step. Input fixes recover at most 23% of failures; decision-step interventions 59-91%. Training-free attention-based selector +11.9 pts function-name accuracy on BFCL. Single-turn only.

Shiyang Chen, "Looking Is Not Picking: An Attention-Segment Account of Tool-Selection Failures in LLM Agents" · Jun 2026
Retrieve tools from an intention and co-usage graph

7,471 tools, three benchmarks: up to +59.8% Global Recall@5 and +28.9% downstream success over retrieval baselines, exposing 99.8% less tool schema.

Xiao et al., "SING: Synthetic Intention Graph for Scalable Active Tool Discovery in LLM Agents" · Jun 2026
Filter tools per step by precondition and effect

102 tasks, 100 tools, 4 model backends, 2,448 runs: matches stronger causal baselines while showing about 1 tool per step instead of 100, ~90% fewer tokens than loading all tools.

Suresh Babu and Ganesh Iyer, "ToolChoiceConfusion: Causal Minimal Tool Filtering for Reliable LLM Agents" · Jun 2026
Front a large tool catalog with hybrid retrieval

Production MCP gateway over 3,000+ tools: 98% top-15 recall, 8.9x faster tool selection and 23.8x fewer tokens than mounting all tools. Systems metrics, not task success.

Li et al., "Scalable LLM Agent Tool Access in the Cloud" · Jul 2026
Expect performance to fall as more tools are available

300+ tools across 7 stateful sandboxes: best models under 60% against 90% for humans; names tool retrieval saturation (performance falls as tools grow), over-confidence and strategic defeatism as the main failure modes. No count-vs-success curve in the abstract.

Li et al., "ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox" · May 2026
Measure selection with plausible distractor toolsMixed

1,000 tasks, 36 real servers, 220 tools, 6-37 offered per task (mean 15.2) with plausible distractors, 20 models: best pass rate 82.2%; 63.3% of diagnosed failures are reasoning failures, not tool-call failures.

Bandi et al. (Scale AI), "MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers" · Jan 2026
Cut the default toolset and group the rest behind expandable virtual tools

Default built-in tools cut from about 40 to 13; the rest in 4 expandable virtual-tool groups, grouped by embedding similarity. Tool-use coverage 94.5% (embeddings) vs 87.5% (LLM grouping) vs 69% (static list). SWE-Lancer and SWE-bench Verified +2-5 points with GPT-5 and Sonnet 4.5; average response time -400 ms, time to first token -190 ms. An oversized toolset made the agent ignore instructions and call unneeded tools.

Anisha Agarwal and Connor Peet (GitHub), "How we're making GitHub Copilot smarter with fewer tools" · Nov 2025
Defer tool loading behind tool search

Tool Search with defer_loading: tool-definition tokens ~77k to 8.7k (-85%); internal MCP evals 49% to 74% (Opus 4) and 79.5% to 88.1% (Opus 4.5). Recommended when definitions exceed 10k tokens or there are 10+ tools. Internal evals only.

Bin Wu et al. (Anthropic), "Introducing advanced tool use on the Claude Developer Platform" · Nov 2025
Consolidate an MCP server's tools around jobs, and regression-test the new set

MCP server cut from 100+ tools to about 40, grouped around jobs; paired regression over 80+ scenarios: task completion matched or improved in every paired suite. No per-suite numbers.

Eric Kim (Amplitude), "Bring headless Amplitude where you work" · Sep 2026
Keep a core toolset loaded and defer the rest behind server-side search

Deferred tools shown by name and description only, a curated always-loaded core, embedding tool search for Anthropic models: server-side tool search cut total tokens about 11% per turn; user error rate -4.01% with Sonnet 4.6.

Ryan Caldwell and Bhavya U (Microsoft), "Improving token efficiency for GitHub Copilot in VS Code" · Jun 2026
Group deferred tools into small namespaces with use-case descriptionsNot measured

Vendor guidance, not measured: namespaces under 10 functions with short use-case descriptions; defer when the catalog is large and each task needs few tools, load eagerly when small; mixing eager and deferred generally not recommended. Living doc; date is the access date.

OpenAI, "Tool search (API guide)" · Sep 2026
Replace the catalog with search, describe and execute meta-toolsMixed

Sonnet 4.5 over 40-400 tools, vendor benchmark: 100% success; input tokens -96.7% (simple) and -91.2% (complex). Cost: 2-3x more tool calls (6-8 vs 3) and about 50% slower.

Chase Crumbaugh (Speakeasy), "Reducing MCP token usage by 100x — you don't need code mode" · Nov 2025
Merge related tools into read and write tools with a method parameter

Consolidated many tools into read/write tools with a method parameter (issue_read, issue_write); smaller default toolset; server instructions. Projects consolidation saved about 23k tokens (~50%). Tools the user's token scopes cannot use are hidden. Tokens only, no accuracy data.

GitHub, "GitHub MCP Server changelogs (Oct 2025, Jan 2026)" · Jan 2026
Expose a large API through search and execute in codeNot measured

2,500+ API endpoints behind search() and execute(); the model writes code against the typed spec in a sandbox. About 1k tokens against 1.17M, calculated not tested.

Matt Carey (Cloudflare), "Code Mode: give agents an entire API in 1,000 tokens" · Feb 2026
Choose the tool-search method by how many results the agent sees

270 tools, 2,700 queries: top-1 retrieval BM25 14%, hybrid 21%, embeddings 38%; top-5 85-90% for all three. Whether a tool is found depends on distinctive terms in its description. Retrieval only, not task success.

Guillaume Lebedel (StackOne), "Comparing BM25, TF-IDF, and Hybrid Search for MCP Tool Discovery" · Feb 2026
Trace tool listing and calls, linked to the model call that chose themNot measured

Traces tools/list and call_tool, each linked to the LLM call that chose the tool; per-tool and per-server error rate, retries and p95 latency; large registries add latency. No wrong-tool or unused-tool metric. Not measured.

Lily Chen et al. (Datadog), "Gain end-to-end visibility into MCP clients with Datadog Agent Observability" · Dec 2025
Grade outcomes, not exact tool-call sequencesMixedNot measured

Warns against grading exact tool-call sequences as too rigid and brittle; grade the outcome and read transcripts; production monitoring is the ground truth after launch. Not about tool counts.

Grace et al. (Anthropic), "Demystifying evals for AI agents" · Jan 2026

Also known as

Consolidate tools into higher-level operations
Progressive tool discovery

In Claude Code

This optimization is a skill in aot-kit, our open-source Claude Code plugin: /aot-kit:reduce-tool-surface. Its library entry holds every source. Have new evidence, or a correction? Suggest a change.

Reduce the tool surface: remove and consolidate tools, then defer the rest behind search