Mahal Systems MetaTool
Fits any agent with a large tool catalog where the first failure is picking the wrong tool. Its four selection subtasks (similar, scenario-specific, reliability, multi-tool) are a ready taxonomy for tool-routing test cases.
What it asks the agent to do
- decide whether a request needs a tool at all
- choose one tool among several with similar descriptions
- choose tools for a request that needs more than one
- avoid an unreliable tool when a safer one exists
What to borrow from it
taxonomycase template
Caveats
Selection only, no execution and no conversation; queries are ChatGPT-generated and the repo has been idle since early 2024.
Details
| Workloads | general assistant, tool calling |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | 195 tool descriptions derived from the OpenAI plugin list |
| Grading | exact match |
| Size | 21,127 |
| Data | synthetic |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | frozen |