Mahal Systems Seal-Tools
Fits teams who need a large, fully made-up tool catalogue for fine-tuning or retrieval tests; borrow the self-instruct pipeline (field, tool, instance) and its format accuracy plus tool and parameter precision/recall/F1 scoring.
What it asks the agent to do
- select one tool out of a large self-generated catalogue and fill its parameters
- call several tools for one instruction
- nest calls so one tool's output becomes another's input
What to borrow from it
case templateprivacy pattern
Caveats
Everything was generated by ChatGPT, nothing executes, test split size is not stated (14,076 instances in total, 586 nested), and the README calls the evaluation code "not well organized".
Details
| Workloads | tool calling |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | 4,076 self instruct generated API like tools, DPR retriever |
| Grading | exact match, programmatic check |
| Size | Not published |
| Data | synthetic |
| Personal data | none |
| License | Apache-2.0 |
| Harness | own repo |
| Status | frozen |