Mahal Systems TaskBench
Fits workflow-automation agents that must plan a tool chain before executing. Borrow the node-F1 and edge-F1 scoring of predicted tool graphs and the back-instruct data generator.
What it asks the agent to do
- decompose a user request into steps
- pick the right tools and order them (chain or graph)
- fill in tool parameters correctly
What to borrow from it
case templategrading designtaxonomy
Caveats
Tools are never executed, so it scores plans not outcomes; fully synthetic instructions; no updates since 2024.
Details
| Workloads | tool calling, planning, workflow automation |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | tool graphs over Hugging Face models, multimedia tools and daily life APIs |
| Grading | programmatic check |
| Size | 28,271 |
| Data | synthetic |
| Personal data | none |
| License | Apache-2.0 |
| Harness | own repo |
| Status | frozen |