Mahal Systems ToolDial
Fits assistants that must ask for missing information before calling an API. The action vocabulary (request, clarify, fail inform) and the API-graph trick for generating dependent calls are worth copying into a case generator.
What it asks the agent to do
- ask a clarifying question when an API parameter is missing
- chain a second API call to obtain a required input
- track the dialogue state (API, parameters, values) each turn
- choose the next system action from 16 dialogue acts
What to borrow from it
case templatetaxonomyuser simulator
Caveats
Dialogues are fully GPT-4o generated and scripted, so the model is graded against one expected path; 11,111 dialogues in total but only 1,166 are test.
Details
| Workloads | general assistant, tool calling, knowledge qa |
|---|---|
| Interaction | multi turn scripted |
| Environment | static |
| End state checked | No |
| Tools | 473 RapidAPI APIs linked by an input output compatibility graph |
| Grading | exact match, llm judge |
| Size | 1,166 |
| Data | synthetic |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | active |