Mahal Systems DialogTool
Fits phone-assistant agents where API state (an open cart, a running timer) persists across turns. Its three-stage life cycle (create, use, respond) is a useful way to split test cases.
What it asks the agent to do
- recognize that the current turn needs a tool
- select and execute the right app API with state carried from earlier turns
- create a new tool when none exists
- reply in a consistent assistant persona after the call
What to borrow from it
taxonomysandbox design
Caveats
No public code or data could be found as of this entry, the paper's app and API counts are inconsistent, and response quality is scored with BLEU, ROUGE and a GPT-4o judge.
Details
| Workloads | mobile, general assistant, tool calling |
|---|---|
| Interaction | multi turn scripted |
| Environment | api sandbox |
| End state checked | No |
| Tools | about 45 APIs across 20 mobile apps in the VirtualMobile simulator |
| Grading | exact match, unit tests, llm judge |
| Size | 900 |
| Data | synthetic |
| Personal data | none |
| License | unknown |
| Harness | none |
| Status | active |