Mahal Systems API-BLEND
Fits teams fine-tuning a model for API calling in travel or assistant domains. Borrow the three metrics (API-F1, parameter-F1, LCS-F1 for order) and the method of converting existing dialogue corpora into API-call cases.
What it asks the agent to do
- detect which APIs a request needs
- fill the slots of each API from the user's words
- order a sequence of API calls correctly
- generalize to API sets not seen in training
What to borrow from it
case templategrading designprivacy pattern
Caveats
Mostly a training corpus; the curated sets are not redistributed because of license clearance, so you must rebuild them from the raw sources with the provided scripts.
Details
| Workloads | tool calling, travel, general assistant |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | API sets converted from ATIS, SNIPS, SGD, MultiWOZ, TopV2 and ToolQA plus out of distribution sets |
| Grading | programmatic check |
| Size | 11,618 |
| Data | real public |
| Personal data | none |
| License | Apache-2.0 |
| Harness | own repo |
| Status | active |