Mahal Systems RestBench
Fits agents that call real REST services from OpenAPI docs. Borrow the small hand-annotated gold-path format and the correct-path-rate metric alongside success.
What it asks the agent to do
- answer a movie question by chaining real TMDB REST calls
- control a music player through Spotify REST calls
- parse a long OpenAPI response and pick the field needed next
What to borrow from it
case templatebaseline
Caveats
Needs live TMDB and Spotify accounts, success was judged by hand, only 157 instructions, and the repo has been idle since 2024.
Details
| Workloads | tool calling, general assistant, knowledge qa |
|---|---|
| Interaction | single turn |
| Environment | api sandbox |
| End state checked | No |
| Tools | 54 TMDB APIs and 40 Spotify APIs with OpenAPI specs |
| Grading | human, trajectory check |
| Size | 157 |
| Data | mixed |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | frozen |