Mahal Systems TravelPlanner
Fits planning assistants that must juggle many constraints. Borrow the split between commonsense and hard constraint pass rates plus a final all-constraints pass rate.
What it asks the agent to do
- produce a full multi-day itinerary from a request
- look up flights, hotels, restaurants and attractions with tools
- satisfy hard constraints (budget, cuisine, room rules) and commonsense ones
What to borrow from it
grading designcase templatebaseline
Caveats
Static offline data; original pass rates were near zero so numbers are hard to compare across versions; no interaction with a user.
Details
| Workloads | travel, planning, tool calling |
|---|---|
| Interaction | single turn |
| Environment | api sandbox |
| End state checked | No |
| Tools | search tools over a static database of about four million records |
| Grading | programmatic check |
| Size | 1,225 |
| Data | mixed |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | active |