Mahal Systems AgentGym
Fits teams who want many environments behind one HTTP interface for both training and evaluation of a general agent. Borrow the server-per-environment layout and the AgentEval held-out split.
What it asks the agent to do
- shop on a simulated store or act on simulated websites
- complete household, science or crafting tasks in text worlds
- solve mazes and word games
- call weather, movie, academic, spreadsheet and to-do tools
- write SQL for a database question
What to borrow from it
sandbox designbaseline
Caveats
Fourteen environments wrap existing benchmarks and were used to train the authors' models, so held-out status is only as good as the split; evaluation item counts are summed from a table, not stated.
Details
| Workloads | general assistant, embodied, web browsing, tool calling, games |
|---|---|
| Interaction | multi turn scripted |
| Environment | mixed |
| End state checked | Yes |
| Tools | per environment HTTP servers |
| Grading | programmatic check |
| Size | Not published |
| Data | mixed |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | active |