Mahal Systems AgentAtlas
Fits any team deciding what to measure beyond pass/fail: the six control decisions and nine failure categories are a ready vocabulary for labeling an agent's trajectories.
What it asks the agent to do
- classify each agent step as act, ask, refuse, stop, confirm or recover
- label a failed trajectory by error source and downstream impact
- audit a benchmark for which behaviors it actually measures
What to borrow from it
taxonomy
Caveats
A taxonomy and audit paper, not a runnable benchmark; the 1,342-item synthetic study set is explicitly not released, and no code repository was found.
Details
| Workloads | general assistant, safety, planning |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | none |
| Grading | trajectory check |
| Size | Not published |
| Data | synthetic |
| Personal data | none |
| License | unknown |
| Harness | none |
| Status | active |