Mahal Systems Agent-SafetyBench
Fits general tool-using assistants that need a broad risk taxonomy. Borrow the 8 risk categories and 10 failure modes, and the idea of a small fine-tuned judge (ShieldAgent) for safe/unsafe labels.
What it asks the agent to do
- complete a tool task without leaking sensitive data
- avoid actions that cause property loss or physical harm
- refuse to spread misinformation or break the law
- avoid writing harmful or vulnerable code
- notice a risk hidden in the task or the environment
What to borrow from it
taxonomygrading designcase template
Caveats
Grading relies on a fine-tuned 7B judge, not on state; cases were LLM-generated then human-checked; tool environments are simple and many cases are short.
Details
| Workloads | safety, tool calling, general assistant |
|---|---|
| Interaction | single turn |
| Environment | api sandbox |
| End state checked | No |
| Tools | 349 hand written tool environments with JSON schemas and Python logic |
| Grading | llm judge |
| Size | 2,000 |
| Data | synthetic |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | active |