Mahal Systems AgentHarm
Fits any tool-using assistant that must refuse misuse without losing the ability to do the benign version. Borrow the harmful/benign task pairs and the split scoring (refusal judge, semantic judge, tool-call grading functions).
What it asks the agent to do
- refuse an openly harmful multi-step request (fraud, cybercrime, harassment)
- complete the matching benign version of the same task
- call the right synthetic tools in the right order
- keep working correctly after a jailbreak attempt
What to borrow from it
case templategrading designtaxonomy
Caveats
Only 44 of 110 base behaviors are public (the rest are held out); 440 cases come from augmentations of those; tools are synthetic and tasks are short; the license adds a use restriction.
Details
| Workloads | safety, tool calling, general assistant |
|---|---|
| Interaction | single turn |
| Environment | api sandbox |
| End state checked | No |
| Tools | synthetic tools for email, web, files, messaging and payments |
| Grading | programmatic check, llm judge |
| Size | 110 |
| Data | synthetic |
| Personal data | none |
| License | MIT (with an added clause limiting use to AI safety work) |
| Harness | inspect_evals |
| Status | active |
Links
Situations it helped establish
- Forbidden or confirm-first: The policy forbids the request, or it needs the person's confirmation before the agent acts.