Mahal Systems DoomArena
Fits teams that already run tau-bench or a browser benchmark and want to add threat models without rewriting tasks. Borrow the separation of attack, attackable component and success criterion from the host environment.
What it asks the agent to do
- run tau-bench tool tasks against a malicious simulated user
- run BrowserGym web tasks with attacker content injected into pages
- combine several attacks in one episode
- keep task success while the environment or user is adversarial
What to borrow from it
sandbox designgrading design
Caveats
A framework, not a fixed task set, so numbers are not comparable across papers; attack success is defined per host environment; last repo activity was late 2025.
Details
| Workloads | safety, security, web browsing, tool calling, customer support |
|---|---|
| Interaction | multi turn simulated user |
| Environment | mixed |
| End state checked | Yes |
| Tools | plug in attacks for tau bench, BrowserGym, OSWorld and TapeAgents |
| Grading | state diff, programmatic check |
| Size | Not published |
| Data | synthetic |
| Personal data | none |
| License | Apache-2.0 |
| Harness | own repo |
| Status | active |