Mahal Systems MT-AgentRisk
Fits agents that take many turns with MCP tools. Borrow the recipe that turns a single-turn harmful task into an addition or decomposition attack sequence, and the complete/rejected/ failed judge labels.
What it asks the agent to do
- follow a conversation where a harmful goal is split across several turns
- operate a browser, terminal, file system, database or Notion via MCP tools
- refuse once the combined intent becomes harmful
- avoid destructive changes to data files or environment state
What to borrow from it
case templatetaxonomy
Caveats
Attack turns are pre-written, not adaptive; tasks are recycled from OpenAgentSafety, SafeArena, P2SQL and MCPMark; success is judged by GPT-4.1; dataset license is not stated separately from the repo.
Details
| Workloads | safety, tool calling, web browsing, sql, coding |
|---|---|
| Interaction | multi turn scripted |
| Environment | containers |
| End state checked | No |
| Tools | Playwright browser, terminal, filesystem, PostgreSQL and Notion MCP servers run under OpenHands |
| Grading | llm judge |
| Size | 365 |
| Data | synthetic |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | active |