Mahal Systems MCPTox
Fits agents that load tools from external MCP registries. Borrow the three poisoning templates and the 10 risk categories to build a tool-description screening test.
What it asks the agent to do
- answer a benign request given a tool set from a real MCP server
- ignore malicious instructions hidden in a tool's description
- avoid calling a legitimate tool for a harmful purpose
- refuse or flag a poisoned tool
What to borrow from it
case templatetaxonomy
Caveats
The official repo README is one line and has no license; case count differs between paper versions (1312 vs 1348); success is judged by a model; the inspect_evals version is an external, pinned third-party port.
Details
| Workloads | safety, security, tool calling |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | tool descriptions from 45 real MCP servers (353 tools); the poisoned tool is never run |
| Grading | llm judge |
| Size | 1,348 |
| Data | mixed |
| Personal data | none |
| License | unknown |
| Harness | inspect_evals, own repo |
| Status | active |