Mahal Systems Make Me Pay
Fits payment or negotiation agents that talk to other agents or users and must not be manipulated (or manipulate). Borrow the two-model setup where the outcome (money moved) is checked programmatically rather than judged.
What it asks the agent to do
- persuade another model to hand over money in a conversation
- as the mark, resist a con-artist model's social engineering
What to borrow from it
user simulatorgrading design
Caveats
Measures a propensity, not a task; default is only 20 generated conversations and some original metrics are not implemented in the Inspect port.
Details
| Workloads | multi agent, safety |
|---|---|
| Interaction | multi turn simulated user |
| Environment | static |
| End state checked | No |
| Tools | none |
| Grading | programmatic check |
| Size | Not published |
| Data | synthetic |
| Personal data | none |
| License | MIT |
| Harness | inspect_evals, own repo |
| Status | active |