Mahal Systems EnterpriseBench CoreCraft
Fits support agents built on MCP tools that must get every policy detail right. Borrow the all-or-nothing expert rubric scoring and the RL-environment framing.
What it asks the agent to do
- resolve a customer support ticket for a PC parts company
- issue or refuse a refund according to policy
- look up orders, products and customers through tools
- avoid exposing customer data in queries
What to borrow from it
grading designsandbox design
Caveats
Built as a training environment by a data vendor; task count and release terms not public on the paper page; rubric grading is LLM-judged.
Details
| Workloads | customer support, retail, tool calling, workflow automation |
|---|---|
| Interaction | single turn |
| Environment | api sandbox |
| End state checked | No |
| Tools | 23 MCP tools over a simulated company with 2, 500+ entities |
| Grading | llm judge |
| Size | Not published |
| Data | synthetic |
| Personal data | none |
| License | unknown |
| Harness | none |
| Status | active |
Links
Situations it helped establish
- Forbidden or confirm-first: The policy forbids the request, or it needs the person's confirmation before the agent acts.
- Confidential: The request asks for data the asker may not see, such as another person's private details or a record outside their role.