Mahal Systems WorkBench
Fits assistants that act on email, calendar and CRM data through tools. Borrow the outcome-centric grading that compares final database state regardless of action order and reports side effects separately.
What it asks the agent to do
- send an email or schedule a meeting
- update a customer record
- pull analytics numbers and report them
- complete a project-management action
- refuse or ask when a request is impossible in the sandbox
What to borrow from it
grading designcase templatesandbox design
Caveats
Small sandbox (a real inbox is far larger); single-turn only; 2024 baselines are stale.
Details
| Workloads | email calendar, crm, data analysis, tool calling, workflow automation |
|---|---|
| Interaction | single turn |
| Environment | database |
| End state checked | Yes |
| Tools | 26 read/write tools over five sandbox databases |
| Grading | state diff |
| Size | 690 |
| Data | synthetic |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | active |