Mahal Systems World of Workflows (WoW-bench)
Fits agents that write to systems with hidden business rules (ITSM, ERP). Borrow the table audit idea: grade on database state changes, including cascading ones the agent never saw.
What it asks the agent to do
- complete a ServiceNow task without violating business rules
- predict which tables change after an action (side effects)
- predict the next tool call in a workflow
- respect data-level constraints when editing records
What to borrow from it
grading designsandbox designtaxonomy
Caveats
Needs a ServiceNow developer instance plus an OpenRouter key; small team release with few published baselines.
Details
| Workloads | it ops, workflow automation, planning |
|---|---|
| Interaction | single turn |
| Environment | api sandbox |
| End state checked | Yes |
| Tools | ServiceNow table and workflow APIs |
| Grading | state diff, programmatic check |
| Size | 234 |
| Data | synthetic |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | active |