Mahal Systems ClawsBench
Fits productivity assistants wired to Workspace and Slack. Borrow the mock services with snapshot/restore and the unsafe-action-rate metric reported alongside success.
What it asks the agent to do
- triage and reply to email
- schedule or move calendar events
- edit a Google Doc or move files in Drive
- post or read messages in Slack
- refuse or stop before an unsafe action (e.g. changing a contract silently)
What to borrow from it
sandbox designgrading designtaxonomy
Caveats
Only 44 tasks and the task definitions were still being released in batches; non-commercial license; grading method is thinly documented.
Details
| Workloads | email calendar, documents, workflow automation, safety, general assistant |
|---|---|
| Interaction | single turn |
| Environment | api sandbox |
| End state checked | Yes |
| Tools | mock Gmail, Google Calendar, Docs, Drive and Slack APIs |
| Grading | programmatic check, trajectory check |
| Size | 44 |
| Data | synthetic |
| Personal data | none |
| License | CC-BY-NC-SA-4.0 |
| Harness | own repo |
| Status | active |