Mahal Systems SCUBA
Fits CRM agents working in Salesforce. Borrow the persona-based task sourcing from user interviews and milestone scoring against org snapshots.
What it asks the agent to do
- configure objects, fields, flows or permissions as a Salesforce admin
- update leads, opportunities and accounts as a sales rep
- handle cases and knowledge as a service agent
- find information or troubleshoot inside the org
What to borrow from it
case templategrading designsandbox design
Caveats
Needs a Salesforce developer org and OAuth login; single product; zero-shot top score 39 percent, about 50 with demonstrations.
Details
| Workloads | crm, sales, customer support, computer use, web browsing |
|---|---|
| Interaction | single turn |
| Environment | browser |
| End state checked | Yes |
| Tools | Salesforce developer org via browser (Playwright) or OSWorld style desktop |
| Grading | state diff, trajectory check |
| Size | 300 |
| Data | synthetic |
| Personal data | none |
| License | Apache-2.0 |
| Harness | own repo |
| Status | active |
Links
Situations it helped establish
- Different asker: The same job asked by a different role, such as a manager wanting an aggregate where a rep wants one record.