Mahal Systems APEX-Agents
Fits agents doing analyst work in files and office tools. Borrow the "world" seeds (31 workspaces, 5,182 files) and the binary-criteria rubrics judged per criterion.
What it asks the agent to do
- complete an investment banking analysis across spreadsheets and documents
- draft or review corporate legal documents from a file set
- build a consulting deliverable from mail, chat and files
- find the relevant files in a messy simulated workspace
What to borrow from it
case templategrading designsandbox design
Caveats
Evaluation-only terms (no training); judge-graded; task count dropped from 480 to 240 between versions, so compare like with like.
Details
| Workloads | finance, legal, documents, spreadsheets, general assistant |
|---|---|
| Interaction | single turn |
| Environment | mixed |
| End state checked | Yes |
| Tools | calendar, chat, code execution, documents, file system, mail, PDFs, spreadsheets, presentations |
| Grading | llm judge |
| Size | 240 |
| Data | synthetic |
| Personal data | none |
| License | CC-BY-4.0 |
| Harness | own repo |
| Status | active |