Mahal Systems GDPval
Fits document-producing assistants in knowledge-work roles. Borrow the occupation-by-task taxonomy (44 occupations, 9 sectors) and the weighted rubrics written by professionals.
What it asks the agent to do
- produce a professional deliverable (report, brief, plan, deck) from a brief and reference files
- build or fix a spreadsheet model
- draft legal, nursing, engineering or sales documents
- review and summarize supplied documents
What to borrow from it
grading designcase templatetaxonomy
Caveats
The main metric is human pairwise judgment; only the 220-task gold set is public (1,320 total); no tool use or environment, so it does not test acting in systems.
Details
| Workloads | documents, spreadsheets, finance, legal, general assistant |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | none required; file inputs and outputs |
| Grading | human, llm judge |
| Size | 220 |
| Data | real public |
| Personal data | none |
| License | unknown |
| Harness | inspect_evals |
| Status | active |