Mahal Systems SpreadsheetBench
Fits spreadsheet or Excel assistants that edit workbooks from a user request. Borrow the online-judge pattern: about three input workbooks per instruction, and the output cells must match on every one.
What it asks the agent to do
- fill or compute cells as a forum user asks, with a formula or a script
- transform a sheet that has several tables or a non-standard layout
- make the solution hold when cell values change across test workbooks
What to borrow from it
grading designcase templatesandbox design
Caveats
Questions come from public Excel forums, so sheets may carry user data; formulas must be recalculated through LibreOffice or Excel before checking; a smaller Verified subset was released later, so compare numbers carefully.
Details
| Workloads | spreadsheets, data analysis, coding |
|---|---|
| Interaction | single turn |
| Environment | containers |
| End state checked | Yes |
| Tools | python with openpyxl, spreadsheet files |
| Grading | state diff, programmatic check |
| Size | 912 |
| Data | real scraped |
| Personal data | possible |
| License | CC-BY-SA-4.0 |
| Harness | own repo |
| Status | active |