Mahal Systems

Overview › Benchmark catalog

SheetCopilot

Fits agents that drive a spreadsheet application through a fixed action set rather than writing code. Borrow the per-task checking file that names which sheet properties to compare against one or more reference workbooks.

What it asks the agent to do

  • format, sort or fill cells from a plain-language request
  • build a chart or pivot table in a workbook
  • carry out a multi-step spreadsheet task through atomic actions

What to borrow from it

grading designsandbox designcase template

Caveats

Tasks were expanded from 67 seed tasks over 28 workbooks, so coverage is narrow; the original evaluator needs Windows and Excel; the README restricts use to non-commercial purposes; results shown are from the GPT-4 era.

Details

Workloadsspreadsheets, workflow automation
Interactionsingle turn
Environmentdesktop
End state checkedYes
Toolsatomic spreadsheet actions over Excel or LibreOffice
Gradingstate diff
Size221
Datasynthetic
Personal datanone
LicenseGPL-3.0-only
Harnessown repo
Statusfrozen

Links