Mahal Systems InfiAgent-DABench
Fits a data analysis assistant that answers questions over uploaded CSV files with code. Borrow the trick of rewriting open-ended questions into closed-form answers so grading is an exact match.
What it asks the agent to do
- answer a question about a CSV file by writing and running Python
- compute summary statistics, correlations or filtered counts from a table
- return the result in a fixed closed-form answer format
What to borrow from it
grading designcase templatebaseline
Caveats
Only the validation split is public; grading first reformats the agent's reply with a GPT-3.5 call, and the data license is non-commercial. Small and from early 2024.
Details
| Workloads | data analysis, coding |
|---|---|
| Interaction | single turn |
| Environment | terminal |
| End state checked | No |
| Tools | python code execution |
| Grading | exact match |
| Size | 257 |
| Data | real public |
| Personal data | none |
| License | CC-BY-NC-4.0 |
| Harness | own repo |
| Status | active |