Mahal Systems DiscoveryBench
Fits research agents that must turn a dataset and a goal into a written finding. Borrow the faceted judge: split gold and predicted hypotheses into context, variables and relationship and score each part separately.
What it asks the agent to do
- form a hypothesis from one or more datasets given a discovery goal
- run the statistical analysis that tests it
- state the finding with its context, variables and relationship
What to borrow from it
grading designtaxonomycase template
Caveats
Scoring depends on a GPT-4 judge; the best system in the paper scored about 25 percent, so numbers are noisy; the extra 903 synthetic tasks are for controlled tests, not realism.
Details
| Workloads | science, data analysis, research |
|---|---|
| Interaction | single turn |
| Environment | terminal |
| End state checked | No |
| Tools | python code execution |
| Grading | llm judge |
| Size | 264 |
| Data | mixed |
| Personal data | none |
| License | ODC-By-1.0 |
| Harness | own repo |
| Status | active |