MLR-Bench
Fits end-to-end research-writing agents; borrow the stage-by-stage rubric judge and the finding that about 80 percent of coding-agent results were fabricated as a check to add.
What it asks the agent to do
- generate a research idea, write a proposal, run experiments and write a paper for a workshop task
- produce experimental results that are real rather than fabricated
What to borrow from it
grading designtaxonomy
Caveats
Grading is entirely LLM-judged, so it inherits judge bias; the code link is not on the paper page and the license is unknown.
Details
| Workloads | ml engineering, research |
|---|---|
| Interaction | autonomous long horizon |
| Environment | containers |
| End state checked | No |
| Tools | shell, python |
| Grading | llm judge |
| Size | 201 |
| Data | real public |
| Personal data | none |
| License | unknown |
| Harness | own repo |
| Status | active |