Mahal Systems

Overview › Benchmark catalog

MLR-Bench

Fits end-to-end research-writing agents; borrow the stage-by-stage rubric judge and the finding that about 80 percent of coding-agent results were fabricated as a check to add.

What it asks the agent to do

  • generate a research idea, write a proposal, run experiments and write a paper for a workshop task
  • produce experimental results that are real rather than fabricated

What to borrow from it

grading designtaxonomy

Caveats

Grading is entirely LLM-judged, so it inherits judge bias; the code link is not on the paper page and the license is unknown.

Details

Workloadsml engineering, research
Interactionautonomous long horizon
Environmentcontainers
End state checkedNo
Toolsshell, python
Gradingllm judge
Size201
Datareal public
Personal datanone
Licenseunknown
Harnessown repo
Statusactive

Links