Mahal Systems DeepResearch Bench
Fits report-writing research agents. Borrow RACE (task-specific weighted criteria scored against a reference report) and FACT (extract claim-URL pairs and check whether the cited page supports each claim).
What it asks the agent to do
- write a long cited research report on a PhD-level question
- search the web and pick sources that support each claim
- follow the instructions in the task about scope and structure
What to borrow from it
grading designbaselinetaxonomy
Caveats
Only 100 tasks, half Chinese and half English; scores depend on the judge model, which the maintainers are switching, so numbers across time are not comparable; FACT needs a scraping API key.
Details
| Workloads | research, web browsing, documents |
|---|---|
| Interaction | autonomous long horizon |
| Environment | browser |
| End state checked | No |
| Tools | web search, page scraping |
| Grading | llm judge, programmatic check |
| Size | 100 |
| Data | real public |
| Personal data | none |
| License | Apache-2.0 |
| Harness | own repo |
| Status | active |