Mahal Systems DeepResearchGym
Fits teams who need reproducible search for a research agent. Borrow the stable search sandbox over public corpora and the three judge dimensions: alignment with the information need, retrieval faithfulness, and report quality.
What it asks the agent to do
- write a cited report for a complex information need
- search a fixed public web index instead of a commercial API
- keep retrieved passages faithful to the report's claims
What to borrow from it
sandbox designgrading design
Caveats
The paper says the code and search API URLs will be released but the arXiv page gives no repository; the corpora are static so recent events are missing; queries come from Researchy Questions (top 1,000 of the test set), not purpose-written tasks.
Details
| Workloads | research, web browsing |
|---|---|
| Interaction | autonomous long horizon |
| Environment | api sandbox |
| End state checked | No |
| Tools | search API over ClueWeb22 and FineWeb |
| Grading | llm judge |
| Size | 1,000 |
| Data | real public |
| Personal data | none |
| License | unknown |
| Harness | none |
| Status | active |