Mahal Systems BrowseComp-Plus
Fits search agents that run over an internal document store rather than the open web. Borrow the fixed corpus with human-verified positives and mined hard negatives, and the separate retrieval metrics (nDCG, recall) next to answer accuracy.
What it asks the agent to do
- answer a multi-hop BrowseComp question using only a fixed document corpus
- plan and issue searches against a retriever, then reason over the hits
- cite the documents that support the answer
What to borrow from it
sandbox designgrading designbaseline
Caveats
The corpus is about 100K documents chosen around the questions, so it is easier than the open web; the LLM judge is Qwen3-32B; leaderboard entries are submitted by email.
Details
| Workloads | web browsing, research, knowledge qa |
|---|---|
| Interaction | autonomous long horizon |
| Environment | api sandbox |
| End state checked | No |
| Tools | search over a fixed corpus, document fetch |
| Grading | llm judge, programmatic check |
| Size | 830 |
| Data | real public |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | active |