Mahal Systems BrowseComp-ZH
Fits agents that must search Chinese-language sites, where English benchmarks say little. Borrow the reverse-engineered question design and the encrypted dataset with a canary token.
What it asks the agent to do
- answer a multi-hop question by browsing the Chinese web
- reconcile conflicting information from several Chinese sources
- return a short verifiable answer such as a date, number or name
What to borrow from it
case templatebaselineprivacy pattern
Caveats
Small (289 questions); live web so results drift; the README restricts the data to academic research use; grading relies on a GPT-4o answer-extraction step.
Details
| Workloads | web browsing, research, knowledge qa |
|---|---|
| Interaction | autonomous long horizon |
| Environment | browser |
| End state checked | No |
| Tools | web search, page browsing |
| Grading | llm judge |
| Size | 289 |
| Data | real public |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | active |