Mahal Systems Deep Research Bench (FutureSearch)
Fits research agents that must stay reproducible as the web changes. Borrow the RetroSearch idea of freezing the pages per task so human-worked answers stay valid, and the trace checks for hallucination, tool use and forgetting.
What it asks the agent to do
- work a multi-step web research task to a number, URL, probability or dataset
- compile an evidence set for a claim
- research against a frozen offline snapshot of the web
What to borrow from it
sandbox designtaxonomy
Caveats
No public code or data repository seen; runs are requested by email; the paper set has 89 instances while the site now lists more, so numbers are not stable across versions; scores across task types are not comparable.
Details
| Workloads | research, web browsing |
|---|---|
| Interaction | autonomous long horizon |
| Environment | api sandbox |
| End state checked | No |
| Tools | search and page fetch over frozen scraped pages |
| Grading | programmatic check, llm judge |
| Size | 89 |
| Data | real scraped |
| Personal data | none |
| License | unknown |
| Harness | none |
| Status | active |