Mahal Systems WideSearch
Fits agents that build lists or spreadsheets from the web (lead lists, product comparisons, compliance registers). Borrow the table grader: per-column match rules (exact, numeric, date, URL or judge), row and cell F1, and a strict all-cells-correct success rate.
What it asks the agent to do
- collect many small facts about every item in a set and return a table
- fill a fixed set of columns for each entity with correct values
- verify each cell against public sources
What to borrow from it
grading designcase templatebaseline
Caveats
Best systems score about 5% success, so it mainly separates failure modes; half the tasks are Chinese; the live web means ground-truth tables can go stale.
Details
| Workloads | research, web browsing, data analysis |
|---|---|
| Interaction | autonomous long horizon |
| Environment | browser |
| End state checked | No |
| Tools | web search, page browsing |
| Grading | programmatic check, llm judge, pass^k |
| Size | 200 |
| Data | real public |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | active |