Mahal Systems Online-Mind2Web
Fits general browser agents on public consumer sites. Borrow WebJudge, a screenshot-based LLM judge with released human labels to calibrate against.
What it asks the agent to do
- complete a user task on a live public website starting from its home page
- find and filter items (clothing, food, housing, transport)
- fill in a form or reach a confirmation page
What to borrow from it
grading designbaseline
Caveats
Live sites drift and break tasks; judge is about 85 percent agreement with humans; no end-state verification; human review of leaderboard entries is outsourced.
Details
| Workloads | web browsing, shopping, travel, general assistant |
|---|---|
| Interaction | single turn |
| Environment | browser |
| End state checked | No |
| Tools | live web browsing |
| Grading | llm judge, human |
| Size | 300 |
| Data | real public |
| Personal data | none |
| License | CC-BY-4.0 |
| Harness | hal, own repo |
| Status | active |