Mahal Systems Mind2Web
Fits teams training or testing step-level action prediction on real site layouts. Borrow the cross-task / cross-website / cross-domain split design.
What it asks the agent to do
- predict the next click, type or select action on a real website page
- follow a multi-step instruction across pages (travel, shopping, government, entertainment)
- generalize to unseen websites and domains
What to borrow from it
case templatetaxonomybaseline
Caveats
Offline replay of recorded pages only, so no real execution or end-state check; pages are 2023 snapshots; size is train 1009 plus test 1341 as listed in the repo.
Details
| Workloads | web browsing, shopping, travel, general assistant |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | page snapshot (HTML) |
| Grading | exact match |
| Size | 2,350 |
| Data | real scraped |
| Personal data | possible |
| License | CC-BY-4.0 |
| Harness | inspect_evals, own repo |
| Status | active |