Mahal Systems WebCanvas (Mind2Web-Live)
Fits any live-site browser agent where the final page cannot be checked directly. Borrow key-node grading: credit for reaching required URLs or elements mid-task.
What it asks the agent to do
- complete a Mind2Web task on the live website
- pass through required intermediate pages or states on the way
What to borrow from it
grading design
Caveats
Key nodes break when sites redesign and need upkeep; no end-state verification; top success at release was 23.1 percent.
Details
| Workloads | web browsing, shopping, travel |
|---|---|
| Interaction | single turn |
| Environment | browser |
| End state checked | No |
| Tools | live web browsing |
| Grading | trajectory check, programmatic check |
| Size | 542 |
| Data | real public |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | active |