Mahal Systems WebVoyager
Fits consumer-facing browser agents on well-known sites. Borrow the screenshot-based auto-judge that reads the last k screenshots.
What it asks the agent to do
- find a flight, hotel or product on a popular public site
- look up a fact on Wikipedia, GitHub, ArXiv or a news site
- navigate a map or booking site to an answer
What to borrow from it
baselinegrading design
Caveats
Tasks on 15 live sites go stale (Booking, Google Flights); judge-only grading with no state check; many vendors now report high scores, so it discriminates poorly.
Details
| Workloads | web browsing, shopping, travel, knowledge qa |
|---|---|
| Interaction | single turn |
| Environment | browser |
| End state checked | No |
| Tools | live web browsing with screenshots |
| Grading | llm judge, human |
| Size | 643 |
| Data | real public |
| Personal data | none |
| License | Apache-2.0 |
| Harness | own repo |
| Status | active |