Mahal Systems WebArena-Verified
Use instead of original WebArena for any browser agent on internal web apps. Borrow the deterministic, type-aware comparators that replaced substring matching and LLM judging.
What it asks the agent to do
- the same 812 WebArena tasks with corrected answers and evaluators
- a 258-task hard subset for cheaper runs
What to borrow from it
grading designbaseline
Caveats
Workshop paper only; numbers are not comparable to the old WebArena leaderboard; still needs the full WebArena site stack.
Details
| Workloads | web browsing, retail, devops, workflow automation |
|---|---|
| Interaction | single turn |
| Environment | browser |
| End state checked | Yes |
| Tools | browser actions on self hosted sites |
| Grading | exact match, programmatic check, state diff |
| Size | 812 |
| Data | mixed |
| Personal data | possible |
| License | Apache-2.0 |
| Harness | own repo |
| Status | active |