Mahal Systems VisualWebArena
Fits agents that must read screenshots or product images while browsing (classifieds, marketplaces). Borrow the image-conditioned task templates and the execution-based checks.
What it asks the agent to do
- find a listing or product that matches a reference image
- post a classified ad with a given picture
- compare items by what is shown in their photos
- answer a question that needs reading an image on the page
What to borrow from it
sandbox designgrading designcase template
Caveats
Built on the WebArena stack with the same hosting burden; some checks use a vision model judge; human success is about 89 percent so ceiling is below 100.
Details
| Workloads | web browsing, retail, shopping |
|---|---|
| Interaction | single turn |
| Environment | browser |
| End state checked | Yes |
| Tools | browser actions on self hosted sites, image input |
| Grading | exact match, programmatic check, state diff, llm judge |
| Size | 910 |
| Data | mixed |
| Personal data | possible |
| License | MIT |
| Harness | own repo |
| Status | active |