Mahal Systems ScreenSpot-Pro (and ScreenSpot)
Fits any screenshot-driven agent whose failures are mis-clicks; use it to test the grounding model alone. Borrow the instruction-plus-bounding-box case format.
What it asks the agent to do
- click the element named by an instruction in a high-resolution professional app screenshot
- ground targets in CAD, creative, developer, scientific and office software
What to borrow from it
baselinecase template
Caveats
Grounding only, no task completion; the older ScreenSpot v1 set is near solved; 23 apps across 3 operating systems so coverage of any one app is thin.
Details
| Workloads | computer use |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | screenshot in, coordinate out |
| Grading | exact match |
| Size | 1,581 |
| Data | real scraped |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | active |