Mahal Systems WeaveBench
Fits coding agents that also need a GUI. Borrow the trajectory-aware judge that flags shortcuts such as faked screenshots or hard-coded results.
What it asks the agent to do
- combine GUI actions with terminal and code edits in one long task
- produce a deliverable file or artifact from a real user request
What to borrow from it
grading design
Caveats
Very new; no public repo found; judge-based grading; best pass rate 41.2 percent.
Details
| Workloads | computer use, coding, workflow automation |
|---|---|
| Interaction | autonomous long horizon |
| Environment | mixed |
| End state checked | Yes |
| Tools | Ubuntu desktop plugin inside a CLI agent runtime, shell, code |
| Grading | llm judge, trajectory check, programmatic check |
| Size | 114 |
| Data | real public |
| Personal data | none |
| License | unknown |
| Harness | none |
| Status | active |