Mahal Systems OSWorld-MCP
Fits agents that mix GUI control with API or MCP tools. Borrow the tool-invocation-rate and average-completion-steps metrics alongside accuracy.
What it asks the agent to do
- finish an OSWorld-style desktop task with 158 MCP tools available
- decide when to call a tool instead of clicking through the GUI
What to borrow from it
grading designcase template
Caveats
Task count not stated in abstract or site; tools were partly auto-generated; license not stated; numbers changed between preprint and ICLR versions.
Details
| Workloads | computer use, tool calling, documents |
|---|---|
| Interaction | single turn |
| Environment | desktop |
| End state checked | Yes |
| Tools | GUI actions, 158 MCP tools across 7 apps |
| Grading | state diff, programmatic check, trajectory check |
| Size | Not published |
| Data | mixed |
| Personal data | none |
| License | unknown |
| Harness | own repo |
| Status | active |