Mahal Systems macOSWorld
Fits agents that must run on Mac desktops or in several languages. Borrow the multilingual instruction and interface variants and the deception subset.
What it asks the agent to do
- complete tasks in 30 macOS apps (system, productivity, media)
- follow instructions in English, Chinese, Arabic, Japanese or Russian
- resist on-screen deception attacks in the safety subset
What to borrow from it
sandbox designtaxonomy
Caveats
Needs an AWS macOS instance and 15 to 20 minutes per task; grading details not confirmed from abstract or README; license not stated.
Details
| Workloads | computer use, safety |
|---|---|
| Interaction | single turn |
| Environment | desktop |
| End state checked | Yes |
| Tools | cloud hosted macOS instance, mouse and keyboard |
| Grading | programmatic check |
| Size | 202 |
| Data | synthetic |
| Personal data | none |
| License | unknown |
| Harness | own repo |
| Status | active |