Mahal Systems CRAB
Fits agents that must coordinate across a phone and a computer. Borrow the graph evaluator that gives partial credit for sub-goals reached in any valid order.
What it asks the agent to do
- complete a task that spans an Ubuntu desktop and an Android phone
- move information between devices
What to borrow from it
grading designsandbox design
Caveats
Small; cross-device setup is complex; license not stated in the README.
Details
| Workloads | computer use, mobile, multi agent |
|---|---|
| Interaction | single turn |
| Environment | mixed |
| End state checked | Yes |
| Tools | Ubuntu VM, Android emulator, Python wrapped actions |
| Grading | state diff, trajectory check |
| Size | 120 |
| Data | synthetic |
| Personal data | none |
| License | unknown |
| Harness | own repo |
| Status | active |