Mahal Systems Gaia2 (Meta ARE)
Fits personal and workplace assistants that must act on time and cope with events arriving during a task. Borrow the write-action verifier against oracle events and the capability splits (execution, search, ambiguity, adaptability, time, noise, agent-to-agent).
What it asks the agent to do
- handle a personal-assistant request across messaging, contacts, calendar and email apps
- act at the right time when an event is scheduled
- ask for clarification when the request is ambiguous
- adapt when the environment changes mid-task or returns noisy results
- coordinate with another agent to finish a task
What to borrow from it
sandbox designgrading designtaxonomyuser simulator
Caveats
Consumer apps, not enterprise systems; synthetic Llama-generated content with Llama license strings attached; scenario counts differ between the paper and dataset card.
Details
| Workloads | general assistant, email calendar, tool calling, multi agent, planning |
|---|---|
| Interaction | multi turn simulated user |
| Environment | api sandbox |
| End state checked | Yes |
| Tools | smartphone style apps in the ARE simulator |
| Grading | trajectory check, llm judge |
| Size | 800 |
| Data | synthetic |
| Personal data | none |
| License | CC-BY-4.0 |
| Harness | own repo |
| Status | active |