Mahal Systems

Overview › Benchmark catalog

MobileWorld

Fits mobile agents in shopping and workplace apps. Borrow the backend-database checks for state and the user-interaction tasks.

What it asks the agent to do

  • complete a workflow across several apps (chat, social, e-commerce)
  • ask the user a clarifying question before acting
  • combine GUI steps with MCP tool calls

What to borrow from it

sandbox designgrading designuser simulator

Caveats

New and single-vendor; apps are open-source stand-ins, not the real products; best framework 51.7 percent, best end-to-end model 20.9.

Details

Workloadsmobile, shopping, workflow automation, tool calling
Interactionmulti turn simulated user
Environmentmobile
End state checkedYes
ToolsAndroid emulator snapshot, self hosted Mattermost / Mastodon / Mall4Uni backends, MCP tools
Gradingstate diff, exact match, programmatic check
Size201
Datasynthetic
Personal datanone
LicenseApache-2.0
Harnessown repo
Statusactive

Links