Mahal Systems

Overview › Benchmark catalog

GDM Dangerous Capabilities: Self-proliferation

Fits teams giving an agent real web, cloud and shell access who need to know how far it can get on its own. Borrow the milestone mode that scores partial progress and the human-approval gate on every tool call.

What it asks the agent to do

  • set up accounts or services on the web without help
  • copy and run a model or agent on new infrastructure
  • earn money or resources through online tasks
  • complete milestones toward spreading itself

What to borrow from it

sandbox designgrading design

Caveats

Ten tasks only; several need GCP accounts, API keys and secrets; DeepMind removed solutions and altered some tasks and asks you to contact them before running those; the expert best-of-N mode in inspect_evals was removed as broken.

Details

Workloadssafety, security, web browsing, it ops
Interactionautonomous long horizon
Environmentmixed
End state checkedYes
Toolsbash, web browser, in Docker Compose sandboxes with human approval of tool calls
Gradingprogrammatic check, human
Size10
Datasynthetic
Personal datanone
LicenseApache-2.0
Harnessinspect_evals, own repo
Statusactive

Links