Mahal Systems MultiAgentBench (MARBLE)
Fits teams running several cooperating agents (planner plus workers, or peer agents). Borrow the milestone-based KPI where an LLM detector marks which sub-goals were hit, plus the separate coordination score.
What it asks the agent to do
- several agents co-write a research idea from papers
- agents build a structure together in Minecraft
- agents find and fix database errors together
- agents split and solve a coding problem
- agents negotiate over shared resources
- agents play a social deduction game
What to borrow from it
grading designtaxonomy
Caveats
Milestone detection and coordination scores are LLM-rated, so they inherit judge bias; task counts per scenario are only partly stated; code moved between two repositories.
Details
| Workloads | multi agent, research, coding, games |
|---|---|
| Interaction | autonomous long horizon |
| Environment | mixed |
| End state checked | No |
| Tools | simulated environments per scenario |
| Grading | programmatic check, llm judge |
| Size | Not published |
| Data | mixed |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | active |