Mahal Systems MAST (Why Do Multi-Agent LLM Systems Fail)
Fits teams running orchestrator-plus-worker systems who need a failure taxonomy for trace review. Borrow the 14 modes in three groups and the released LLM annotator with its human-agreement study.
What it asks the agent to do
- label a multi-agent trace with one of 14 failure modes
- tell apart specification problems, inter-agent misalignment and weak verification
- run an LLM annotator over new traces to find failure patterns
What to borrow from it
taxonomygrading design
Caveats
It is an annotated trace dataset (over 1,600 traces) rather than a task benchmark; traces come from seven open-source frameworks on older tasks; the paper is CC BY-NC-ND and the repo license was not stated.
Details
| Workloads | multi agent, general assistant |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | none |
| Grading | llm judge, human |
| Size | Not published |
| Data | real public |
| Personal data | none |
| License | unknown |
| Harness | own repo |
| Status | active |