Mahal Systems LegalAgentBench
Fits legal research or due-diligence agents that query several databases and write up the result. Borrow its progress-rate grading (keyword checks on annotated intermediate steps) and its tool-over-corpus sandbox.
What it asks the agent to do
- look up a company, case or statute across several legal databases
- answer a multi-hop question that chains several lookups
- draft a legal document from retrieved facts
- choose the right tool among 37 legal tools
What to borrow from it
grading designsandbox designcase template
Caveats
Chinese law and Chinese language only; 300 tasks; corpora are real legal records so names of parties appear; ReAct-style runner needs an OpenAI-compatible API key; no public leaderboard.
Details
| Workloads | legal, tool calling, knowledge qa, documents |
|---|---|
| Interaction | single turn |
| Environment | api sandbox |
| End state checked | No |
| Tools | 37 lookup and retrieval tools over 14 tables and 3 document stores |
| Grading | exact match, trajectory check |
| Size | 300 |
| Data | real public |
| Personal data | possible |
| License | MIT |
| Harness | own repo |
| Status | active |