Mahal Systems DocBench
Fits document question-answering assistants over contracts, filings or papers. Borrow the four question types (including unanswerable) over 229 real documents in five domains.
What it asks the agent to do
- answer a question about a long real document (PDF)
- extract a number or table value from a document
- reason over several parts of one document
- handle questions that cannot be answered from the document
What to borrow from it
case templatetaxonomy
Caveats
Read-only; grading relies on an OpenAI judge; no explicit data license; paper marked work in progress.
Details
| Workloads | documents, knowledge qa, finance, legal |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | document file as input |
| Grading | llm judge |
| Size | 1,102 |
| Data | real public |
| Personal data | possible |
| License | unknown |
| Harness | own repo |
| Status | active |