Mahal Systems OmniDocBench
Fits the parsing step of any document pipeline, where PDFs become text for an agent to reason over. Borrow its per-element metrics (edit distance for text, TEDS for tables, CDM for formulas) and its page attribute labels for slicing results by layout and language.
What it asks the agent to do
- convert a PDF page to markdown with correct reading order
- extract a table from a page as structured markup
- transcribe formulas from a page
- parse hard layouts such as newspapers, handwriting and financial reports
What to borrow from it
grading designbaselinetaxonomy
Caveats
Tests parsing only, not question answering or extraction of business fields; the dataset is research-use only even though the code is Apache-2.0; version labels in the README are inconsistent (v1.6 leaderboard, v1.7 data); 1,651 pages.
Details
| Workloads | documents |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | none |
| Grading | programmatic check |
| Size | 1,651 |
| Data | real public |
| Personal data | possible |
| License | Apache-2.0 |
| Harness | own repo |
| Status | active |