Mahal Systems

Overview › Benchmark catalog

DocILE

Fits invoice and purchase-order extraction agents. Borrow its 55-class field taxonomy, its line-item grouping task, and its scorer that matches predicted fields to ground truth by bounding-box overlap and reports precision, recall and F1 per field.

What it asks the agent to do

  • locate and extract key fields (amounts, dates, identifiers) from an invoice or order
  • extract each line item from a table on a business document
  • handle a document layout never seen in training

What to borrow from it

grading designcase templatetaxonomy

Caveats

The 2023 competition has ended; the data download needs a token from docile.rossum.ai and the dataset license is not stated in the paper; only 6.7k annotated documents, padded with 100k synthetic ones; built for layout models, not for agent workflows.

Details

Workloadsdocuments, finance, workflow automation
Interactionsingle turn
Environmentstatic
End state checkedNo
Toolsnone
Gradingprogrammatic check
Size6,700
Datamixed
Personal datapossible
LicenseMIT
Harnessown repo
Statusfrozen

Links