Mahal Systems TAT-QA
Fits agents that read financial statements where the answer needs both a table and the surrounding text. Borrow its answer-type tagging (span, multi-span, count, arithmetic) and its exact-match and F1 scorer with scale handling.
What it asks the agent to do
- answer a question from a table plus the paragraphs around it in a financial report
- add, subtract, count or compare numbers drawn from a table and text
- return the right scale and unit with a numeric answer
What to borrow from it
case templategrading designbaseline
Caveats
From 2021 and pre-dates current models; DocMath-Eval re-annotated part of it, which suggests label noise; the 2,757 contexts come from a limited set of report layouts.
Details
| Workloads | finance, documents, data analysis |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | none |
| Grading | exact match |
| Size | 16,552 |
| Data | real public |
| Personal data | none |
| License | CC-BY-4.0 |
| Harness | own repo |
| Status | active |