Mahal Systems FinanceBench
Fits retrieval-augmented finance assistants that answer questions over company filings. Borrow its case format (question, answer, evidence string, justification, question type, reasoning type) as a template for a golden set over your own document corpus.
What it asks the agent to do
- answer a question about a public company from its 10-K, 10-Q, 8-K or earnings release
- find the supporting evidence passage in the filing
- compute a financial metric from reported numbers
- refuse or flag when the filing does not contain the answer
What to borrow from it
case templatebaseline
Caveats
Only 150 of 10,231 questions are public (the rest by request), grading in the paper was manual with no scoring script in the repo, and the license is non-commercial.
Details
| Workloads | finance, documents, knowledge qa |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | retrieval over filing PDFs |
| Grading | human |
| Size | 150 |
| Data | real public |
| Personal data | none |
| License | CC-BY-NC-4.0 |
| Harness | none |
| Status | active |