Mahal Systems FinAgentBench
Fits retrieval-augmented filing agents where the retrieval step is the weak point. Borrow the two-stage split (document-type ranking then chunk ranking) and the ten query categories to diagnose where retrieval fails before judging final answers.
What it asks the agent to do
- choose which filing type (10-K, 10-Q, 8-K, earnings call) answers a question
- rank passages within the chosen filing by relevance
What to borrow from it
taxonomygrading design
Caveats
Measures retrieval ranking (nDCG, MAP, MRR at 5) only, not end-to-end answers or tool use; no public data or code release was found at the time of cataloging; the size is given as about 26K in v4 while earlier versions reported 3,429.
Details
| Workloads | finance, documents, research |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | none |
| Grading | programmatic check |
| Size | 26,000 |
| Data | real public |
| Personal data | none |
| License | unknown |
| Harness | none |
| Status | active |