Mahal Systems BizFinBench
Fits retail-investor assistants and finance chat agents. Borrow the IteraJudge idea (an iterated judge that refines its decision boundary) and the nine-category task split when designing a judge for objective finance answers.
What it asks the agent to do
- attribute an unusual market move to its cause
- compute a financial number from given data
- reason about dates and time windows in finance
- describe a table of financial data in words
- pick and fill the right financial tool call for a request
- answer financial knowledge questions
What to borrow from it
grading designtaxonomybaseline
Caveats
Question answering only, not an environment; v1 is Chinese-only and v2 changed the question count, so quote the version; data is non-commercial (code is Apache-2.0); the judge is itself an LLM, so calibrate it before trusting scores.
Details
| Workloads | finance, knowledge qa, data analysis, tool calling |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | none; tool use tasks are judged on the written call |
| Grading | llm judge, exact match |
| Size | 6,781 |
| Data | mixed |
| Personal data | none |
| License | CC-BY-NC-4.0 |
| Harness | own repo |
| Status | active |