Mahal Systems UNDERWRITE
Fits insurance and lending agents that must collect information before deciding. Borrow the imperfect simulated user and the pass^k reliability drop as a reported metric.
What it asks the agent to do
- gather applicant details from a simulated user over several turns
- look up underwriting guidelines through noisy tools
- decide whether and how to underwrite an insurance policy
What to borrow from it
user simulatorgrading design
Caveats
Vendor benchmark with no public code or task data; leaderboard already shows top models above 90%.
Details
| Workloads | finance, customer support, tool calling |
|---|---|
| Interaction | multi turn simulated user |
| Environment | api sandbox |
| End state checked | No |
| Tools | underwriting knowledge and policy lookup tools |
| Grading | programmatic check, pass^k |
| Size | 300 |
| Data | synthetic |
| Personal data | none |
| License | unknown |
| Harness | none |
| Status | active |