Mahal Systems EHRNoteQA
Fits chart-summarization or note question-answering assistants that read long discharge summaries. Borrow the clinician-reviewed question generation over multiple notes and the reported correlation with clinician judgments as a way to validate a proxy benchmark.
What it asks the agent to do
- answer a clinician's question from one or more discharge summaries
- find information spread across several admissions of the same patient
- choose the answer in multiple-choice form
What to borrow from it
case templategrading design
Caveats
Question answering only, no tools; the dataset and the MIMIC-IV-Note text both sit behind PhysioNet credentialed access and a HIPAA-compliant model endpoint is required; open-ended scoring uses a GPT-4 judge; MIT covers the code, the data carries the PhysioNet license.
Details
| Workloads | healthcare, documents, knowledge qa |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | none |
| Grading | llm judge, exact match |
| Size | 962 |
| Data | real public |
| Personal data | possible |
| License | MIT |
| Harness | own repo |
| Status | active |