Mahal Systems LongDA
Fits analytics agents whose hard part is reading documentation, such as codebooks or data dictionaries, before querying data. Borrow the two metrics: answer structure coverage and numeric match within a tolerance.
What it asks the agent to do
- find variable definitions and rules in long survey documentation
- combine facts from several documents before computing
- write and run code over raw survey files to produce a number or list
What to borrow from it
grading designcase template
Caveats
No code repository or license was found; the agent tool set is prescribed by the authors' own framework; survey microdata may carry respondent-level records.
Details
| Workloads | data analysis, documents, research |
|---|---|
| Interaction | single turn |
| Environment | mixed |
| End state checked | No |
| Tools | document search and reading, python code execution |
| Grading | programmatic check |
| Size | 505 |
| Data | real public |
| Personal data | possible |
| License | unknown |
| Harness | none |
| Status | active |