Mahal Systems LiveDRBench
Fits research agents whose output should be a set of findings rather than prose. Borrow the claim-set intermediate representation graded by precision, recall and F1 per category.
What it asks the agent to do
- find all claims or items that answer a broad research question
- search for prior art or datasets on a scientific topic
- compile facts about a public event such as a flight incident
What to borrow from it
grading designtaxonomy
Caveats
Only 100 tasks; the data is under CDLA v2 while code is MIT; the live refresh is a stated plan, not yet a feature; default judge is gpt-4o.
Details
| Workloads | research, web browsing, science |
|---|---|
| Interaction | autonomous long horizon |
| Environment | browser |
| End state checked | No |
| Tools | web search |
| Grading | llm judge |
| Size | 100 |
| Data | real public |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | active |