Mahal Systems MMDeepResearch-Bench
Fits research agents whose inputs include charts, screenshots or photos. Borrow the three separate scores for report quality, citation fidelity and text-to-image consistency, and the pass/fail gate on visual evidence.
What it asks the agent to do
- write a cited report from a bundle of images and text
- search for sources that back claims about visual evidence
- keep text, citations and figure references consistent
What to borrow from it
grading designtaxonomy
Caveats
New and small (140 tasks, 100 research and 40 daily); the composite score mixes four weighted parts, so a single number hides which part failed; one earlier commit had a scoring bug.
Details
| Workloads | research, web browsing, documents |
|---|---|
| Interaction | autonomous long horizon |
| Environment | browser |
| End state checked | No |
| Tools | web search, image understanding |
| Grading | llm judge, programmatic check |
| Size | 140 |
| Data | real public |
| Personal data | none |
| License | Apache-2.0 |
| Harness | own repo |
| Status | active |