Mahal Systems FrontierScience
Fits science reasoning assistants where you want a hard, unsaturated reference number. Borrow its research-track rubrics that give partial credit for progress through a task rather than only the final answer.
What it asks the agent to do
- solve an olympiad-level physics, chemistry or biology problem
- work through a PhD-level open research sub-task
- show intermediate reasoning that a rubric can credit
What to borrow from it
grading designbaseline
Caveats
Only the 160-question gold set is public (100 olympiad, 60 research); the rest is held by OpenAI; no tools or environment, so it measures reasoning not agent behaviour; no official repository or license found, only the inspect_evals port.
Details
| Workloads | science, knowledge qa, research |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | none |
| Grading | exact match, llm judge |
| Size | 160 |
| Data | synthetic |
| Personal data | none |
| License | unknown |
| Harness | inspect_evals |
| Status | active |