Mahal Systems LiveCodeBench Pro
Fits teams that need a hard, uncontaminated algorithmic test with expert error labels; borrow the Olympiad-medalist failure taxonomy for grading reasoning errors.
What it asks the agent to do
- solve an Olympiad-level Codeforces, ICPC or IOI problem in C++
- handle hard algorithmic reasoning and case analysis, not just implementation
What to borrow from it
taxonomybaseline
Caveats
Problem count and license are not stated; leaderboard entry is by emailing results, so it is not independently run; algorithm puzzles, not software engineering.
Details
| Workloads | coding |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | none |
| Grading | unit tests |
| Size | Not published |
| Data | real scraped |
| Personal data | none |
| License | unknown |
| Harness | inspect_evals, own repo |
| Status | active |