Mahal Systems IFEvalCode
Fits coding assistants that must honor house style or constraints; borrow the split scoring of correctness versus instruction-following as two separate numbers.
What it asks the agent to do
- write correct code that also obeys explicit constraints on style, naming, algorithm or complexity
- follow the same coding instruction given in Chinese or English
What to borrow from it
grading designcase template
Caveats
About 1.6K samples but the exact count and license are not on the abstract; constraints are checked by scripts that may not cover every phrasing; no repo context.
Details
| Workloads | coding |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | none |
| Grading | unit tests, programmatic check |
| Size | Not published |
| Data | synthetic |
| Personal data | none |
| License | unknown |
| Harness | inspect_evals, own repo |
| Status | active |