Mahal Systems SWE-smith
Fits teams that need many cheap coding cases from their own repository: point it at a repo, it breaks tests and writes issues. Borrow the bug-injection and keep-only-if-a-test-fails filter.
What it asks the agent to do
- fix a synthetically injected bug in a real repo from a generated issue text
- localize the file that a planted failure comes from
What to borrow from it
case templateprivacy patternsandbox design
Caveats
Bugs and issue texts are synthetic, so they are easier and more uniform than real issues; a training toolkit, not a comparable leaderboard.
Details
| Workloads | coding |
|---|---|
| Interaction | autonomous long horizon |
| Environment | containers |
| End state checked | Yes |
| Tools | shell, file editor |
| Grading | unit tests |
| Size | 52,000 |
| Data | mixed |
| Personal data | possible |
| License | MIT |
| Harness | own repo |
| Status | active |