Mahal Systems SWE-Gym
A training environment rather than a test set; fits teams fine-tuning their own coding agent or verifier. Borrow the prebuilt per-instance Docker images and the Lite split of 234 tasks.
What it asks the agent to do
- resolve a GitHub issue in one of 11 Python repos with a runnable environment
- produce agent trajectories and verifier training data for coding agents
What to borrow from it
sandbox designcase template
Caveats
Meant for training, so do not report it as a held-out score; Python only and overlaps in repo choice with SWE-bench.
Details
| Workloads | coding |
|---|---|
| Interaction | autonomous long horizon |
| Environment | containers |
| End state checked | Yes |
| Tools | shell, file editor |
| Grading | unit tests |
| Size | 2,438 |
| Data | real public |
| Personal data | possible |
| License | Apache-2.0 |
| Harness | own repo |
| Status | active |