Mahal Systems SWE-bench-java
An early Java-only issue set; useful if your agent targets JVM codebases and you want a tiny hand-verified pilot. Borrow the Docker-per-instance build setup for Maven projects.
What it asks the agent to do
- resolve a GitHub issue in a Java project such as gson or jackson
- make a failing JUnit test pass inside a Docker build
What to borrow from it
case template
Caveats
Only 91 instances from six repos, more than half from jackson-databind; folded into Multi-SWE-bench, which is the version to use now.
Details
| Workloads | coding |
|---|---|
| Interaction | autonomous long horizon |
| Environment | containers |
| End state checked | Yes |
| Tools | shell, file editor |
| Grading | unit tests |
| Size | 91 |
| Data | real public |
| Personal data | possible |
| License | unknown |
| Harness | own repo |
| Status | frozen |