Terminal-Bench 1.0 (Core v0.1)

The original terminal task format (task.yaml, Dockerfile, solution.sh, test script) is a clean template for any shell-based agent case; borrow the oracle-solution check.

What it asks the agent to do

• complete a sysadmin or build task in a Linux shell such as compiling code or configuring a server
• train a small model or process data from the command line
• leave the container in a state that a hidden test script accepts

What to borrow from it

sandbox designgrading designcase template

Caveats

Replaced by 2.0 and later versions; the beta had roughly 100 tasks of uneven quality and the site now reports only the newest version, so old scores are not comparable.

Details

Links

• Paper
• Github
• Leaderboard