Mahal Systems UltraHorizon
Fits teams stress-testing agents on very long sessions (200k+ tokens, 400+ calls). Borrow the eight-way error taxonomy (in-context locking versus capability gaps).
What it asks the agent to do
- explore an unknown environment over hundreds of tool calls to discover hidden rules
- keep track of findings across a very long context
- run experiments and update hypotheses (grid, sequence and genetics puzzles)
What to borrow from it
taxonomybaseline
Caveats
Abstract puzzle environments, not work tasks; expensive runs; scoring uses a judge model.
Details
| Workloads | planning, general assistant, research |
|---|---|
| Interaction | autonomous long horizon |
| Environment | api sandbox |
| End state checked | No |
| Tools | environment specific exploration tools |
| Grading | llm judge, programmatic check |
| Size | Not published |
| Data | synthetic |
| Personal data | none |
| License | unknown |
| Harness | own repo |
| Status | active |