Mahal Systems

Overview › Benchmark catalog

MemoryArena

Fits agents whose sessions build on each other (a buyer, planner or researcher that returns to an unfinished job). Borrow the interdependent-subtask design where memory errors compound.

What it asks the agent to do

  • carry what was learned in one session into the next dependent subtask
  • bundle purchases across sessions under a budget or preference constraint
  • narrow a product search progressively using earlier results
  • plan a group trip while honoring constraints gathered earlier
  • chain formal reasoning steps across sessions

What to borrow from it

case templatesandbox design

Caveats

Task count and grading details are not stated on the paper page or README; data license is CC BY 4.0 but the code license was not visible.

Details

Workloadsmemory, shopping, travel, web browsing, planning
Interactionautonomous long horizon
Environmentmixed
End state checkedNo
Toolssimulated shopping site, search, travel planner
Gradingprogrammatic check
SizeNot published
Datasynthetic
Personal datanone
LicenseCC-BY-4.0
Harnessown repo
Statusactive

Links