Mahal Systems

Overview › Benchmark catalog

Evo-Memory

Fits agents that are expected to get better over a deployment by learning from their own past runs. Borrow the idea of turning a fixed test set into a sequential stream and measuring gain over time.

What it asks the agent to do

  • solve a stream of related tasks and reuse what earlier tasks taught
  • update stored experience after each interaction
  • answer single-turn reasoning questions using memory of past attempts

What to borrow from it

grading designbaseline

Caveats

Code was promised but no repository link was found; the ten source datasets are existing public sets re-ordered, so leakage from pretraining is possible.

Details

Workloadsmemory, knowledge qa, general assistant
Interactionmulti turn scripted
Environmentstatic
End state checkedNo
Toolsnone
Gradingexact match
SizeNot published
Datamixed
Personal datanone
Licenseunknown
Harnessnone
Statusactive

Links