Mahal Systems MemoryAgentBench
Fits any agent with an external memory store; the four competencies (retrieval, test-time learning, long-range understanding, conflict resolution) are a usable taxonomy and the repo already wires in Letta, Cognee and HippoRAG. This is the closest match to the seed name "Agent Memory Benchmark".
What it asks the agent to do
- retrieve a fact from information delivered in chunks over many turns
- learn a classification rule from examples given incrementally
- understand a long narrative delivered piece by piece
- give the latest value when earlier facts were overwritten
What to borrow from it
taxonomygrading designbaseline
Caveats
Mostly repackaged existing datasets (RULER, InfBench, LongMemEval and others), strict exact match on several subsets, and question counts are not stated in the README.
Details
| Workloads | memory, knowledge qa, general assistant |
|---|---|
| Interaction | multi turn scripted |
| Environment | static |
| End state checked | No |
| Tools | none |
| Grading | exact match, llm judge |
| Size | Not published |
| Data | mixed |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | active |