Mahal Systems LoCoMo
Fits assistants that must remember facts a user told them weeks ago across sessions. Borrow the persona-plus-event-graph recipe for generating long, consistent fake conversations without real user data.
What it asks the agent to do
- answer a question about something said many sessions ago in a long chat
- summarize the events that happened across a long conversation
- continue a multi-session dialogue consistently with earlier turns
What to borrow from it
case templateprivacy patternbaseline
Caveats
Only ten conversations (about 300 turns each), so results are noisy; original scoring is token-overlap and most later users swap in an LLM judge; the paper is CC BY-NC-SA and the repo license file was not confirmed.
Details
| Workloads | memory, knowledge qa |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | none |
| Grading | exact match, llm judge |
| Size | Not published |
| Data | synthetic |
| Personal data | none |
| License | unknown |
| Harness | own repo |
| Status | active |