Mahal Systems Mem2ActBench
Fits tool-using assistants that should act on remembered user details rather than merely recall them. Borrow the reverse-generation trick of writing the memory chain first, then the task that depends on it.
What it asks the agent to do
- pick the right tool for a returning user based on earlier sessions
- fill tool arguments from facts the user gave in past sessions
- continue an interrupted task without asking the user to repeat details
What to borrow from it
case templategrading design
Caveats
Sessions are synthesized from ToolACE, BFCL and OASST1, no repository was found, and the scoring rule for tool and argument correctness is not spelled out on the paper page.
Details
| Workloads | memory, tool calling, general assistant |
|---|---|
| Interaction | multi turn scripted |
| Environment | static |
| End state checked | No |
| Tools | simulated function calling APIs |
| Grading | programmatic check |
| Size | 400 |
| Data | synthetic |
| Personal data | none |
| License | unknown |
| Harness | none |
| Status | active |