Mahal Systems MCP-RADAR
Fits teams that care about efficiency (rounds, tokens, time) as well as correctness in tool use, and that want mock servers for email and calendar with a controlled local database. Borrow its accuracy-versus-efficiency reporting and its mock-server design.
What it asks the agent to do
- solve a math problem with a calculator MCP tool and give an exact answer
- answer a question using a web search tool
- carry out an email or calendar operation (send, reply, schedule) via mock servers
- perform the right file-management or terminal command
- finish in as few tool rounds and tokens as possible
What to borrow from it
grading designsandbox design
Caveats
Code and data are only on an anonymous 4open.science link, no GitHub repo or license; per-domain task counts sum to 515, not the stated 507; email, calendar, file and terminal tasks are graded by matching the final tool call, not the end state.
Details
| Workloads | tool calling, email calendar, it ops, knowledge qa |
|---|---|
| Interaction | single turn |
| Environment | mixed |
| End state checked | No |
| Tools | 49 MCP tools (calculator, DuckDuckGo search, filesystem, terminal are real; email, calendar and a local database are mocked) |
| Grading | exact match, programmatic check |
| Size | 507 |
| Data | mixed |
| Personal data | none |
| License | unknown |
| Harness | none |
| Status | frozen |