Mahal Systems MCP-Universe
Fits agents that drive real third-party services (maps, GitHub, browser, market data) over long tool-calling sessions. Borrow its evaluator pattern (extract a value from the output, compare with an operator) and its "dynamic evaluators" that fetch live ground truth for time-sensitive tasks instead of freezing an answer.
What it asks the agent to do
- find places and routes with a maps server
- inspect or change a GitHub repository
- answer a financial analysis question from market data
- build a 3D object in Blender through its MCP server
- complete a browser automation task
- answer a time-sensitive question with web search
What to borrow from it
grading designcase templatebaseline
Caveats
Needs paid API keys (SerpAPI, Google Maps, GitHub, Notion) and a local Blender install; GitHub tasks perform real writes so a throwaway account is required; 231 hand-written tasks across six domains only; 57% of tasks depend on live data that changes.
Details
| Workloads | tool calling, web browsing, finance, devops, research |
|---|---|
| Interaction | autonomous long horizon |
| Environment | mixed |
| End state checked | No |
| Tools | 11 real MCP servers (Google Maps, GitHub, financial data, Blender, Playwright, web search, Notion) |
| Grading | programmatic check, exact match |
| Size | 231 |
| Data | mixed |
| Personal data | possible |
| License | Apache-2.0 |
| Harness | own repo |
| Status | active |