Mahal Systems ToolEyes
Fits general assistants with a broad tool library. Borrow the five-dimension scoring (format, intent, planning, tool selection, answer) that mixes rule checks with a judge instead of grading only the final answer.
What it asks the agent to do
- understand the intent behind a real-world request
- plan which tools to call and in what order
- call a real tool with correct format and parameters
- organize the tool output into a usable answer
What to borrow from it
grading designtaxonomy
Caveats
Depends on live third-party tools and GPT-4 judging, so results drift; run scripts are not documented in the README.
Details
| Workloads | general assistant, tool calling, finance, research |
|---|---|
| Interaction | single turn |
| Environment | api sandbox |
| End state checked | No |
| Tools | 568 real tools in 7 scenarios (text generation, data understanding, real time search, app manipulation, personal life, information retrieval, financial transactions) |
| Grading | llm judge, programmatic check |
| Size | 382 |
| Data | mixed |
| Personal data | none |
| License | Apache-2.0 |
| Harness | none |
| Status | active |