Mahal Systems

Overview › Benchmark catalog

ToolHop

Fits research or lookup agents that must chain calls where each step depends on the last. Borrow the query-driven method of writing the tools after the question so every case has a verifiable answer.

What it asks the agent to do

  • answer a question that needs three or more dependent tool calls
  • pick the next tool from the result of the previous call
  • stop and give a verifiable final answer

What to borrow from it

case templatebaseline

Caveats

One user query per case with no conversation; questions derive from MuSiQue, and the code lives in a Hugging Face dataset repo rather than a maintained GitHub project.

Details

Workloadsknowledge qa, tool calling, research
Interactionsingle turn
Environmentapi sandbox
End state checkedNo
Tools3,912 locally executable code tools generated per query
Gradingexact match
Size995
Datamixed
Personal datanone
LicenseApache-2.0
Harnessown repo
Statusactive

Links