Mahal Systems

Overview › Benchmark catalog

DeepShop

Fits shopping research assistants. Borrow the fine-grained scoring of attributes, filters and sort order plus the easy / medium / hard query evolution.

What it asks the agent to do

  • find products that satisfy several attributes, filters and a sort preference
  • compare options across live shopping sites and report a result

What to borrow from it

grading designtaxonomy

Caveats

No code or license found; query count not stated; live sites drift; success rates under 35 percent at release.

Details

Workloadsshopping, web browsing, research
Interactionsingle turn
Environmentbrowser
End state checkedNo
Toolslive shopping websites, search
Gradingprogrammatic check, llm judge
SizeNot published
Datamixed
Personal datanone
Licenseunknown
Harnessnone
Statusactive

Links