Mahal Systems

Overview › Benchmark catalog

WebShop

Fits shopping assistants; borrow the self-hosted product catalog sandbox and the attribute-and-price reward computed from the bought item.

What it asks the agent to do

  • search a simulated store for a product matching a text request
  • pick options (size, color) and a price limit
  • add to cart and buy

What to borrow from it

sandbox designgrading design

Caveats

Simplified text-first site from 2022, not updated; single-site, single-purchase flow; 1.18 million real products scraped from Amazon.

Details

Workloadsshopping, retail, web browsing
Interactionsingle turn
Environmentbrowser
End state checkedYes
Toolssimulated e commerce site (search, click, buy)
Gradingprogrammatic check
Size12,087
Datamixed
Personal datanone
LicenseMIT
Harnessown repo
Statusfrozen

Links