Mahal Systems WDC Products
Fits product catalog or deduplication agents that must say whether two records are the same item. Borrow its difficulty dials (share of corner cases, share of unseen entities, training set size) across 27 variants built from 11,715 offers of 2,162 products.
What it asks the agent to do
- decide whether two product offers from different shops describe the same product
- match offers with hard corner cases such as near-identical model numbers
- generalize to products never seen in the training data
- classify an offer into one of many product entities (multi-class form)
What to borrow from it
taxonomybaselinecase template
Caveats
A classic supervised entity matching dataset, not an agent benchmark: F1 on labeled pairs with no tools or environment; size above is pairs per pairwise test set; data is scraped from 3,259 shops via Common Crawl in 2020 and no license is stated.
Details
| Workloads | data analysis, retail, shopping |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | none |
| Grading | programmatic check |
| Size | 4,500 |
| Data | real scraped |
| Personal data | none |
| License | unknown |
| Harness | none |
| Status | active |