Case Study — Cleaner Training Data for Recommendations — Straight Up AI
Case Study · E-commerce · £2m a year

Creating a £2 million opportunity through optimised product matching

The client's recommendation model learned by comparing the product a shopper chose against thousands they didn't. During the training process we found examples where the shopper had engaged with different products. This meant the model was being trained to penalise historic engagement with different products. In fixing this we were able to generate an estimated £2 million of additional sales annually — at no extra running cost.

Client
E-commerce
Service
ML Engineer · Recommendations
Focus
Training data quality & recommendation accuracy
Stack
PyTorch · TorchRec · Databricks · MLflow
The Challenge

Duplication rendered popular products unpopular

Like a human, AI models learn by seeing examples over and over again. In this case the model was receiving examples of products the user had engaged with and similar products they didn't. This allowed the model to suggest genuine products of interest against products that similar users were interested in.

This approach breaks down when the same product turns up twice in a round. For example, 4 shoppers all interact with a trending dress. The model is then told that the dress is both an item of interest and a close, but irrelevant match.

The result is the trending dress becomes downweighted across all the shoppers. And due to popularity of the product the model sees the example more often, reinforcing the incorrect association.

The Approach

Measure first, then fix

↑ ↓ ↓ ↓ ↓ ↓ ↓ ↑ ↓ ↓ ↓ ↓ ↓ ↓ ↑ ↓ ↓ ↓ ↓ ↓ ↓ ↑ ↓ ↓ ↓ ↓ ↓ ↓ ↑ ↓ ↓ ↓ ↓ ↓ ↓ ↑
→
↑ ↓ ↓ ↓ ↓ ↓ ↓ ↑ ↓ ↓ ↓ ↓ ↓ ↑ ↓ ↓ ↓ ↓ ↓ ↑ ↓ ↓ ↓ ↓ ↓ ↑ ↓ ↓ ↓ ↓ ↓ ↑
The Results
+1.8%
Increase in returning items the shopper actually wanted.
~10%
Improvement in data dedeuplication during training.
£2m
Estimated additional sales a year from that lift.
More Work

Related case studies

2.6 Million New Customers
Halved customer interaction threshold in recommender system with zero impact to power users.
E-commerce · Cold Start
£500k Unlock For Startup Launch
Character drift for startup project unblocked, allowing for ship on schedule.
Content Creation · GenAI
75% Reduction in LLM Tokens
In a KPI scoring system we cut LLM calls from 25 to 6 by grouping related KPIs into single prompt requests.
Healthcare · GenAI

Is your model learning from noise?

See if we're a fit →
Previous

Recommendation Cold Start
Next

Token Reduction