Case Study — Feature Caching & Candidate Decorrelation — Straight Up AI

Case Study · Enterprise Computer Vision

Caching the expensive parts so runtime stops tracking the dataset

The client's matching workflow recomputed the same heavy 3D geometry on every request, so a bigger comparison set meant a slower run. We decoupled the ML from the business logic and cached the precomputed assets — decorrelating runtime from candidate volume, and unlocking offline experimentation along the way.

Client

Enterprise · CV matching platform

Engagement

Lead ML Engineer

Focus

Feature caching & decorrelation

Stack

Python · ICP / point clouds · MLflow · SageMaker

Decorrelated

Workflow runtime broken away from haystack size — precomputed features removed the per-request geometry cost.

Faster & flatter

Cached runs were consistently shorter and far less spiky than uncached runs at the same load.

~0.92

Match rate held steady with caching on (0.926 vs 0.932), confirming the optimisation changed speed, not accuracy.

The Challenge

Paying for the same computation over and over

Item matching compares 3D meshes. The most expensive step — registration / ICP — depends on downsampling each mesh, which recomputes surface normals across every face. In the original workflow, this geometry work was redone on every request, for every candidate in the haystack. The bigger the comparison set, the longer the run: runtime was tightly coupled to candidate volume.

There was a second, structural problem. The ML was entangled with the production business logic, which made it impossible to experiment offline — to trial model changes without touching the live system. Any improvement to matching had to be tested in production-shaped conditions, which slowed iteration to a crawl.

And because this was a refactor of a critical path, it carried real risk: code can pass smoke tests yet fall over at production volumes. The work needed to be proven under representative load, not just unit tests.

The Approach

Decouple, precompute, then load-test for real

Create core ML contracts

Defined ML interfaces that run identically online and offline, separating the model from the production business logic and standing up a dedicated experimentation zone.

Unify feature extraction

Simplified the workflows behind a common feature-extraction and ML interface, so similarity, shape and registration all draw from one consistent representation.

Cache the expensive assets

Precomputed and stored the costly per-item assets — derived similarity/shape features and downsampled meshes for ICP — so registration no longer pays the geometry cost at request time.

Load-test the migration

Ran smoke and regression tests at production volumes (up to 500-item batches), publishing results to MLflow and comparing cached vs uncached runs for both speed and correctness.

The Results

Same answers, without the per-request tax

On matched 500-item regression runs, caching cut whole-experiment time and — more importantly — flattened the relationship between haystack size and runtime. Match rate was effectively unchanged, isolating the win to performance.

Source: internal load-testing report, 2025. Cached runs were also markedly less spiky and decorrelated from haystack size.
Run (500 items)	Match rate	True positives	False positives	Experiment duration
No cache	0.932	0.920	0.012	21.1 min
Cached	0.926	0.922	0.004	17.6 min

What We Built

The engineering underneath the numbers

Online/offline ML contractsOne set of interfaces that runs the same in production and in the experimentation zone.

A precomputed item-asset cacheDerived shape/similarity features and downsampled meshes stored once, reused on every match.

A load-testing harnessRepeatable smoke and regression tests at production volume, with results tracked in MLflow.

A path to higher concurrencyRemoved DB overheads from the hot path, clearing the way to safely raise throughput next.

Paying for the same compute twice?

See if we're a fit →