Computer Vision in Indian Retail: What Works on the Floor

Shelf-monitoring models trained mostly on neatly faced, evenly lit modern-trade shelves tend to struggle in Indian general trade. SKUs are stacked three-deep, lighting mixes tube lights and daylight, and one brand can have many pack variants on the same shelf. The computer vision stack that works here looks meaningfully different from a generic one. Here is what works on the floor.

The conditions that break off-the-shelf models

• Stacking and occlusion. SKUs partially hidden by other SKUs. Detection models trained on cleanly faced shelves can badly under-detect stock that is only partly visible.
• Pack variant proliferation. One brand, several sizes, several flavours and regular packaging refreshes. Image classification needs constant updating.
• Mixed lighting. Tube light, daylight from the door, phone flash. White balance drifts within a single store visit.
• Phone-captured input. Reps walk the aisle and take photos. Angles, distances and motion blur vary widely.

The stack that performs

1. SKU-level detection with India-specific fine-tuning. Start with a strong general detector (YOLO11, RT-DETR) and fine-tune on locally captured shop-floor images across store formats (kirana, chemist, modern trade) and across cities. Build the test set from stores the model has never seen, or your accuracy numbers will be optimistic.
2. Embedding-based recognition for SKU identity. Detection finds the box; embedding retrieval matches it to the SKU catalogue. Update the embedding gallery whenever a pack refreshes, which is far cheaper than retraining a classifier.
3. Pack-variant matrix. Don't collapse "Brand X 50g classic" and "Brand X 50g masala" into "Brand X 50g". Keep pack-level granularity in the catalogue and in the evaluation.
4. Field-app feedback loop. Reps confirm or correct detections on the device. Corrections flow back into the training pipeline.

Capture guidelines matter as much as the model

A lot of accuracy is won or lost before inference. Build a capture guide into the rep app:

• Stand square to the shelf, one bay per photo, whole bay in frame.
• Use an on-screen overlay to keep distance and angle roughly constant.
• Check blur and exposure on the device and ask for a retake immediately.
• Record a store-level photo, location and time for audit.

On-device quality checks are cheap and prevent the most common failure: a blurred photo that produces a confident but wrong count.

Planogram compliance vs share-of-shelf

These are different problems and need different evaluations. Planogram compliance asks "is this SKU in its designated facing?" and needs spatial accuracy. Share-of-shelf asks "what percentage of the visible shelf is brand X?"; it tolerates some spatial error but needs accurate SKU classification. For compliance, measure position accuracy per facing against the planogram. For share-of-shelf, measure the error in each brand's share against a manual count. Build the metric for the question the brand is actually paying for.

On-device vs server inference

Field connectivity is variable. On-device detection gives reps immediate feedback, usually with TensorFlow Lite (now LiteRT) or ONNX Runtime, while server-side reprocessing delivers analytics-grade accuracy. Both paths use the same training data; the on-device model is quantised and pruned. Queue uploads until the phone has signal, and test on the budget and mid-range Android phones reps actually carry, not only on flagships. Don't make reps wait for the cloud.

The label problem nobody warns you about

Shop-floor data is expensive to label well. SKUs are tiny in frame, partial and sometimes upside-down. Base your labelling-time estimates on a timed pilot batch, not on how fast clean catalogue images can be labelled; shelf images are usually several times slower. Annotation teams in Tier-2 cities can be cost-effective and deliver comparable quality when properly supervised. Our annotation pipelines piece covers pre-labelling, calibration and QA for this layer.

A rollout sequence

1. Pilot. A handful of stores per format and one category, with manual counts as ground truth.
2. Measure. Detection recall, SKU accuracy and share error, reported per store format.
3. Fix. Capture guide, variant catalogue and fine-tuning on the failure cases.
4. Expand by category, not only by store count. Each new category brings new packs and new confusions.
5. Operate. A regular catalogue refresh and a monthly drift review.

Common mistakes

• Reporting accuracy on the same stores the model was trained on.
• Collapsing pack variants to make the numbers look better.
• Rolling out to every category at once.
• Ignoring the rep experience. If the app is slow or fussy, photo quality drops, and accuracy drops with it.

FAQ

How do we keep up with new packs?

Add a few reference images of each new pack to the embedding gallery before it reaches shelves. Ask brand teams for their launch calendars so this happens ahead of time.

Do we need depth cameras?

For most share-of-shelf work, no. Phone photos with a good capture guide are enough. Depth helps when counting stacked stock accurately matters.

How often should we retrain the detector?

When the drift review shows recall falling for a store format or category, rather than on a fixed calendar. In practice the embedding gallery needs frequent updates while the detector changes less often.

How we approach this at Velura Labs

Our AI & Data Solutions service handles end-to-end retail computer vision: data collection, annotation, model training and on-device deployment. Pair it with Mobile App Development for the rep-facing field app, and see our Bharat design patterns for the rep UX. Talk to us if your shelf-monitoring accuracy plateaued after the initial rollout; a drifting embedding gallery is a common cause.

Available to businesses across the United States (Washington, California, Texas, New York), Europe (France, Italy and the wider EU), the Middle East (Dubai and the Gulf) and India. Get in touch to scope your build.