In the retail world, shelves are pivotal to product visibility and consumer engagement. Shelf analysis involves the examination and evaluation of retail shelves to optimize product placement, monitor stock levels, and improve overall store performance. Customer experience and operational efficiency are two areas where good shelf analysis is especially important. The placement of each item is a critical factor that can make or break the success of your product. Traditionally, this work has relied on manual methods and required human intervention, such as time-consuming and error-prone store audits.
Product recognition allows retailers to monitor shelf inventory using pre-built AI model to identify and recognize various products, as well as gaps. It also enables customizable AI models to classify individual packaged goods with tags including their brand, type, size, and packaging. After training your custom AI model, you can also assess planogram compliance - whether your shelf meets the store shelf schema in terms of the quantity and position of the retail products.
At Microsoft Build 2023, we were thrilled to announce the public preview of product recognition, the latest feature in Azure Cognitive Services for Vision. Vision Services uses a multi-modal foundation model, Florence, to help retailers develop intelligent applications to streamline operations and automate manual processes, unlocking valuable insights.
These capabilities offer a powerful solution for retailers to monitor stock levels, identify out-of-stock items, and assess discrepancies with planogram schema – all without the manual process. Through product recognition, retailers can revolutionize the way they manage their shelves by surfacing accurate insights on store inventory. Product recognition can also help store associates replenish goods faster to maximize sales opportunities, and invest more time to focus on their customers.
To get started with using product recognition, check out our full documentation on Azure Docs. We also provide you with a set of detailed how-to-guides:
- Product Recognition with prebuilt model
- Product Recognition with custom model
- Assess planogram compliance
Customer Collaboration: Blue Yonder
Blue Yonder provides industry-leading solutions in supply chain management, such as planogram compliance specializations. They have extensive experience in Retail and Consumer Packaged Goods industries. We are honored and excited to partner with Blue Yonder. Quoting from Andy Hawkins, product director at Blue Yonder: “Integrating Azure with these edge devices and services with Blue Yonder’s category management and space planning software means retailers can make more informed decisions about product placement, which leads to increased sales and better product visibility for customers.”
Getting started on Vision Studio
To get started, you can try out our product recognition capabilities on Vision Studio where you will be able to run our pre-built model for product vs. gap detection, as well as sample models for custom label classification.
- If you would like to test the pre-built model, select "Pretrained Vision Model" from the drop-down. Once you select a sample image, you will notice the product vs. gap detections, as well as the corresponding bounding box highlighted.
- If you would like to test the custom model without training a custom model on your own, we provide you with our sample image custom model to let you see what the custom model responses will be like. Select "Sample Custom Model" from the drop-down, and once you select a sample image, you will notice the individual classifications of the product names appear in the detected products list.
- If you would like to train you own custom model, you can easily get started by selecting "Train a custom model" link under the drop-down.
After you have finished trying out product recognition above, you can scroll down the page to try out our planogram compliance capability with our sample images. Planogram matching feature is only available for custom-trained models, and not available for pre-trained models. This is because pre-trained model only detects “products” vs. “gaps”, and does not have any custom-labeled products to be detected.
For you to get an idea of what planogram compliance response is, we provide you the planogram matching try out experience for the provided sample images using the sample custom models we had trained. When you select a sample image, you will see the planogram schema that was used for the sample image, as well as the matching results with the corresponding matches highlighted in the form of bounding boxes.
Getting Started with Python SDK
You can also try out our capabilities using Python in our Azure samples Python SDK repository:
You can follow the tutorials in Python notebooks:
- Image Composition: This tutorial will teach you how to stitch together the segmented shelf images using Image Stitching API, as well as adjust any slanted or squished shelf images to a correct orientation using Image Rectification API
- Product Recognition: This tutorial will teach you how to detect products and gaps on a shelf image using a pre-built model, and individually classify the detected products using customized model, both using Product Understanding API
- Planogram Compliance: this tutorial will teach you how to assess the matchings between a planogram schema and the detected products on a shelf using Planogram Compliance API
Check out the links above and see for yourself how easy it can be to deploy Microsoft's cutting-edge AI vision capabilities in your stores.
