Did you know chameleons can see in almost any direction – meaning a single eye can rotate 360 degrees? Think of Azure Percept a bit like this. While the Azure Percept Dev Kit may only have one “eye,” it doesn’t mean developers can’t use Azure Percept Vision—Percept’s camera used for image and video analytics—to see from many angles.
At Tata Consultancy Services (TCS), we utilize video analytics to provide solutions for retail providers looking to use machine learning for inventory management and secure customer transactions. This is one of the many use cases to drive transformation in retail operations. For example, we’re doing surveillance of self-checkout machines to guarantee each item undergoes a proper transaction, and we’re also doing surveillance of shelves to ensure that items are restocked as needed. Azure Percept can be a part of such a solution which aims to tackle multiple aspects of retail management.
Our solution hack for a recent Azure Percept Bootcamp demonstrates the capability of Azure Percept with respect to smart retail store solution using Azure Percept. This blog explains our solution hack and how Azure Percept can be used in the context of a retail management system. We provide instructions for how an item identification solution can be deployed with Azure Percept using machine learning and a Custom Vision project on Azure easily.
Overview of the Solution
The solution we developed using Azure Percept was an Item Identification Solution within the context of a retail management system. We used the item classification capabilities of Azure Precept to classify an item as either a bag of chips or a book. A brief description of the solution can be found in the following YouTube video:
Below is a diagram for the architecture of how our solution would fit into the larger picture of a retail management system using Azure:
In the context of retail management system, either a database or a notification system can provide capabilities for rapid response to an area in retail which requires immediate attention, such as a self-checkout gone wrong or an empty shelf. In cases where very rapid or automatic response is needed, it is not desirable to visualize such data before action is taken. One example of this would be in the case of an empty shelf, where an automatic notification could be provided to store managers once Azure Percept Vision fails to detect an item where it usually is. After the shelf is replenished, the results of this action should be reflected in the area available to Azure Percept Vision and, subsequently, in the notification system used by managers. In other scenarios, such as in the case of consistently missed item at checkout, data visualization may be necessary to see exactly what items are being missed the most, and using this knowledge, a retailer can work internally, with vendors, or with customers to help solve the problem.
In terms of next steps, we’re integrating TCS Smart Store, which aims to improve customer experience and enhance the daily operations of retail through edge-processing, with Azure Percept.
This makes for even smarter retail solutions by implementing AI models to extract insights, like a missing item from self-checkout or an empty shelf, from video inputs. Developers can use this information to quickly build new video analytics solutions or integrate their own AI-powered solutions on the Azure platform to extract actionable insights. In other words, retailers can quickly implement solutions—like security, shelf management, and workplace safety—in their stores and at scale. Insights from these varied solutions all working together ultimately help customers reduce pilfering and workplace accidents. This, in turn, helps reduce costs, ensure better shelf management, and boost revenue.