Azure Cognitive Services include a set of computer vision capabilities to help transform business operations and experiences with the power of AI. A high-impact example is the Spatial analysis service that ingests video from cameras, extracts insights and generates events to be used by other systems.
Spatial analysis uses computer vision AI on real-time video and offers the ability to understand people’s movements in a physical space, significantly increasing efficiency and resolution of customer data. The service can also determine if people are wearing uniforms or safety equipment. The People Counting operation, which helps count the number of people in a specific zone over time, can be used to estimate the number of people in a space, or generate an alert when a person appears. The Spatial analysis operations monitor how long people stay in an area or when they enter through a doorway.
You can get started with Spatial analysis by visiting Vision Studio, where you can find demo videos and information about use cases.
Safety and Security use cases
Safety and security are paramount for many businesses. The Counting People operation enables the analysis of real-time video to count the number of people in a designated zone in a camera’s field of view. The detect when people cross a line feature enables the detection of a person crossing a line, and the detect when people enter/exit a zone feature detects when a person enters or exits a zone in the camera’s field of view. These Spatial analysis operations can be used to aid in detecting tailgating, detecting people in restricted areas, or monitoring intrusion in secure spaces.
One of the AI capabilities of person detection is the Person Attribute classifier. The classifier’s AI model can be added to determine if people are wearing employee uniforms or personal protective equipment (PPE) like hard hats. This can be used for scenarios such as workplace safety.
Another set of capabilities of Spatial analysis includes vehicle detection and classification with vehicle type and vehicle color. This enables use cases for monitoring illegally parked vehicles, vehicle trespassing, or curbside pickup.
NVIDIA GPUs power Vision AI
Spatial analysis runs at the edge, and soon, in the Azure cloud. When deployed on the edge, the vision AI models run on devices with the NVIDIA T4 Tensor Core GPU, as well as the NVIDIA A2 Tensor Core GPU. The power of NVIDIA GPUs enables processing video frames at high frame rate for up to 20 video streams on a single GPU unit. The recommended edge devices are Azure Stack Edge Pro with the NVIDIA T4 Tensor Core GPU, or Azure Stack HCI devices with NVIDIA GPUs.
When running in the cloud, the video stream is processed at the edge for detection of motion on low-compute devices and then streamed to the Azure cloud for running the vision AI models for person and vehicle detection on Azure VMs powered by NVIDIA GPUs. Many of these vision models come from different frameworks (such as PyTorch and TensorFlow). These can be converted to the Open Neural Network Exchange (ONNX) format, which is the open standard format representing AI and deep learning models for further optimizations. ONNX Runtime is a high-performance inference engine to run machine learning models, with multi-platform support and a flexible execution provider interface to integrate hardware-specific libraries. ONNX Runtime integrates TensorRT as one execution provider for model inference acceleration on NVIDIA GPUs by harnessing the TensorRT optimizations. Based on the TensorRT capability, ONNX Runtime partitions the model graph and offloads the parts that TensorRT supports to TensorRT execution provider for efficient model execution on NVIDIA hardware.
Responsible AI and Innovation
Microsoft is releasing computer vision for Spatial analysis together with responsible deployment guidance grounded in user and societal research.
Microsoft developed the responsible deployment recommendations by applying many of the responsible innovation best practices in collaboration with customers to uncover deployment recommendations for Spatial analysis in accordance with Microsoft Responsible AI Principles: fairness, reliability and safety, privacy and security, inclusiveness, transparency and human accountability.
Microsoft’s principled approach enables developers to build rich solutions while upholding human dignity and the needs of everyone impacted by the technology.
Get Started Today
Learn more with computer vision for Spatial analysis documentation
Watch session topics featured in this blog
Get started with Microsoft Learn to build skills
Explore Azure announcements at Microsoft Ignite