Are You Alive: Enhancing Azure AI Vision Face API with Liveness Detection

We are excited to announce the public preview of Liveness Detection, an addition to the existing Azure AI Face API service. Facial recognition technology has been a longstanding method for verifying a user's identity in device and online account login for its convenience and efficiency. However, the system encounters escalating challenges as malicious actors persist in their attempts to manipulate and deceive the system through various spoofing techniques. This issue is expected to intensify with the emergence of generative AIs such as DALL-E and ChatGPT. Many online services such as LinkedIn and Microsoft Entra, now support “Verified Identities” that attest to there being a real human behind the identity. 

Face Liveness Detection 

Liveness detection aims to verify that the system engages with a physically present, living individual during the verification process. This is achieved by differentiating between a real (live) and fake (spoof) representation which may include photographs, videos, masks, or other means to mimic a real person. 

Face Liveness Detection has been an integrated part of Windows Hello Face for nearly a decade. "Every day, people around the world get access to their device in a convenient, personal and secure way using Windows Hello face recognition", said Katharine Holdsworth, Principal Group Program Manager, Windows Security. The new Face API liveness detection is a combination of mobile SDK and Azure AI service. It is conformant to ISO/IEC 30107-3 PAD (Presentation Attack Detection) standards as validated by iBeta level 1 and level 2 conformance testing. It successfully defends against a plethora of spoof types ranging from paper printouts, 2D/3D masks, and spoof presentations on phones and laptops. Liveness detection is an active area of research, with continuous improvements being made to counteract increasingly sophisticated spoofing attacks over time, and continuous improvement will be rolled out to the client and the service components as the overall solution gets hardened against new types of attacks over time. 

A simple illustration of the user experience from the SDK sample is shown below, where the screen background goes from white to black before the final liveness detection result is shown on the last page.

The following video demonstrates how the product works under different spoof conditions.  

While blocking spoof attacks is the primary focus of the liveness solution, paramount importance is also given to allowing real users to successfully pass the liveness check with low friction. Additionally, the liveness solution complies with the comprehensive responsible AI and data privacy standards to ensure fair usage across demographics around the world through extensive fairness testing. 

For more information, please visit: Empowering responsible AI practices | Microsoft AI. 

Implementation 

The overall liveness solution integration involves 2 different components: a mobile application and an app server/orchestrator. 

Prepare mobile application to perform liveness detection 

The newly released Azure AI Vision Face Client SDK for mobile (iOS and Android) allows for ease-of-integration of the liveness solution on front-end applications. To get started you would need to apply for the Face API Limited Access Features to get access to the SDK. For more information, see the Face Limited Access page. 

Once you have access to the SDK, follow the instructions and samples included in the github-azure-ai-vision-sdk to integrate the UI and the code needed into your native mobile application. The SDK supports both Java/Kotlin for Android and Swift for iOS mobile applications. 

Upon integration into your application, the SDK will handle starting the camera, guide the end-user to adjust their position, compose the liveness payload and then call the Face API to process the liveness payload. 

Orchestrating the liveness solution 

The high-level steps involved in the orchestration are illustrated below: 

For more details, refer to the tutorial. 

Face Liveness Detection with Face Verification 

Integrating face verification with liveness detection enables biometric verification of a specific person of interest while ensuring an additional layer of assurance that the individual is physically present. 

For more details, refer to the tutorial.