New innovations to protect custom AI applications with Defender for Cloud

Today’s blog post introduced new capabilities to enhance AI security and governance across multi-model and multi-cloud environments. This follow-on blog post dives deeper into how Microsoft Defender for Cloud can help organizations protect their custom-built AI applications.

 

The AI revolution has been transformative for organizations, driving them to integrate sophisticated AI features and products into their existing systems to maintain a competitive edge. However, this rapid development often outpaces their ability to establish adequate security measures for these advanced applications. Moreover, traditional security teams frequently lack the visibility and actionable insights needed, leaving organizations vulnerable to increasingly sophisticated attacks and struggling to protect their AI resources.

To address these challenges, we are excited to announce the general availability (GA) of threat protection for AI services, a capability that enhances threat protection in Microsoft Defender for Cloud. Starting May 1, 2025, the new Defender for AI Services plan will support models in Azure AI and Azure OpenAI Services.

 

 

“Security is paramount at Icertis. That’s why we've partnered with Microsoft to host our Contract Intelligence platform on Azure, fortified by Microsoft Defender for Cloud. As large language models (LLMs) became mainstream, our Icertis ExploreAI Service leveraged generative AI and proprietary models to transform contract management and create value for our customers.

Microsoft Defender for Cloud emerged as our natural choice for the first line of defense against AI-related threats. It meticulously evaluates the security of our Azure OpenAI deployments, monitors usage patterns, and promptly alerts us to potential threats. These capabilities empower our Security Operations Center (SOC) teams to make more informed decisions based on AI detections, ensuring that our AI-driven contract management remains secure, reliable, and ahead of emerging threats.” 

Subodh Patil, Principal Cyber Security Architect at Icertis

 

With these new threat protection capabilities, security teams can:

• Monitor suspicious activity in Azure AI resources, abiding by security frameworks like the OWASP Top 10 threats for LLM applications to defend against attacks on AI applications, such as direct and indirect prompt injections, wallet abuse, suspicious access to AI resources, and more.
• Triage and act on detections using contextual and insightful evidence, including prompt and response evidence, application and user context, grounding data origin breadcrumbs, and Microsoft Threat Intelligence details.
• Gain visibility from cloud to code (right to left) for better posture discovery and remediation by translating runtime findings into posture insights, like smart discovery of grounding data sources. Requires Defender CSPM posture plan to be fully utilized.
• Leverage frictionless onboarding with one-click, agentless enablement on Azure resources. This includes native integrations to Defender XDR, enabling advanced hunting and incident correlation capabilities.

Detect and protect against AI threats

Defender for Cloud helps organizations secure their AI applications from the latest threats. It identifies vulnerabilities and protects against sophisticated attacks, such as jailbreaks, invisible encodings, malicious URLs, and sensitive data exposure. It also protects against novel threats like ASCII smuggling, which could otherwise compromise the integrity of their AI applications. Defender for Cloud helps ensure the safety and reliability of critical AI resources by leveraging signals from prompt shields, AI analysis, and Microsoft Threat Intelligence. This provides comprehensive visibility and context, enabling security teams to quickly detect and respond to suspicious activities.

Figure 1 - An ASCII smuggling detection with relevant prompt evidence.

Prompt analysis-based detections aren’t the full story. Detections are also designed to analyze the application and user behavior to detect anomalies and suspicious behavior patterns. Analysts can leverage insights into user context, application context, access patterns, and use Microsoft Threat Intelligence tools to uncover complex attacks or threats that escape prompt-based content filtering detectors. For example, wallet attacks are a common threat where attackers aim to cause financial damage by abusing resource capacity. These attacks often appear innocent because the prompts' content looks harmless. However, the attacker's intention is to exploit the resource capacity when left unconstrained. While these prompts might go unnoticed as they don't contain suspicious content, examining the application's historical behavior patterns can reveal anomalies and lead to detection.

Figure 2 - A wallet abuse detection, showcasing an extreme anomaly in usage pattern in an AI application.

Respond and act on AI detections effectively

The lack of visibility into AI applications is a real struggle for security teams. The detections contain evidence that is hard or impossible for most SOC analysts to access.

For example, in the below credential exposure detection, the user was able to solicit secrets from the organizational data connected to the Contoso Outdoors chatbot app.

How would the analyst go about understanding this detection?

The detection evidence shows the user prompt and the model response (secrets are redacted). The evidence also explicitly calls out what kind of secret was exposed. The prompt evidence of this suspicious interaction is rarely stored, logged, or accessible anywhere outside the detection. The prompt analysis engine also tied the user request to the model response, making sense of the interaction.

Figure 3 - A credential exposure alert about Contoso Outdoors AI application, showing application context, the user prompt, and model response as well as the type of credential exposed.

What is most helpful in this specific detection is the application and user context. The application name instantly assists the SOC in determining if this is a valid scenario for this application. Contoso Outdoors chatbot is not supposed to access organizational secrets, so this is worrisome.

Next, the user context reveals who was exposed to the data, through what IP (internal or external) and their supposed intention. Most AI applications are built behind AI gateways, proxies, or Azure API Management (APIM) instances, making it challenging for SOC analysts to obtain these details through conventional logging methods or network solutions. Defender for Cloud addresses this issue by using a straightforward approach that fetches these details directly from the application’s API request to Azure AI.

Now, the analyst can reach out to the user (internal) or block (external) the identity or the IP.

Figure 4 - A credential exposure alert about Contoso Outdoors AI application, showing user context details of IP and identity.

Finally, to resolve this incident, the SOC analyst intends to remove and decommission the secret to mitigate the impact of the exposure. The final piece of evidence presented reveals the origin of the exposed data. This evidence substantiates the fact that the leak is genuine and originates from internal organizational data. It also provides the analyst with a critical breadcrumb trail to successfully remove the secret from the data store and communicate with the owner on next steps.

Figure 5 - A credential exposure alert about Contoso Outdoors AI application, showing details of the data store used to ground this specific model response.

Trace the invisible lines between your AI application and the grounding sources

Defender for Cloud excels in continuous feedback throughout the application lifecycle. While posture capabilities help triage detections, runtime protection provides crucial insights from traffic analysis, such as discovering data stores used for grounding AI applications. The AI application's connection to these stores is often hidden from current control or data plane tools. The credential leak example provided a real-world connection that was then integrated into our resource graph, uncovering previously overlooked data stores. Tagging these stores improves attack path and risk factor identification during posture scanning, ensuring safe configuration. This approach reinforces the feedback loop between runtime protection and posture assessment, maximizing cloud-native application protection platform (CNAPP) effectiveness.

Figure 6 - The cloud security explorer in Defender portal, showing evidence that the specific Search service is being used to ground the Contoso Outdoors AI application and tagged with the relevant insight “Used for AI grounding”.

Align with AI security frameworks

Our guiding principle is widely recognized by OWASP Top 10 for LLMs. By combining our posture capabilities with runtime monitoring, we can comprehensively address a wide range of threats, enabling us to proactively prepare for and detect AI-specific breaches with Defender for Cloud.

As the industry evolves and new regulations emerge, frameworks such as OWASP, the EU AI Act, and NIST 600-1 are shaping security expectations. Our detections are aligned with these frameworks as well as the MITRE ATLAS framework, ensuring that organizations stay compliant and are prepared for future regulations and standards.

Get started with threat protection for AI services

To get started with threat protection capabilities in Defender for Cloud, it’s as simple as one-click to enable it on your relevant subscription in Azure. The integration is agentless and requires zero intervention in the application dev lifecycle. More importantly, the native integration directly inside Azure AI pipeline does not entail scale or performance degradation in the application runtime.

Consuming the detections is easy, it appears in Defender for Cloud’s portal, but is also seamlessly connected to Defender XDR and Sentinel, leveraging the existing connectors. SOC analysts can leverage the correlation and analysis capabilities of Defender XDR from day one.

Figure 7 - A multistage incident in Defender XDR, showing an attack that included jailbreak, credential exposure, and wallet abuse.

Explore these capabilities today with a free 30-day trial*. You can leverage your existing AI application and simply enable the “AI workloads” plan on your chosen subscription to start detecting and responding to AI threats.

*Trial free period is limited to up to 75B tokens scanned.

Learn more about the innovations designed to help your organization protect data, defend against cyber threats, and stay compliant. Join Microsoft leaders online at Microsoft Secure on April 9.

Explore additional resources

• Learn more about Runtime protection
• Learn more about Posture capabilities
• Watch the Defender for Cloud in the Field episode on securing AI applications
• Get started with Defender for Cloud