Accelerating the Anomalous Sign-Ins detection with Microsoft Entra ID and Security Copilot

Overview

In today’s complex threat landscape, identity protection is critical for securing organizational assets. A common sign of compromise is user activity indicating connections from multiple locations separated by over X- kilometers within a short period. Such events might represent risky sign-ins, requiring Security Analysts to determine whether they are true positives (indicating malicious activity) or false positives (such as misconfigured settings or benign anomalies).

To enhance efficiency and accelerate the investigation process, organizations can leverage AI tools like Microsoft Security Copilot. By integrating Security Copilot with Microsoft Entra ID mainly AADUserRiskEvent and developing custom Promptbooks, organizations can investigate risky sign-ins, reduce manual workloads, and enable proactive decision-making to boost SOC efficiency in such scenarios.

Use Case: Challenge and Solution

Challenge

Organizations face significant challenges in investigating and triaging identity protection alerts for sign-in anomalies, especially when users appear to log in from geographically disparate locations within hours. These challenges include:

• Volume of Alerts: Large organizations generate numerous risky sign-in events daily.
• False Positives: Legitimate activities, such as VPN connections or device relocations, may be flagged.
• Resource Constraints: Security teams must efficiently prioritize true positives for investigation.

Solution

Using Microsoft Security Copilot with a tailored Promptbook, Security Analysts can automate the initial triage process and focus on meaningful insights. This approach combines data querying, AI-driven analysis, and actionable recommendations to improve investigation workflows.

Figure 1: Custom promptbook structure

Promptbook Structure

The custom Promptbook comprises two key prompts:

1. First Prompt: Data Retrieval from Defender XDR via KQL Query

This query retrieves users flagged for risky sign-ins within a 1-day window, focusing on events where the distance between locations exceeds 500 kilometers within 3 hours as example.

Retrieve Defender XDR information using this KQL query: let riskyusers = AADUserRiskEvents | where TimeGenerated is greater than or equal ago(<TimeIntervalByDays>) | project UserPrincipalName, TimeGenerated, Location, IpAddress, RiskState, Id, RiskEventType; riskyusers | join kind=inner ( riskyusers | extend TimeGenerated1 = TimeGenerated, LocationDetails1 = Location ) on UserPrincipalName | where TimeGenerated is less than TimeGenerated1 and datetime_diff('hour', TimeGenerated1, TimeGenerated) is less than or equal <ConnectionsInterbalByHrs> | extend latyy = Location.geoCoordinates.latitude | extend longy= Location.geoCoordinates.longitude | extend latyy1 = LocationDetails1.geoCoordinates.latitude | extend longy1 = LocationDetails1.geoCoordinates.longitude | extend distance = geo_distance_2points(todouble(Location.geoCoordinates.latitude), todouble(Location.geoCoordinates.longitude), todouble(LocationDetails1.geoCoordinates.latitude), todouble(LocationDetails1.geoCoordinates.longitude)) | where distance is greater than or equal <SepratedDistanceByKM> | summarize arg_max(TimeGenerated, *) by Id | where RiskState is not equal @"dismissed" | project UserPrincipalName, TimeGenerated, IpAddress, Location, TimeGenerated1, IpAddress1, LocationDetails1, RiskEventType, distance

Please make sure to set value for the following input parameters:

<TimeIntervalByDays> example: 7d

<ConnectionsInterbalByHrs> example: 3

<SepratedDistanceByKM> example: 5000

2. Second Prompt: AI Analysis for Patterns and Recommendations

This prompt enables Security Copilot to analyze the retrieved data, identify patterns (e.g., recurring IP addresses or anomalous locations), and suggest further investigative steps and mitigative actions.

/AnalyzeSecurityData Provide your insights as Security Analyst about what anomalies or similarity patterns can you identify. Provide a list of prompts for Security Copilot to investigate further and a list of recommendations. Use as input security data the information in the table from the previous prompt in this session.

Figure 2: Prompting example

Figure 3: Prompting insights and recommendations

Automating the Process with Azure Logic Apps

Organizations can further streamline the process by automating risky sign-in investigations using Azure Logic Apps. Here’s how:

1. Create a Logic App: Set up a Logic App in the Azure portal.
2. Trigger Configuration: Use a recurring schedule trigger to run the investigation daily.
3. Integration with Security Copilot:
• Configure the Logic App to execute the Security Copilot’s Promptbook.
• Automate prompts for insights and recommendations.
4. Notification Mechanism: Send results via email to the SOC team or log them in a ticketing system for further action.

Figure 4: Logic app designer

Note: to send only the result of the last prompt in the promptbook, use: last(body('Run_a_Security_Copilot_promptbook')?['evaluationResults'])['evaluationResultContent']

Figure 5: Notification mechanism

Benefits of the Approach

• Efficiency: Reduces manual efforts by automating repetitive tasks.
• Accuracy: AI analysis helps filter out false positives and prioritize true positives.
• Scalability: Easily extendable for other security use cases.
• Fast triage: Enables SOC teams to act quickly and decisively.

Conclusion

Incorporating Microsoft Security Copilot with a custom Promptbook into daily operations empowers Security Analysts to efficiently investigate and triage risky sign-in events. By automating processes through Azure Logic Apps, organizations can maintain a proactive security posture and better protect their identities and assets.

Try it out: If your organization is looking to enhance its SOC capabilities, consider implementing this solution to harness the power of AI for identity protection. The Promptbook added to the github Security Copilot repo : Click here