Ask Microsoft Anything: Purview Data Security Investigations Part 2
Microsoft Purview Data Security Investigations is now generally available! Data Security Investigations enables customers to quickly uncover and mitigate data security and sensitive data risks buried in their data using AI‑powered deep content analysis—both proactively and reactively. With Data Security Investigations, security teams can identify investigation-relevant data, analyze it at scale with AI, and mitigate uncovered risks in a single unified solution. By streamlining complex, time‑consuming investigative workflows, organizations can move from signal to insight in hours rather than weeks or months. Whether you're responding to an active data security incident or proactively assessing data exposure, DSI gives data security teams the clarity, speed, and confidence to investigate data risk in today's threat landscape. Join us for an AMA with the team that developed Microsoft Purview's newest solution to go over new features, our refined business model and more! What is an AMA? An 'Ask Microsoft Anything' (AMA) session is an opportunity for you to engage directly with Microsoft employees! This AMA will consist of a short presentation followed by taking questions on-camera from the comment section down below! Ask your questions/give your feedback and we will have our awesome Microsoft Subject Matter Experts engaging and responding directly in the video feed. We know this timeslot might not work for everyone, so feel free to ask your questions at any time leading up to the event and the experts will do their best to answer during the live hour. This page will stay up so come back and use it as a resource anytime. We hope you enjoy!Live AMA: Defining AI boundaries with data sensitivity
As AI becomes embedded in everyday work, traditional data security models break down. Copilots and agents can search, summarize, and recombine information at machine speed, creating new exposure paths for sensitive data — even when nothing is formally shared or exfiltrated. In this session, we’ll explain why data sensitivity, not data location, is now the true security boundary, and what that shift means for protecting information in the age of AI. We’ll walk through how organizations can establish a shared understanding of what data is sensitive, use sensitivity labels to consistently define how that data should be handled, and automatically enforce protections wherever data is created or used — including in AI experiences. We’ll close with a live Ask Me Anything (AMA), where you can bring real-world questions about securing Copilot and agents, scaling classification and labeling, and turning sensitivity into consistent, enforceable controls with Microsoft Purview.
Optimizing Microsoft Sentinel: Resolving AMA-Induced Syslog & CEF Duplicates
2) Recommended Solutions When collecting both Syslog and CEF logs from the same Linux collector using the Azure Monitor Agent (AMA) in Microsoft Sentinel, duplicate log entries can occur. These duplicates arise because the same event may be ingested through both the Syslog and CEF pipelines, leading to redundancy in the Log Analytics Workspace (LAW). The following solutions aim to eliminate or reduce duplicate log ingestion, ensuring that: CEF events are parsed correctly and only once. Syslog data remains clean and non-redundant. Storage and analytics efficiency is improved. Alerting and incident investigation are not skewed by duplicate entries. Each option provides a different strategy based on your environment’s flexibility and configuration capabilities—from facility-level separation, to ingestion-time filtering, to daemon-side log routing. Option 1: Facility Separation (Preferred) Configure devices to emit CEF logs on a dedicated facility (for example, 'local4'), and adjust the Data Collection Rules (DCRs) so that the CEF stream includes only that facility, while the Syslog stream excludes it. This ensures CEF events are parsed once into 'CommonSecurityLog' and never land in 'Syslog'. CEF via AMA DCR (include only CEF facility): { "properties": { "dataSources": { "syslog": [ { "streams": ["Microsoft-CommonSecurityLog"], "facilityNames": ["local4"], "logLevels": ["*"], "name": "cefDataSource" } ] }, "dataFlows": [ { "streams": ["Microsoft-CommonSecurityLog"], "destinations": ["laDest"] } ] } } Syslog via AMA DCR (exclude CEF facility): { "properties": { "dataSources": { "syslog": [ { "streams": ["Microsoft-Syslog"], "facilityNames": [ "auth","authpriv","cron","daemon","kern","mail", "syslog","user","local0","local1","local2","local3", "local5","local6","local7" ], "logLevels": ["*"], "name": "syslogDataSource" } ] }, "dataFlows": [ { "streams": ["Microsoft-Syslog"], "destinations": ["laDest"] } ] } } Option 2: Ingest-time Transform (Drop CEF from Syslog) If facility separation is not feasible, apply a transformation to the Syslog stream in the DCR so that any CEF-formatted messages are dropped during ingestion. Syslog stream transformKql: { "properties": { "dataFlows": [ { "streams": ["Microsoft-Syslog"], "transformKql": "source | where not(SyslogMessage startswith 'CEF:')", "destinations": ["laDest"] } ] } } Option 3: Daemon-side Filtering/Rewriting (rsyslog/syslog-ng) Filter or rewrite CEF messages before AMA sees them. For example, route CEF messages to a dedicated facility using syslog-ng and stop further processing: # Match CEF filter f_cef { message("^CEF:"); }; # Send CEF to local5 and stop further processing log { source(s_src); filter(f_cef); rewrite { set_facility(local5); }; destination(d_azure_mdsd); flags(final); } 3) Verification Steps with KQL Queries Detect CEF messages that leaked into Syslog: Syslog | where TimeGenerated > ago(1d) | where SyslogMessage startswith "CEF:" | summarize count() by Computer | order by count_ desc Estimate duplicate count across Syslog and CommonSecurityLog: let sys = Syslog | where TimeGenerated > ago(1d) | where SyslogMessage startswith "CEF:" | extend key = hash_sha256(SyslogMessage); let cef = CommonSecurityLog | where TimeGenerated > ago(1d) | extend key = hash_sha256(RawEvent); cef | join kind=innerunique (sys) on key | summarize duplicates = count() Note : You should identify the RawEvent that might be causing the duplicates. 3.1) Duplicate Detection Query Explained This query helps quantify duplicate ingestion when both Syslog and CEF connectors ingest the same events. It works as follows: Build the Syslog set (sys): Filter the 'Syslog' table for the last day and keep only messages that start with 'CEF:'. Compute a SHA-256 hash of the entire message as a stable join key ("key"). Build the CEF set (cef): Filter the 'CommonSecurityLog' table for the last day and compute a SHA-256 hash of the 'RawEvent' field as the same-style join key. Join on the key: Use 'join kind=innerunique' to find messages that exist in both sets (i.e., duplicates). Summarize: Count the number of matching rows to get a duplicate total. 4) Common Pitfalls - Overlapping DCRs applied to the same collector VM causing overlapping facilities/severities. - CEF and Syslog using the same facility on sources, leading to ingestion on both streams. - rsyslog/syslog-ng filters placed after AMA’s own configuration include (ensure your custom rules run before '10-azuremonitoragent.conf'). 5) References - Microsoft Learn: Ingest syslog and CEF messages to Microsoft Sentinel with AMA (https://learn.microsoft.com/en-us/azure/sentinel/connect-cef-syslog-ama)Ask Microsoft Anything: Security Service Edge (SSE)
Ask Microsoft Anything about securing access to any app or resource, from anywhere. Our panel of experts will answer you questions about Microsoft Entra Internet Access, Microsoft Entra Private Access, and how these products work together as part of Microsoft's Security Service Edge solution. This session is part of the Microsoft Entra Suite Tech Accelerator. Get a head start Watch Zero Trust in the Age of AI to learn how to simplify your Zero Trust strategy with the latest end-to-end security innovations.Ask Microsoft Anything: Discover the Microsoft Entra Suite
Ask Microsoft Anything…about the Microsoft Entra Suite! Join our panel of experts with your questions about unifying Conditional Access policies for identities and networks, and ensuring least privilege access for all users accessing all resources and apps, including AI. Want to improve the user experience for both in-office and remote workers while reducing the complexity and cost of managing security tools? We're here to help with tips! This session is part of the Microsoft Entra Suite Tech Accelerator. Get a head start Watch Zero Trust in the Age of AI to learn how to simplify your Zero Trust strategy with the latest end-to-end security innovations.AMA: Security for AI
Looking for tips on how to prepare your environment for secure AI adoption? What’s the best way to protect your AI stack and sensitive data? Join this Ask Microsoft Anything session to get answers to your questions from the teams adopting security for AI at Microsoft! Big questions or small ones—-we’re here to help you confidently embrace the age of AI with industry-leading cybersecurity and compliance solutions. This session is part of Tech Community Live: Microsoft Security edition.AMA: Microsoft Security Exposure Management
If you are responsible for maintaining a strong security posture in your organization, don’t miss this Ask Microsoft Anything session! Microsoft Security Exposure Management can help you get a unified view of attack surface across your organization, manage and investigate attack paths, manage exposure, and better prioritize and safeguard critical assets. Check out our Tech Community blogs aka.ms/exposuremanagement/blogs and bring your questions about getting ahead of attackers by using and getting the most from Exposure Management. This session is part of Tech Community Live: Microsoft Security edition.AMA: Security Service Edge (SSE)
Join us to explore Microsoft’s SSE (Global Secure Access) partner ecosystem, where we collaborate with top industry leaders to deliver integrated, identity-centric solutions for enhanced security and seamless connectivity. Ask Microsoft Anything and learn how these partnerships are simplifying security and networking, empowering you to support your hybrid workforce effectively. This session is part of Tech Community Live: Microsoft Security edition.AMA: Microsoft Defender for Cloud
Ask Microsoft Anything…about Microsoft Defender for Cloud and get your questions answered! Join our panel of experts to discuss our latest innovations announced at Microsoft Ignite, including Endor Labs integration for supply chain security, multiple posture management enhancements, and new container security features. This session is part of Tech Community Live: Microsoft Security edition.AMA: Microsoft Entra Suite
Join our panel of experts with your questions about the Microsoft Entra Suite. Want to secure access for your employees and extend Conditional Access to all apps – including on-premises? Want to retire old VPNs and automate identity lifecycle workflows? We’re here to share best practices and insights about how our identity and network access products can help. Ask Microsoft Anything! This session is part of Tech Community Live: Microsoft Security edition.
