Introducing the New Microsoft Sentinel Logstash Output Plugin (Public Preview!)
Many organizations rely on Logstash as a flexible, trusted data pipeline for collecting, transforming, and forwarding logs from on-premises and hybrid environments. Microsoft Sentinel has long supported a Logstash output plugin, enabling customers to send data directly into Sentinel as part of their existing pipelines. The original plugin was implemented in Ruby, and while it has served its purpose, it no longer meets Microsoft’s Secure Future Initiative (SFI) standards and has limited engineering support. To address both security and sustainability, we have rebuilt the plugin from the ground up in Java, a language that is more secure, better supported across Microsoft, and aligned with long-term platform investments. To ensure a seamless transition, the new implementation is still packaged and distributed as a standard Logstash Ruby gem. This means the installation and usage experience remains unchanged for customers, while benefiting from a more secure and maintainable foundation. What's New in This Version Java‑based and SFI‑compliant Same Logstash plugin experience, now rebuilt on a stronger foundation. The new implementation is fully Java‑based, aligning with Microsoft’s Secure Future Initiative (SFI) and providing improved security, supportability, and long-term maintainability. Modern, DCR‑based ingestion The plugin now uses the Azure Monitor Logs Ingestion API with Data Collection Rules (DCRs), replacing the legacy HTTP Data Collection API (For more info, see Migrate from the HTTP Data Collector API to the Log Ingestion API - Azure Monitor | Microsoft Learn). This gives customers full schema control, enables custom log tables, and supports ingestion into standard Microsoft Sentinel tables as well as Microsoft Sentinel data lake. Flexible authentication options Authentication is automatically determined based on your configuration, with support for: Client secret (App registration / service principal) Managed identity, eliminating the need to store credentials in configuration files Sovereign cloud support: The plugin supports Azure sovereign clouds, including Azure US Government, Azure China, and Azure Germany. Standard Logstash distribution model The plugin is published on RubyGems.org, the standard distribution channel for Logstash plugins, and can be installed directly using the Logstash plugin manager, no change to your existing installation workflow. What the Plugin Does Logstash plugin operates as a three-stage data pipeline: Input → Filter → Output. Input: You control how data enters the pipeline, using sources such as syslog, filebeat, Kafka, Event Hubs, databases (via JDBC), files, and more. Filter: You enrich and transform events using Logstash’s powerful filtering ecosystem, including plugins like grok, mutate, and Json, shaping data to match your security and operational needs. Output: This is where Microsoft comes in. The Microsoft Sentinel Logstash Output Plugin securely sends your processed events to an Azure Monitor Data Collection Endpoint, where they are ingested into Sentinel via a Data Collection Rule (DCR). With this model, you retain full control over your Logstash pipeline and data processing logic, while the Sentinel plugin provides a secure, reliable path to ingest data into Microsoft Sentinel. Getting Started Prerequisites Logstash installed and running An Azure Monitor Data Collection Endpoint (DCE) and Data Collection Rule (DCR) in your subscription Contributor role on your Log Analytics workspace Who Is This For? Organizations that already have Logstash pipelines, need to collect from on-premises or legacy systems, and operate in distributed/hybrid environments including air-gapped networks. To learn more, see: microsoft-sentinel-log-analytics-logstash-output-plugin | RubyGems.org | your community gem hostMicrosoft Sentinel data lake FAQ
Microsoft Sentinel data lake (generally available) is a purpose‑built, cloud‑native security data lake. It centralizes all security data in an open format, serving as the foundation for agentic defense, enhanced security insights, and graph-based enrichment. It offers cost‑effective ingestion, long‑term retention, and advanced analytics. In this blog we offer answers to many of the questions we’ve heard from our customers and partners. General questions What is the Microsoft Sentinel data lake? Microsoft has expanded its industry-leading SIEM solution, Microsoft Sentinel, to include a unified, security data lake, designed to help optimize costs, simplify data management, and accelerate the adoption of AI in security operations. This modern data lake serves as the foundation for the Microsoft Sentinel platform. It has a cloud-native architecture and is purpose-built for security—bringing together all security data for greater visibility, deeper security analysis, contextual awareness and agentic defense. It provides affordable, long-term retention, allowing organizations to maintain robust security while effectively managing budgetary requirements. What are the benefits of Sentinel data lake? Microsoft Sentinel data lake is purpose built for security offering flexible analytics, cost management, and deeper security insights. Sentinel data lake: Centralizes security data delta parquet and open format for easy access. This unified data foundation accelerates threat detection, investigation, and response across hybrid and multi-cloud environments. Enables data federation by allowing customers to access data in external sources like Microsoft Fabric, ADLS and Databricks from the data lake. Federated data appears alongside native Sentinel data, enabling correlated hunting, investigation, and custom graph analysis across a broader digital estate. Offers a disaggregated storage and compute pricing model, allowing customers to store massive volumes of security data at a fraction of the cost compared to traditional SIEM solutions. Allows multiple analytics engines like Kusto, Spark, and ML to run on a single data copy, simplifying management, reducing costs, and supporting deeper security analysis. Integrates with GitHub Copilot and VS Code empowering SOC teams to automate enrichment, anomaly detection, and forensic analysis. Supports AI agents via the MCP server, allowing tools like GitHub Copilot to query and automate security tasks. The MCP Server layer brings intelligence to the data, offering Semantic Search, Query Tools, and Custom Analysis capabilities that make it easier to extract insights and automate workflows. Provides streamlined onboarding, intuitive table management, and scalable multi-tenant support, making it ideal for MSSPs and large enterprises. The Sentinel data lake is designed for security workloads, ensuring that processes from ingestion to analytics meet evolving cybersecurity requirements. Is Microsoft Sentinel SIEM going away? No. Microsoft is expanding Sentinel into an AI powered end-to-end security platform that includes SIEM and new platform capabilities - Security data lake, graph-powered analytics and MCP Server. SIEM remains a core component and will be actively developed and supported. Getting started What are the prerequisites for Sentinel data lake? To get started: Connect your Sentinel workspace to Microsoft Defender prior to onboarding to Sentinel data lake. Once in the Defender experience see data lake onboarding documentation for next steps. Note: Sentinel is moving to the Microsoft Defender portal and the Sentinel Azure portal will be retired by March 31, 2027. I am a Sentinel-only customer, and not a Defender customer. Can I use the Sentinel data lake? Yes. You must connect Sentinel to the Defender experience before onboarding to the Sentinel data lake. Microsoft Sentinel is generally available in the Microsoft Defender portal, with or without Microsoft Defender XDR or an E5 license. If you have created a log analytics workspace, enabled it for Sentinel and have the right Microsoft Entra roles (e.g. Global Administrator + Subscription Owner, Security Administrator + Sentinel Contributor), you can enable Sentinel in the Defender portal. For more details on how to connect Sentinel to Defender review these sources: Microsoft Sentinel in the Microsoft Defender portal In what regions is Sentinel data lake available? For supported regions see: Geographical availability and data residency in Microsoft Sentinel | Azure Docs. Is there an expected release date for Microsoft Sentinel data lake in GCC, GCC-H, and DoD? While the exact date is not yet finalized, we plan to expand Sentinel data lake to the US Government environments. . How will URBAC and Entra RBAC work together to manage the data lake given there is no centralized model? Entra RBAC will provide broad access to the data lake (URBAC maps the right permissions to specific Entra role holders: GA/SA/SO/GR/SR). URBAC will become a centralized pane for configuring non-global delegated access to the data lake. For today, you will use this for the “default data lake” workspace. In the future, this will be enabled for non-default Sentinel workspaces as well – meaning all workspaces in the data lake can be managed here for data lake RBAC requirements. Azure RBAC on the Log Analytics (LA) workspace in the data lake is respected through URBAC as well today. If you already hold a built-in role like log analytics reader, you will be able to run interactive queries over the tables in that workspace. Or, if you hold log analytics contributor, you can read and manage table data. For more details see: Roles and permissions in the Microsoft Sentinel platform | Microsoft Learn Data ingestion and storage How do I ingest data into the Sentinel data lake? To ingest data into the Sentinel data lake, you can use existing Sentinel data connectors or custom connectors to bring data from Microsoft and third-party sources. Data can be ingested into the analytics tier or the data lake tier. Data ingested into the analytics tier is automatically mirrored to the lake (at no additional cost). Alternatively, data that is not needed in the analytics tier can be ingested directly into the data lake. Data retention is configured directly in table management, for both analytics retention and data lake storage. Note: Certain tables do not support data lake-only ingestion via either API or data connector UI. See here for more information: Custom log tables. What is Microsoft’s guidance on when to use analytics tier vs. the data lake tier? Sentinel data lake offers flexible, built-in data tiering (analytics and data lake tiers) to effectively meet diverse business use cases and achieve cost optimization goals. Analytics tier: Is ideal for high-performance, real-time, end-to-end detections, enrichments, investigation and interactive dashboards. Typically, high-fidelity data from EDRs, email gateways, identity, SaaS and cloud logs, threat intelligence (TI) should be ingested into the analytics tier. Data in the analytics tier is best monitored proactively with scheduled alerts and scheduled analytics to enable security detections Data in this tier is retained at no cost for up to 90 days by default, extendable to 2 years. A copy of the data in this tier is automatically available in the data lake tier at no extra cost, ensuring a unified copy of security data for both tiers. Data lake tier: Is designed for cost-effective, long-term storage. High-volume logs like NetFlow logs, TLS/SSL certificate logs, firewall logs and proxy logs are best suited for data lake tier. Customers can use these logs for historical analysis, compliance and auditing, incident response (IR), forensics over historical data, build tenant baselines, TI matching and then promote resulting insights into the analytics tier. Customers can run full Kusto queries, Spark Notebooks and scheduled jobs over a single copy of their data in the data lake. Customers can also search, enrich and promote data from the data lake tier to the analytics tier for full analytics. For more details see documentation. What does it mean that a copy of all new analytics tier data will be available in the data lake? When Sentinel data lake is enabled, a copy of all new data ingested into the analytics tier is automatically duplicated into the data lake tier. This means customers don’t need to manually configure or manage this process, every new log or telemetry added to the analytics tier becomes instantly available in the data lake. This allows security teams to run advanced analytics, historical investigations, and machine learning models on a single, unified copy of data in the lake, while still using the analytics tier for real-time SOC workflows. It’s a seamless way to support both operational and long-term use cases—without duplicating effort or cost. What is the guidance for customers using data federation capability in Sentinel data lake? Starting April 1, 2026, federate data from Microsoft Fabric, ADLS, and Azure Databricks into Sentinel data lake. Use data federation when data is exploratory, infrequently accessed, or must remain at source due to governance, compliance, sovereignty, or contractual requirements. Ingest data directly into Sentinel to unlock full SIEM capabilities, always-on detections, advanced automation, and AI‑driven defense at scale. This approach lets security teams start where their data already lives — preserving governance, then progressively ingest data into Sentinel for full security value. Is there any cost for retention in the analytics tier? Analytics ingestion includes 90 days of interactive retention, at no additional cost. Simply set analytics retention to 90 days or less. Analytics retention beyond 90 days will incur a retention cost. Data can be retained longer within the data lake by using the “total retention” setting. This allows you to extend retention within the data lake for up to 12 years. While data is retained within the analytics tier, there is no charge for the mirrored data within the lake. Retaining data in the lake beyond the analytics retention period incurs additional storage costs. See documentation for more details: Manage data tiers and retention in Microsoft Sentinel | Microsoft Learn What is the guidance for Microsoft Sentinel Basic and Auxiliary Logs customers? If you previously enabled Basic or Auxiliary Logs plan in Sentinel: You can view Basic Logs in the Defender portal but manage it from the Log Analytics workspace. To manage it in the Defender portal, you must change the plan from Basic to Analytics. Once the table is transitioned to the analytics tier, if desired, it can then be transitioned to the data lake. Existing Auxiliary Log tables will be available in the data lake tier for use once the Sentinel data lake is enabled. Billing for these tables will automatically switch to the Sentinel data lake meters. Microsoft Sentinel customers are recommended to start planning their data management strategy with the data lake. While Basic and Auxiliary Logs are still available, they are not being enhanced further. Sentinel data lake offers more capabilities at a lower price point. Please plan on onboarding your security data to the Sentinel data lake. Azure Monitor customers can continue to use Basic and Auxiliary Logs for observability scenarios. What happens to customers that already have Archive logs enabled? If a customer has already configured tables for Archive retention, existing retention settings will not change and will be automatically inherited by the Sentinel data lake. All data, including existing data in archive retention will be billed using the data lake storage meter, benefiting from 6x data compression. However, the data itself will not move. Existing data in archive will continue to be accessible through Sentinel search and restore experiences: o Data will not be backfilled into the data lake. o Data will be billed using the data lake storage meter. New data ingested after enabling the data lake: o Will be automatically mirrored to the data lake and accessible through data lake explorer. o Data will be billed using the data lake storage meter. Example: If a customer has 12 months of total retention enabled on a table, 2 months after enabling ingestion into the Sentinel data lake, the customer will still have access to 10 months of archived data (through Sentinel search and restore experiences), but access to only 2 months of data in the data lake (since the data lake was enabled). Key considerations for customers that currently have Archive logs enabled: The existing archive will remain, with new data ingested into the data lake going forward; previously stored archive data will not be backfilled into the lake. Archive logs will continue to be accessible via the Search and Restore tab under Sentinel. If analytics and data lake mode are enabled on table, which is the default setting for analytics tables when Sentinel data lake is enabled, all new data will be ingested into the Sentinel data lake. There will only be one storage meter (which is data lake storage) going forward. Archive will continue to be accessible via Search and Restore. If Sentinel data lake-only mode is enabled on table, new data will be ingested only into the data lake; any data that’s not already in the Sentinel data lake won’t be migrated/backfilled. Only data that was previously ingested under the archive plan will be accessible via Search and Restore. What is the guidance for customers using Azure Data Explorer (ADX) alongside Microsoft Sentinel? Some customers might have set up ADX cluster for their DIY lake setup. Customers can choose to continue using that setup and gradually migrate to Sentinel data lake for new data that they want to manage. The lake explorer will support federation with ADX to enable the customers to migrate gradually and simplify their deployment. What happens to the Defender XDR data after enabling Sentinel data lake? By default, Defender XDR tables are available for querying in advanced hunting, with 30 days of analytics tier retention included with the XDR license. To retain data beyond this period, an explicit change to the retention setting is required, either by extending the analytics tier retention or the total retention period. You can extend the retention period of supported Defender XDR tables beyond 30 days and ingest the data into the analytics tier. For more information see Manage XDR data in Microsoft Sentinel. You can also ingest XDR data directly into the data lake tier. See here for more information. A list of XDR advanced hunting tables supported by Sentinel are documented here: Connect Microsoft Defender XDR data to Microsoft Sentinel | Microsoft Learn. KQL queries and jobs Is KQL and Notebook supported over the Sentinel data lake? Yes, via the data lake KQL query experience along with a fully managed Notebook experience which enables spark-based big data analytics over a single copy of all your security data. Customers can run queries across any time range of data in their Sentinel data lake. In the future, this will be extended to enable SQL query over lake as well. Note: Triggering a KQL job directly via an API or Logic App is not yet supported but is on the roadmap. Why are there two different places to run KQL queries in Sentinel experience? Advanced hunting queries both XDR and analytics tables, with compute cost included. Data lake explorer only queries data in the lake and incurs a separate compute cost. Consolidating advanced hunting and KQL explorer user interfaces is on the roadmap. This will provide security analysts a unified query experience across both analytics and data lake tiers. Where is the output from KQL jobs stored? KQL jobs are written into existing or new custom tables in the analytics tier. Is it possible to run KQL queries on multiple data lake tables? Yes, you can run KQL interactive queries and jobs using operators like join or union. Can KQL queries (either interactive or via KQL jobs) join data across multiple workspaces? Security teams can run multi-workspace KQL queries for broader threat correlation Pricing and billing How does a customer pay for Sentinel data lake? Billing is automatically enabled at the time of onboarding based on Azure Subscription and Resource Group selections. Customers are then charged based on the volume of data ingested, retained, and analyzed (e.g. KQL Queries and Jobs). See Sentinel pricing page for more details. 2. What are the pricing components for Sentinel data lake? Sentinel data lake offers a flexible pricing model designed to optimize security coverage and costs. At a high level, pricing is based on the volume of data ingested/processed, the volume of data retained, and the volume of data processed. For specific meter definitions, see documentation. 3. How does the business model for Sentinel SIEM change with the introduction of the data lake? There is no change to existing Sentinel analytics tier ingestion business model. Sentinel data lake has separate meters for ingestion, storage and analytics. 4. What happens to the existing Sentinel SIEM and related Azure Monitor billing meters when a customer onboards to Sentinel data lake? When a customer onboards to the Sentinel data lake, nothing changes with analytic ingestion or retention. Customers using data archive and Auxiliary Logs will automatically transition to the new data lake meters. How does data lake storage affect cost efficiency for high volume data retention? Sentinel data lake offers cost-effective, long-term storage with uniform data compression of 6:1 across all data sources, applicable only to data lake storage. Example: For 600GB of data stored, you are only billed for 100GB compressed data. This approach allows organizations to retain greater volumes of security data over extended periods cost-effectively, thereby reducing security risks without compromising their overall security posture. here How “Data Processing” billed? To support the ingestion and standardization of diverse data sources, the Data Processing feature applies a $0.10 per GB (US East) charge for all data ingested into the data lake. This feature enables a broad array of transformations like redaction, splitting, filtering and normalization. The data processing charge is applied per GB of uncompressed data Note: For regional pricing, please refer to the “Data processing” meter within the Microsoft Sentinel Pricing official documentation. Does “Data processing” meter apply to analytics tier data mirrored in the data lake? No. Data processing charge will not be applied to mirrored data. Data mirrored from the analytic tier is not subject to either data ingestion or processing charges. How is retention billed for tables that use data lake-only ingestion & retention? Sentinel data lake decouples ingestion, storage, and analytics meters. Customers have the flexibility to pay based on how data is retained and used. For tables that use data lake‑only ingestion, there is no included free retention—unlike the analytics tier, which includes 90 days of analytics retention. Retention charges begin immediately once data is stored in the data lake. Data lake storage billing is based on compressed data size rather than raw ingested volume, which significantly reduces storage costs and delivers lower overall retention spend for customers. Does data federation incur charges? Data federation does not generate any ingestion or storage fees in Sentinel data lake. Customers are billed only when they run analytics or queries on federated data, with charges based on Sentinel data lake compute and analytics meters. This means customers pay solely for actual data usage, not mere connectivity. How do I understand Sentinel data lake costs? Sentinel data lake costs driven by three primary factors: how much data is ingested, how long that data is retained, and how the data is used. Customers can flexibly choose to ingest data into the analytics tier or data lake tier, and these architectural choices directly impact cost. For example, data can be ingested into the analytics tier—where commitment tiers help optimize costs for high data volumes—or ingested data directly into the Sentinel data lake for lower‑cost ingestion, storage, and on‑demand analysis. Customers are encouraged to work with their Microsoft account team to obtain an accurate cost estimate tailored to their environment. See Sentinel pricing page to understand Sentinel pricing. How do I manage Sentinel data lake costs? Built-in cost management experiences help customers with cost predictability, billing transparency, and operational efficiency. Reports provide customers with insights into usage trends over time, enabling them to identify cost drivers and optimize data retention and processing strategies. Set usage-based alerts on specific meters to monitor and control costs. For example, receive alerts when query or notebook usage passes set limits, helping avoid unexpected expenses and manage budgets. See our Sentinel cost management documentation to learn more. If I’m an Auxiliary Logs customer, how will onboarding to the Sentinel data lake affect my billing? Once a workspace is onboarded to Sentinel data lake, all Auxiliary Logs meters will be replaced by new data lake meters. Do we charge for data lake ingestion and storage for graph experiences? Microsoft Sentinel graph-based experiences are included as part of the existing Defender and Purview licenses. However, Sentinel graph requires Sentinel data lake and specific data sources to build the underlying graph. Enabling these data sources will incur ingestion and data lake storage costs. Note: For Sentinel SIEM customers, most required data sources are free for analytics ingestion. Non-entitled sources such as Microsoft Entra ID logs will incur ingestion and data lake storage costs. How is Entra asset data and ARG data billed? Data lake ingestion charges of $0.05 per GB (US EAST) will apply to Entra asset data and ARG data. Note: This was previously not billed during public preview and is billed since data lake GA. To learn more, see: https://learn.microsoft.com/azure/sentinel/datalake/enable-data-connectors When a customer activates Sentinel data lake, what happens to tables with archive logs enabled? To simplify billing, once the data lake is enabled, all archive data will be billed using the data lake storage meter. This provides consistent long-term retention billing and includes automatic 6x data compression. For most customers, this change results in lower long‑term retention costs. However, customers who previously had discounted archive retention pricing will not automatically receive the same discounts on the new data lake storage meters. In these cases, customers should engage their Microsoft account team to review pricing implications before enabling the Sentinel data lake. Thank you Thank you to our customers and partners for your continued trust and collaboration. Your feedback drives our innovation, and we’re excited to keep evolving Microsoft Sentinel to meet your security needs. If you have any questions, please don’t hesitate to reach out—we’re here to support you every step of the way. Learn more: Get started with Sentinel data lake today: https://aka.ms/Get_started/Sentinel_datalake Microsoft Sentinel AI-ready platform: https://aka.ms/Microsoft_Sentinel Sentinel data lake videos: https://aka.ms/Sentineldatalake_videos Latest innovations and updates on Sentinel: https://aka.ms/msftsentinelblog Sentinel pricing page: https://aka.ms/MicrosoftSentinel_PricingMicrosoft Sentinel for SAP Agentless connector GA
Dear Community, Today is the day: Our new agentless connector for Microsoft Sentinel Solution for SAP applications is Generally Available now! Fully onboarded to SAP’s official Business Accelerator Hub and ready for prime time wherever your SAP systems are waiting – on-premises, hyperscalers, RISE, or GROW – to be protected. Let’s hear from an agentless customer: “With the Microsoft Sentinel Solution for SAP and its new agentless connector, we accelerated deployment across our SAP landscape without the complexity of containerized agents. This streamlined approach elevated our SOC’s visibility into SAP security events, strengthened our compliance posture, and enabled faster, more informed incident response” SOC Specialist, North American aviation company Use the video below to kick off your own agentless deployment today. #Kudos to the amazing mvigilante for showing us around the new connector! But we didn’t stop there! Security is being reengineered for the AI era - moving from static, rule-based controls to platform-driven, machine-speed defence that anticipates threats before they strike. Attackers think in graphs - Microsoft does too. We’re bringing relationship-aware context to Microsoft Security - so defenders and AI can see connections, understand the impact of a potential compromise (blast radius), and act faster across pre-breach and post-breach scenarios including SAP systems - your crown jewels. See it in action in below phishing-compromise which lead to an SAP login bypassing MFA with followed operating-system activities on the SAP host downloading trojan software. Enjoy this clickable experience for more details on the scenario. Shows how a phishing compromise escalated to an SAP MFA bypass, highlighting cross-domain correlation. The Sentinel Solution for SAP has AI-first in mind and directly integrates with our security platform on the Defender portal for enterprise-wide signal correlation, Security Copilot reasoning, and Sentinel Data Lake usage. Your real-time SAP detections operate on the Analytics tier for instant results and threat hunting, while the same SAP logs get mirrored to the lake for cost-efficient long-term storage (up to 12 years). Access that data for compliance reporting or historic analysis through KQL jobs on the lake. No more – yeah, I have the data stored somewhere to tick the audit report check box – but be able to query and use your SAP telemetry in long term storage at scale. Learn more here. Findings from the Agentless Connector preview During our preview we learned that majority of customers immediately profit from the far smoother onboarding experience compared to the Docker-based approach. Deployment efforts and time to first SAP log arrival in Sentinel went from days and weeks to hours. ⚠️ Deprecation notice for containerized data connector agent ⚠️ The containerised SAP data connector will be deprecated on September 14th, 2026. This change aligns with the discontinuation of the SAP RFC SDK, SAP's strategic integration roadmap, and customer demand for simpler integration. Migrate to the new agentless connector for simplified onboarding and compliance with SAP’s roadmap. All new deployments starting October 31, 2025, will only have the new agentless connector option, and existing customers should plan their migration using the guidance on Microsoft Learn. It will be billed at the same price as the containerized agent, ensuring no cost impact for customers. Note📌: To support transition for those of you on the Docker-based data connector, we have enhanced our built-in KQL functions for SAP to work across data sources for hybrid and parallel execution. Follow our agentless migration playlist for a smooth transition. Spotlight on new Features with agentless Inspired by the feedback of early adopters we are shipping two of the most requested new capabilities with GA right away. Customizable polling frequency: Balance threat detection value (1min intervals best value) with utilization of SAP Integration Suite resources based on your needs. ⚠️Warning! Increasing the intervals may result in message processing truncation to avoid SAP CPI saturation. See this blog for more insights. Refer to the max-rows parameter and SAP documentation to make informed decisions. Customizable API endpoint path suffix: Flexible endpoints allow running all your SAP security integration flows from the agentless connector and adherence to your naming strategies. Furthermore, you can add the community extensions like SAP S/4HANA Cloud public edition (GROW), the SAP Table Reader, and more. Displays the simplified onboarding flow for the agentless SAP connector You want more? Here is your chance to share additional feature requests to influence our backlog. We would like to hear from you! Getting Started with Agentless The new agentless connector automatically appears in your environment – make sure to upgrade to the latest version 3.4.05 or higher. Sentinel Content Hub View: Highlights the agentless SAP connector tile in Microsoft Defender portal, ready for one-click deployment and integration with your security platform The deployment experience on Sentinel is fully automatic with a single button click: It creates the Azure Data Collection Endpoint (DCE), Data Collection Rule (DCR), and Microsoft Entra ID app registration assigned with RBAC role "Monitoring Metrics Publisher" on the DCR to allow SAP log ingest. Explore partner add-ons that build on top of agentless The ISV partner ecosystem for the Microsoft Sentinel Solution for SAP is growing to tailor the agentless offering even further. The current cohort has flagship providers like our co-engineering partner SAP SE themselves with their security products SAP LogServ & SAP Enterprise Threat Detection (ETD), and our mutual partners Onapsis and SecurityBridge. Ready to go agentless? ➤ Get started from here ➤ Explore partner add-ons here. ➤ Share feature requests here. Next Steps Once deployed, I recommend to check AryaG’s insightful blog series for details on how to move to production with the built-in SAP content of agentless. Looking to expand protection to SAP Business Technology Platform? Here you go. #Kudos to the amazing Sentinel for SAP team and our incredible community contributors! That's a wrap 🎬. Remember: bringing SAP under the protection of your central SIEM isn't just a checkbox - it's essential for comprehensive security and compliance across your entire IT estate. Cheers, MartinGo agentless with Microsoft Sentinel Solution for SAP
What a title during Agentic AI times 😂 📢UPDATE: Agentless reached GA! See details here. Dear community, Bringing SAP workloads under the protection of your SIEM solution is a primary concern for many customers out there. The window for defenders is small “Critical SAP vulnerabilities being weaponized in less than 72 hours of a patch release and new unprotected SAP applications provisioned in cloud (IaaS) environments being discovered and compromised in less than three hours.” (SAP SE + Onapsis, Apr 6 2024) Having a turn-key solution as much as possible leads to better adoption of SAP security. Agent-based solutions running in Docker containers, Kubernetes, or other self-hosted environemnts are not for everyone. Microsoft Sentinel for SAP’s latest new capability re-uses the SAP Cloud Connector to profit from already existing setups, established integration processes, and well-understood SAP components. Meet agentless ❌🤖 The new integration path leverages SAP Integration Suite to connect Microsoft Sentinel with your SAP systems. The Cloud integration capability of SAP Integration Suite speaks all relevant protocols, has connectivity into all the places where your SAP systems might live, is strategic for most SAP customers, and is fully SAP RISE compatible by design. Are you deployed on SAP Business Technology Platform yet? Simply upload our Sentinel for SAP integration package (see bottom box in below image) to your SAP Cloud Integration instance, configure it for your environment, and off you go. Best of all: The already existing SAP security content (detections, workbooks, and playbooks) in Microsoft Sentinel continues to function the same way as it does for the Docker-based collector agent variant. The integration marks your steppingstone to bring your SAP threat signals into the Unified Security Operations Platform – a combination of Defender XDR and Sentinel – that looks beyond SAP at your whole IT estate. Microsoft Sentinel solution for SAP applications is certified for SAP S/4HANA Cloud, Private Edition RISE with SAP, and SAP S/4HANA on-premises. So, you are all good to go😎 You are already dockerized or agentless? Then proceed to this post to learn more about what to do once the SAP logs arrived in Sentinel. Final Words During the preview we saw drastically reduced deployment times for SAP customers being less familiar with Docker, Kubernetes and Linux administration. Cherry on the cake: the network challenges don’t have to be tackled again. The colleagues running your SAP Cloud Connector went through that process a long time ago. SAP Basis rocks 🤘 Get started from here on Microsoft Learn. Find more details on our blog on the SAP Community. Cheers MartinRun agentless SAP connector cost-efficiently
The SAP agentless connector uses SAP Integration Suite (Cloud Integration/CPI) to fetch SAP audit log data and forward it to Microsoft Sentinel. Because SAP CPI billing typically reflects message counts and data volume, you can tune the connector to control costs—while preserving reliability and timeliness. Cost reductions primarily come from sending fewer CPI messages by increasing the polling interval. The max-rows parameter is a stability safeguard that caps events per run to protect CPI resources; it is not a direct cost-optimization lever. After any change, monitor CPI execution time and resource usage. ☝️Note: It may not be feasible to increase the polling interval on busy systems processing large data volumes. Larger intervals can lengthen CPI execution time and cause truncation when event spikes exceed max-rows. Cost optimization via longer intervals generally works best on lower-utilization environments (for example, dev and test) where event volume is modest and predictable. Tunable parameters Setting Default Purpose Cost impact Risk / trade-off Polling interval 1 minute How often the connector queries SAP and triggers a CPI message. Lower message count at longer intervals → potential cost reduction. Larger batches per run can extend CPI execution time; spikes may approach max-rows after which message processing for that interval is truncated. max-rows 150,000 Upper bound on events packaged per run to protect CPI stability. None (safeguard)—does not reduce message count on its own. If too low, frequent truncation; if too high, runs may near CPI resource limits. Adjust cautiously and observe. ☝️Note: When event volume within one interval exceeds max-rows, the batch is truncated by design. Remaining events are collected on subsequent runs. Recommended approach Start with defaults. Use a 1-minute polling interval and max-rows = 150,000. Measure your baseline. Understand average and peak ingestion per minute (see KQL below). Optimize the polling interval first to reduce message count when costs are a concern. Treat max-rows as a guardrail. Change only if you consistently hit the cap; increase in small steps. Monitor after each change. Track CPI run duration, CPU/memory, retries/timeouts, and connector health in both SAP CPI and Sentinel. 💡Aim for the lowest interval that keeps CPI runs comfortably within execution-time and resource limits. Change one variable at a time and observe for at least a full business cycle. 🧐Consider the Azure Monitor Log Ingestion API limits to close the loop on your considerations. Analyze ingestion profile (KQL) ABAPAuditLog | where TimeGenerated >= ago(90d) | summarize IngestedEvents = count() by bin(UpdatedOn, 1m) | summarize MaxEvents = max(IngestedEvents), AverageEvents = toint(avg(IngestedEvents)), P95_EventsPerMin = percentile(IngestedEvents, 95) How to use these metrics? AverageEvents → indicates typical per-minute volume. P95_EventsPerMin → size for spikes: choose a polling interval such that P95 × interval (minutes) remains comfortably below max-rows. If MaxEvents × interval approaches max-rows, expect truncation and catch-up behavior—either shorten the interval or, if safe, modestly raise max-rows. Operational guidance ⚠️❗Large jumps (for example, moving from a 1-minute interval to 5 minutes and raising max-rows simultaneously) can cause CPI runs to exceed memory/time limits. Adjust gradually and validate under peak workloads (e.g., period close, audit windows). Document changes (interval, max-rows, timestamp, rationale). Alert on CPI anomalies (timeouts, retries, memory warnings). Re-evaluate regularly in higher-risk periods when SAP event volume increases. Balancing Audit Log Tuning and Compliance in SAP NetWeaver: Risks of Excluding Users and Message Classes When tuning SAP NetWeaver audit logging via transaction SM19 (older releases) or RSAU_CONFIG (newer releases), administrators can filter by user or message class to reduce log volume - such as excluding high-volume batch job users or specific event types - but these exclusions carry compliance risks: omitting audit for certain users or classes may undermine traceability, violate regulatory requirements, or mask unauthorized activities, especially if privileged or technical users are involved. Furthermore, threat hunting in Sentinel for SAP gets "crippled" due to missing insights. Best practice is to start with comprehensive logging, only apply exclusions after a documented risk assessment, and regularly review settings to ensure that all critical actions remain auditable and compliant with internal and external requirements. Cost-Efficient Long-Term Storage for Compliance Microsoft Sentinel Data Lake enables organizations to retain security logs - including SAP audit data - for up to 12 years at a fraction of traditional SIEM storage costs, supporting compliance with regulations such as NIS2, DORA and more. By decoupling storage from compute, Sentinel Data Lake allows massive volumes of security data to be stored cost-effectively in a unified, cloud-native platform, while maintaining full query and analytics capabilities for forensic investigations and regulatory reporting. This approach ensures that organizations can meet strict data retention and auditability requirements without compromising on cost or operational efficiency. Summary Use the polling interval to reduce message count (primary cost lever). Keep max-rows as a safety cap to protect CPI stability. Measure → adjust → monitor to achieve a stable, lower-cost configuration tailored to your SAP workload. Use built-in mirroring to the Sentinel Data Lake to store the SAP audit logs cost-efficient for years Next Steps Explore agentless SAP deployment config guide on Microsoft Learn Expand deployment to SAP Business Technology Platform Use the insightful blog series by AryaG for details on how to move to production with the built-in SAP content of agentless
Enhancing Security Monitoring: Integrating GitLab Cloud Edition with Microsoft Sentinel
Maximize your security operations by combining GitLab Cloud Edition with Microsoft Sentinel. This blog covers how to fill the void of a missing native connector for GitLab in Sentinel. Utilize GitLab's API endpoints, Azure Monitor Data Collection Rules, and Data Collection Endpoints, as well as Azure Logic Apps and Key Vault, to simplify log collection and improve immediate threat identification. Our detailed guide will help you integrate smoothly and strengthen your security defences.Microsoft Sentinel & Cyberint Threat Intel Integration Guide
Explore comprehensive guide on "Microsoft Sentinel & Cyberint Threat Intel Integration Guide," to learn how to integrate Cyberint's advanced threat intelligence with Microsoft Sentinel. This detailed resource will walk you through the integration process, enabling you to leverage enriched threat data for improved detection and response. Elevate your security posture and ensure robust protection against emerging threats. Read the guide to streamline your threat management and enhance your security capabilities.Integrating Fluent Bit with Microsoft Sentinel
This guide will walk you through the steps required to integrate Fluent Bit with Microsoft Sentinel. Beware that in this article, we assume you already have a Sentinel workspace, a Data Collection Endpoint and a Data Collection Rule, an Entra ID application and finally a Fluent Bit installation. As mentioned above, log ingestion API supports ingestion both in custom tables as built-in tables, like CommonSecurityLog, Syslog, WindowsEvent and more. In case you need to check which tables are supported please the following article: https://learn.microsoft.com/en-us/azure/azure-monitor/logs/logs-ingestion-api-overview#supported-tables Prerequisites: Before beginning the integration process, ensure you have the following: An active Azure subscription with Microsoft Sentinel enabled; Microsoft Entra ID Application taking note of the ClientID, TenantID and Client Secret – create one check this article: https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-register-app?tabs=certificate A Data Collection Endpoint (DCE) – to create a data collection endpoint, please check this article: https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/data-collection-endpoint-overview?tabs=portal A Data Collection Rule (DCR) – fields from the Data Collection Rule need to match exactly to what exists in table columns and also the fields from the log source. To create a DCR please check this article: https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/data-collection-rule-create-edit?tabs=cli Depending on the source, it might require a custom table to be created or an existing table from log analytics workspace; Fluent Bit installed on your server or container – In case you haven’t yet installed Fluent Bit, in the following article you'll find the instructions per type of operating system: https://docs.fluentbit.io/manual/installation/getting-started-with-fluent-bit High level architecture: Step 1: Setting up Fluent Big configuration file Before we step-in into the configuration, Fluent Bit has innumerous output plugins and one of those is through Log Analytics Ingestion API both to supported Sentinel tables but also for custom tables. You can check more information about it here in Fluent Bit documentation: https://docs.fluentbit.io/manual/pipeline/outputs/azure_logs_ingestion Moving forwarder, in order to configure Fluent Bit to send logs into Sentinel log analytics workspace, please take note of the specific input plugin you are using or intend to use to receive logs and how can you use it to output the logs to Sentinel workspace. For example most of the Fluent Bit plugins allow to set a “tag” key which can be used within the output plugin so that there’s a match in which logs are intended to send. On the other hand, in a scenario where multiple input plugins are used and all are required send logs to Sentinel, then a match of type wildcard "*" could be used as well. Another example, in a scenario where there are multiple input plugins of type “HTTP” and you want to just send a specific one into Sentinel, then the “match” field must be set according to the position of the required input plugin, for example “match http.2”, if the input plugin would the 3 rd in the list of HTTP inputs. If nothing is specified in the "match" field, then it will assume "http.0" by default. For better understanding, here’s an example of how a Fluent Bit config file could look: First, the configuration file is located under the path ”/etc/fluent-bit/fluent-bit.conf” The first part is the definition of all “input plugins”, then follows the “filter plugins” which you can use for example to rename fields from the source to match for what exists within the data collection rule schema and Sentinel table columns and finally there’s the output plugins. Below is a screenshot of a sample config file: INPUT plugins section: In this example we’re going to use the “dummy input” to send sample messages to Sentinel. However, in your scenario you could leverage other’s input plugins within the same config file. After everything is configured in the input section, make sure to complete the “FILTER” section if needed, and then move forward to the output plugin section, screenshot below. OUTPUT plugins section: In this section, we have output plugins to write on a local file based on two tags “dummy.log” and “logger”, an output plugin that prints the outputs in json format and the output plugin responsible for sending data to Microsoft Sentinel. As you can see, this one is matching the “tag” for “dummy.log” where we’ve setup the message “{“Message”:”this is a sample message for testing fluent bit integration to Sentinel”, “Activity”:”fluent bit dummy input plugn”, “DeviceVendor”:”Ubuntu”}. Make sure you insert the correct parameters in the output plugin, in this scenario the "azure_logs_ingestion" plugin. Step 2: Fire Up Fluent Bit When the file is ready to be tested please execute the following: sudo /opt/fluent-bit/bin/fluent-bit -c /etc/fluent-bit/fluent-bit.conf Fluent bit will start initialization all the plugins it has under the config file. Then you’re access token should be retrieved if everything is well setup under the output plugin (app registration details, data collection endpoint URL, data collection rule id, sentinel table and important to make sure the name of the output plugin is actually “azure_logs_ingestion”). In a couple of minutes you should see this data under your Microsoft Sentinel table, either an existing table or a custom table created for the specific log source purpose. Summary Integrating Fluent Bit with Microsoft Sentinel provides a powerful solution for log collection and analysis. By following this guide, hope you can set up a seamless integration that enhances your organization's ability to monitor and respond to security threats, just carefully ensure that all fields processed in Fluent Bit are mapped exactly to the fields in Data Collection Rule and Sentinel table within Log Analytics Workspace. Special thanks to “Bindiya Priyadarshini” that collaborated with me on this blog post. Cheers!What’s New: Exciting new Microsoft Sentinel Connectors Announcement - Ignite 2024
Microsoft Sentinel continues to be a leading cloud-native security information and event management (SIEM) solution, empowering organizations to detect, investigate, and respond to threats across their digital ecosystem at scale. Microsoft Sentinel offers robust out of the box (OOTB) content, allowing seamless connections with a wide array of data sources from both Microsoft and third-party providers. This enables comprehensive collection and analysis of security signals across multicloud, multiplatform environments, enhancing your overall security posture. In this Ignite 2024 blog post, we are thrilled to present the latest integrations contributed by our esteemed Partners. These new integrations further expand the capabilities of Microsoft Sentinel, enabling you to connect your existing security solutions and leverage Microsoft Sentinel’s powerful analytics and automation capabilities to fortify your defenses against evolving cyber threats. Featured ISV 1Password for Microsoft Sentinel The integration between 1Password Extended Access Management and Microsoft Sentinel provides businesses with real-time visibility and alerts for login attempts and account changes. It enables quick detection of security threats and streamlines reporting by monitoring both managed and unmanaged apps from a single, centralized platform, ensuring faster response times and enhanced security. Cisco Secure Email Threat Defense Sentinel Application This application collects threat information from Cisco Secure Email Threat Defense and ingests it into Microsoft Sentinel for visualization and analysis. It enhances email security by detecting and blocking advanced threats, providing comprehensive visibility and fast remediation. Cribl Stream Solution for Microsoft Sentinel Cribl Stream accelerates SIEM migrations by ingesting, transforming, and enriching third party data into Microsoft Sentinel. It simplifies data onboarding, optimizes data in various formats, and helps maintain compliance, enhancing security operations and threat detection. FortiNDR Cloud FortiNDR Cloud integrates Fortinet’s network detection and response capabilities with Microsoft Sentinel, providing advanced threat detection and automated response. Fortinet FortiNDR Cloud enhances network security by helping to identify and mitigate threats in real-time. Pure Storage Solution for Microsoft Sentinel This solution integrates Pure Storage’s data storage capabilities with Sentinel, providing enhanced data protection and performance. It helps optimize storage infrastructure and improve data security. New and Notable CyberArk Audit for Microsoft Sentinel This solution extracts audit trail data from CyberArk and integrates it with Microsoft Sentinel, providing a comprehensive view of system and user activities. It enhances incident response with automated workflows and real-time threat detection. Cybersixgill Actionable Alerts for Microsoft Sentinel Cybersixgill provides contextual and actionable alerts based on data from the deep and dark web. It helps SOC analysts detect phishing, data leaks, and vulnerabilities, enhancing incident response and threat remediation. Cyware For Microsoft Sentinel Cyware integrates with Microsoft Sentinel to automate incident response and enhance threat hunting. It uses Logic Apps and hunting queries to streamline security operations and provides contextual threat intelligence. Ermes Browser Security for Microsoft Sentinel Ermes Browser Security ingests security and audit events into Microsoft Sentinel, providing enhanced visibility and reporting. It helps monitor and respond to web threats, improving the organization’s security posture. Gigamon Data Connector for Microsoft Sentinel This solution integrates Gigamon GigaVUE Cloud Suite, including Application Metadata Intelligence, with Microsoft Sentinel, providing comprehensive network traffic visibility and insights. It helps detect anomalies and optimize network performance, enhancing overall security. Illumio Sentinel Integration Illumio integrates its micro-segmentation capabilities with Microsoft Sentinel, providing real-time visibility and control over network traffic. It helps prevent lateral movement of threats and enhances overall network security. Infoblox App for Microsoft Sentinel The Infoblox solution enhances SecOps capabilities by seamlessly integrating Infoblox's AI-driven analytics, providing actionable insights, dashboards, and playbooks derived from DNS intelligence. These insights empower SecOps teams to achieve rapid incident response and remediation, all within the familiar Microsoft Sentinel user interface. LUMINAR Threat Intelligence for Microsoft Sentinel LUMINAR integrates threat intelligence and leaked credentials data into Microsoft Sentinel, helping organizations maintain visibility of their threat landscape. It provides timely, actionable insights to help detect and respond to threats before they impact the organization. Prancer PenSuite AI Prancer PenSuite AI now supercharges Microsoft Sentinel by injecting pentesting and real-time AppSec data into SOC operations. With powerful red teaming simulations, it empowers teams to detect vulnerabilities earlier, respond faster, and stay ahead of evolving threats. Phosphorus Connector for Microsoft Sentinel Phosphorus Cybersecurity’s Intelligent Active Discovery provides in-depth context for xIoT assets, that enhances threat detection and allows for targeted responses, enabling organizations to isolate or secure specific devices based on their criticality. Silverfort for Microsoft Sentinel Silverfort integrates its Unified Identity Protection Platform with Microsoft Sentinel, securing authentication and access to sensitive systems, both on-premises and in the cloud without requiring agents or proxies. Transmit Security Data Connector for Sentinel Transmit Security integrates its identity and access management capabilities with Sentinel, providing real-time monitoring and threat detection for user activities. It helps secure identities and prevent unauthorized access. In addition to commercially supported integrations, Microsoft Sentinel Content Hub also connects you to hundreds of community-based solutions as well as thousands of practitioner contributions. For more details and instructions on how to set up these integrations see Microsoft Sentinel data connectors | Microsoft Learn. To our partners: Thank you for your unwavering partnership and invaluable contributions on this journey to deliver the most comprehensive, timely insights and security value to our mutual customers. Security is indeed a team sport, and we are grateful to be working together to enhance the security landscape. Your dedication and innovation are instrumental in our collective success. We hope you find these new partner solutions useful, and we look forward to hearing your feedback and suggestions. Stay tuned for more updates and announcements on Microsoft Sentinel and its partner ecosystem. Learn More Microsoft’s commitment to Security Microsoft’s Secure Future Initiative Unified SecOps | SIEM and XDR Solutions Unified Platform documentation | Microsoft Defender XDR What else is new with Microsoft Sentinel? Microsoft Sentinel product home Schema Mapping Microsoft Sentinel Partner Solution Contributions Update – Ignite 2023 Additional resources: Sentinel Ignite 2024 Blog Latest Microsoft Tech Community Sentinel blog announcements Microsoft Sentinel solution for SAP Microsoft Sentinel solution for Power Platform Microsoft Sentinel pricing Microsoft Sentinel customer stories Microsoft Sentinel documentation
