Understand New Sentinel Pricing Model with Sentinel Data Lake Tier
Introduction on Sentinel and its New Pricing Model Microsoft Sentinel is a cloud-native Security Information and Event Management (SIEM) and Security Orchestration, Automation, and Response (SOAR) platform that collects, analyzes, and correlates security data from across your environment to detect threats and automate response. Traditionally, Sentinel stored all ingested data in the Analytics tier (Log Analytics workspace), which is powerful but expensive for high-volume logs. To reduce cost and enable customers to retain all security data without compromise, Microsoft introduced a new dual-tier pricing model consisting of the Analytics tier and the Data Lake tier. The Analytics tier continues to support fast, real-time querying and analytics for core security scenarios, while the new Data Lake tier provides very low-cost storage for long-term retention and high-volume datasets. Customers can now choose where each data type lands—analytics for high-value detections and investigations, and data lake for large or archival types—allowing organizations to significantly lower cost while still retaining all their security data for analytics, compliance, and hunting. Please flow diagram depicts new sentinel pricing model: Now let's understand this new pricing model with below scenarios: Scenario 1A (PAY GO) Scenario 1B (Usage Commitment) Scenario 2 (Data Lake Tier Only) Scenario 1A (PAY GO) Requirement Suppose you need to ingest 10 GB of data per day, and you must retain that data for 2 years. However, you will only frequently use, query, and analyze the data for the first 6 months. Solution To optimize cost, you can ingest the data into the Analytics tier and retain it there for the first 6 months, where active querying and investigation happen. After that period, the remaining 18 months of retention can be shifted to the Data Lake tier, which provides low-cost storage for compliance and auditing needs. But you will be charged separately for data lake tier querying and analytics which depicted as Compute (D) in pricing flow diagram. Pricing Flow / Notes The first 10 GB/day ingested into the Analytics tier is free for 31 days under the Analytics logs plan. All data ingested into the Analytics tier is automatically mirrored to the Data Lake tier at no additional ingestion or retention cost. For the first 6 months, you pay only for Analytics tier ingestion and retention, excluding any free capacity. For the next 18 months, you pay only for Data Lake tier retention, which is significantly cheaper. Azure Pricing Calculator Equivalent Assuming no data is queried or analyzed during the 18-month Data Lake tier retention period: Although the Analytics tier retention is set to 6 months, the first 3 months of retention fall under the free retention limit, so retention charges apply only for the remaining 3 months of the analytics retention window. Azure pricing calculator will adjust accordingly. Scenario 1B (Usage Commitment) Now, suppose you are ingesting 100 GB per day. If you follow the same pay-as-you-go pricing model described above, your estimated cost would be approximately $15,204 per month. However, you can reduce this cost by choosing a Commitment Tier, where Analytics tier ingestion is billed at a discounted rate. Note that the discount applies only to Analytics tier ingestion—it does not apply to Analytics tier retention costs or to any Data Lake tier–related charges. Please refer to the pricing flow and the equivalent pricing calculator results shown below. Monthly cost savings: $15,204 – $11,184 = $4,020 per month Now the question is: What happens if your usage reaches 150 GB per day? Will the additional 50 GB be billed at the Pay-As-You-Go rate? No. The entire 150 GB/day will still be billed at the discounted rate associated with the 100 GB/day commitment tier bucket. Azure Pricing Calculator Equivalent (100 GB/ Day) Azure Pricing Calculator Equivalent (150 GB/ Day) Scenario 2 (Data Lake Tier Only) Requirement Suppose you need to store certain audit or compliance logs amounting to 10 GB per day. These logs are not used for querying, analytics, or investigations on a regular basis, but must be retained for 2 years as per your organization’s compliance or forensic policies. Solution Since these logs are not actively analyzed, you should avoid ingesting them into the Analytics tier, which is more expensive and optimized for active querying. Instead, send them directly to the Data Lake tier, where they can be retained cost-effectively for future audit, compliance, or forensic needs. Pricing Flow Because the data is ingested directly into the Data Lake tier, you pay both ingestion and retention costs there for the entire 2-year period. If, at any point in the future, you need to perform advanced analytics, querying, or search, you will incur additional compute charges, based on actual usage. Even with occasional compute charges, the cost remains significantly lower than storing the same data in the Analytics tier. Realized Savings Scenario Cost per Month Scenario 1: 10 GB/day in Analytics tier $1,520.40 Scenario 2: 10 GB/day directly into Data Lake tier $202.20 (without compute) $257.20 (with sample compute price) Savings with no compute activity: $1,520.40 – $202.20 = $1,318.20 per month Savings with some compute activity (sample value): $1,520.40 – $257.20 = $1,263.20 per month Azure calculator equivalent without compute Azure calculator equivalent with Sample Compute Conclusion The combination of the Analytics tier and the Data Lake tier in Microsoft Sentinel enables organizations to optimize cost based on how their security data is used. High-value logs that require frequent querying, real-time analytics, and investigation can be stored in the Analytics tier, which provides powerful search performance and built-in detection capabilities. At the same time, large-volume or infrequently accessed logs—such as audit, compliance, or long-term retention data—can be directed to the Data Lake tier, which offers dramatically lower storage and ingestion costs. Because all Analytics tier data is automatically mirrored to the Data Lake tier at no extra cost, customers can use the Analytics tier only for the period they actively query data, and rely on the Data Lake tier for the remaining retention. This tiered model allows different scenarios—active investigation, archival storage, compliance retention, or large-scale telemetry ingestion—to be handled at the most cost-effective layer, ultimately delivering substantial savings without sacrificing visibility, retention, or future analytical capabilities.
Behavior Analytics, investigation Priority
Hello, Regarding the field investigation Priority in the Behavior Analytics table, what would be the value that Microsoft considers to be high/critical to look into the user's account? By analyzing the logs i would say, 7 or higher, if someone could tell me, and thank you in advance.
UEBA not recommended in Lighthouse?
My new org is considering whether to enable UEBA. We have a multi-tenant Lighthouse-enabled environment. Previous Engineers discussed UEBA with Microsoft, and Microsoft indicated that UEBA would be inaccurate in a Lighthouse environment. Unfortunately I haven't been able to gather any additional details or explanations, and otherwise haven't found this topic documented by Microsoft. Can anyone confirm if UEBA is recommended and compatible with Lighthouse? Thanks!!!FinOps In Microsoft Sentinel
Microsoft Sentinel's security analytics and operations data is stored in an Azure Monitor Log Analytics workspace. Billing is based on the volume of data analyzed in Microsoft Sentinel and stored in the Log Analytics workspace. The cost of both is combined in a simplified pricing tier. Microsoft 365 data sources are always free to ingest for all Microsoft Sentinel users: Billable data sources: Although alerts are free, the raw logs for Microsoft Endpoint Defender, Defender for Cloud Apps, Microsoft Entra ID sign in and audit logs, and Azure Information Protection (AIP) data types are paid: Microsoft Sentinel data retention is free for the first 90 days. Enable Microsoft Sentinel on an Azure Monitor Log Analytics workspace and the first 10 GB/day is free for 31 days. The cost for both Log Analytics data ingestion and Microsoft Sentinel analysis charges up to the 10 GB/day limit are waived during the 31-day trial period. This free trial is subject to a 20 workspace limit per Azure tenant • By default, all tables in your workspace inherit the workspace's interactive retention setting and have no archive. • You can modify the retention and archive settings of individual tables Azure Monitor Logs retains data in two states: - Interactive retention: Lets you retain Analytics logs for interactive queries of up to 2 years. - Archive: Lets you keep older, less used data in your workspace at a reduced cost. • You can access data in the archived state by using search jobs, restore and keep data in archived state for up to 12 years • Its very important for cost management in MS Sentinel when you define short data retention period, but firstly go in Log Analytics WS | Workbooks | Workspace Usage in order to see tables size Use this workbook to analyze the the sizes of the different tables in your workspace: Where can save your money? Ingestion • Carefully plan what data is sent into your Microsoft sentinel workspace • Utilize filtering mechanisms to reduce ingestions to what the SOC needs • Set daily cap (good for PoC scenarios but not recommend for production) Retention • Send data to other storage platforms that have cheaper storage costs (Azure blob storage, Azure data explorer) Compute • Shutdown Azure machine learning compute during off hours, consider using reserved instances pricing • Set quotas on your subscription and workspaces • Use low-priority virtual machine (VM) Bandwidth • Sending data across Azure regions might incur into additional costs Ingestion planning • Analyze your data sources and decides what data is needed by your SOC for detection, investigations, hunting and enrichment. Take use-driven approach • Plan your workspace design • Existing workspaces might be ingesting data not needed by the SOC • Consider using separate workspace for Microsoft Sentinel • When possible enable Defender for Servers on the same workspace where you enable Microsoft Sentinel, you get 500 MB of free data ingestion per day • If you configure your Log Analytics agent to send data to two or more different Log Analytics workspaces (multi-homing), you'll get 500-MB free data ingestion for each workspace. Retention • Microsoft Sentinel retention is charged ($0.1/GB/month) and can become a big portion of the Microsoft Sentinel cost • 1.2 TB/day ingestion with 1-year retention (East US list prices) Ingestion: ~ $89К/month Retention: ~ $33К/month • If you require more than 90 days retention, determine if you need it for the whole workspace or just some tables • Consider using other storage platform for long storage retention (Azure blob storage, Azure data explorer) Long term retention options: • Azure blob storage • Cheaper than Microsoft sentinel retention • Difficult for query • Ideal for audit/compliance purposes Azure Data explorer Stores security logs in Azure Data Explorer on a long-term basis. Minimizes costs and provides easy access when you need to query the data and stores most of the data in the cold cache, minimizing the computing cost. Log Analytics doesn't currently support exporting custom log tables. In this scenario, you can use Azure Logic Apps to export data from Log Analytics workspaces. Because Azure Data Explorer provides long-term storage, you can reduce your Sentinel retention costs with this approach and ideal for forensic investigation and hunting on older data Can achieve up to 75% saving on retention costs Instead of using Azure Data Explorer for long-term storage of security logs, you can use Storage. This approach simplifies the architecture and can help control the cost. A disadvantage is the need to rehydrate the logs for security audits and interactive investigative queries. With Azure Data Explorer, you can move data from the cold partition to the hot partition by changing a policy. This functionality speeds up data exploration. Bandwidth Sending telemetry from one Azure region to another can incur in bandwidth costs this only affect Azure VMs that send telemetry across Azure regions data sources based on diagnostics settings are not affected not a big cost component compared to ingestion or retention Example: 1000 VMs, where each generates 1GB/day, sending data from US to EU: 1000 VMs * 1GB/day *30 days/month*$0.05/GB =$1.500/month Ingestion Cost Alert Playbook Managing cost for cloud services is an essential part of ensuring that you get maximum value for your investment in solutions running on this computing platform. Azure Sentinel is no different. To help you exercise greater control over your budget for Azure Sentinel this playbook will send you an alert should you exceed a budget that you define for your Azure Sentinel Workspace within a given time-frame With the ingestion cost alert playbook, you can set up an alert based on the budget defined in your Microsoft Sentinel workspace within a given timeframe. Ingestion Anomaly Alert Playbook This playbook sends you an alert should there be an ingestion spike into your workspace. The playbook uses the series_decompose_anomalies KQL function to determine anomalous ingestion The Workspace Usage Report workbook The Workspace Usage Report workbook provides your workspace's data consumption, cost, and usage statistics. The workbook gives the workspace's data ingestion status and amount of free and billable data. You can use the workbook logic to monitor data ingestion and costs, and to build custom views and rule-based alerts. This workbook also provides granular ingestion details. The workbook breaks down the data in your workspace by data table, and provides volumes per table and entry to help you better understand your ingestion patterns. Azure pricing model – based on volume of data ingested User Entity Behavior Analytics Approximately 10% of the cost of logs selected for UEBA Reduce To change your pricing tier commitment, select one of the other tiers on the pricing page, and then select Apply. You must have Contributor or Owner role in Microsoft Sentinel to change the pricing tier costs for Microsoft Sentinel Useful links: Tools that are related to FinOps on Azure Sentinel (Azure Pricing Calculator, Azure Cost Management, Azure Advisor, TCO Calculator, Azure Hybrid Benefit Savings Calculator) https://techcommunity.microsoft.com/t5/fasttrack-for-azure/the-azure-finops-guide/ba-p/3704132 Manage and monitoring Costs for Microsoft Sentinel https://learn.microsoft.com/en-us/azure/sentinel/billing-monitor-costs Reduce costs for Microsoft Sentinel https://learn.microsoft.com/en-us/azure/sentinel/billing-reduce-costs Ingestion Cost Spike Detection Playbook https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/ingestion-cost-spike-detection-playbook/ba-p/2591301 Ingestion Cost Alert Playbook https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/ingestion-cost-alert-playbook/ba-p/2006003 Introducing Microsoft Sentinel Optimization Workbook https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/introducing-microsoft-sentinel-optimization-workbook/ba-p/3901489
Analytic rules, KQL queries and UEBA pricing
Hi, I am interested if there is any additional cost when talking about Log Analytics Workspace (without Sentinel) when it comes to running KQL queries? Are there any "data processing" costs that occur or is it free in that sense? On this link https://azure.microsoft.com/en-us/pricing/details/monitor/ I didn't see any mention of "data processing costs", Microsoft only mentions "Log data processing" feature name "Log data ingestion and transformation" but writing KQL queries is not data transformation in that sense -> https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/data-collection-transformations When talking about Sentinel, should I expect larger bill if I enable 50-500 Analytic rules from Sentinel templates or content hub? Do these or custom analytic rules occur any additional "processing" costs? On this link https://azure.microsoft.com/en-us/pricing/details/microsoft-sentinel/ Microsoft only mentions "Search jobs". I assume Analytic rules and issuing KQL queries fall into category search jobs. What if someone is not using Sentinel but only Log Analytics Workspace and writing KQL queries? Since this (search jobs) is not mentioned on https://azure.microsoft.com/en-us/pricing/details/monitor/ is documentation just not up to date and this same search job price applies to KQL queries in Log Analytics deployments without Sentinel? Microsoft states UEBA doesn't cost any additional money. Is it truly no additional cost or some cost will occur since it processes data from Audit Logs, Azure Activity, Security Events and SignIn Logs tables, namely as described by "search jobs"?New Survey | UEBA Engine
Our Sentinel engineering group is looking for your input for our UEBA engine. Please reserve about fifteen minutes to respond to this survey. As a preparation for the next semester of engineering work, we want to understand more about your use in the UEBA capabilities space and how we can make it more relevant and focused for your needs. Find our survey here. Original Post: New Survey | UEBA Engine - Microsoft Tech Community
