*Thank you to Javier-Soriano for working on estimations and comparisons during the preview.*
*Updated pricing inaccuracies*
*Updated 5/25 with additional questions and answers.*
With such a large and exciting suite of features introduced to Microsoft Sentinel, we anticipate many questions. Please see some common questions below and feel free to add more questions to the discussion at the bottom of the page!
Learning Materials:
Pricing
What are some of the links for Microsoft Sentinel pricing resources?
- Microsoft Sentinel Pricing Page: Azure Sentinel Pricing | Microsoft Azure.
- Microsoft Monitor Pricing Page: Pricing - Azure Monitor | Microsoft Azure.
- Azure Pricing Calculator: Pricing Calculator | Microsoft Azure
- Reference billing documentation for in-depth info about billing, free data tables, and more.
Search
1. When should I use the Search experience instead of regular KQL queries?
Regular KQL queries only supports search on log data within the standard interactive retention period of the workspace or the first 8 days of log data in the Basic Logs plan. Searches using regular KQL queries are subject to a 10-minute timeout making them less than ideal for searches across massive data volumes. KQL queries are ideal for interactive, searches across smaller data volumes.
The Sentinel search experience supports searching across multiple log plans within a single search job (Analytics, Basic, and/or Archived). Sentinel Search breaks up a single search into multiple parallel jobs and has a 24-hour timeout, making it ideal for search on massive data volumes. (See the workspace overview documentation for more information on the different log plans)
2. When should I use Search instead of Log Restore?
Microsoft Sentinel search is best used for searches returning log events that match a search term. Log Restore is built for restoring large chucks of log data or log events from a single specified table, without the need to specify a search term.
3. Why should I use the Archived Logs with Sentinel Search vs/ Continuous Data Export to Azure Data Explorer?
Using Search and Archived Logs allows for a simplified, maintenance free architecture while providing low-cost archive storage within the same Log Analytics workspace. Using Continuous Data Export with Azure Data Explorer adds additional development and maintenance overhead for hosting services that enable this integration.
4. How long are Search and Restore tables kept in my workspace?
The Search results tables that are created will inherit the interactive retention settings from the workspace. If the workspace is set to 90 days, then the tables will be retained for 90 days. Restored tables are available as long you need them and will be billed per hour until they are deleted.
5. Will I be charged when performing searches via the Sentinel Search UX on logs enabled for Basic Logs or Archived Logs?
At the current time in the preview, no, you will not be charged for searches. In the future, search jobs will be charged at 0.005/GB across both Basic and Archived log data scanned. Since the search results are re-ingested into the workspace, there will also be a charge incurred for the ingestion of the search results at the regular Analytics Log ingestion rates.
6. Can a Search job search across multiple tables at one time?
Not at this time but search across multiple tables is being looked into.
7. How will Search handle RBAC on tables?
Search will honor the RBAC of tables set within the workspace. If a user without read permissions generates a Search job, they will not get results. Separate RBAC roles for Search and Log Restoration are on the feature roadmap.
8. Are there any tables that are not supported for Search?
No, each table within the workspace is eligible for Search jobs.
9. Is this feature available in GCC-H?
Not at this time. The features will be available in GCC-H at a later time.
10. Since the table created by a Search job inherits the workspace retention policy, is the table automatically removed at the end of retention?
Yes.
11. If Search seems superior, why use a regular query for data?
Both options are great to use. Search is a great option for looking for results across each tier of data in the workspace while regular KQL can be used for analytic tier and basic tier data separately. Search jobs do not generate cost when querying the analytics tier but keep in mind that searches across basic and archive tiers.
12. If just Analytic tier data is scanned in a Search job, is cost generated for that Search job?
No.
Basic Logs Ingestion
1. Are there any restrictions on what data can be configured for Basic Logs?
At the time, the following tables can be configured for Basic Logs - AppTraces, ContainerLog, and any Custom Log (requires migration to DCR-based custom logs). See link for more information on the new DCR-based custom logs.
2. Can I convert a table from the Basic Logs to Analytics Logs and vice versa?
Yes, a table can be converted between table ingestion plans Analytics Logs to Basic Logs interchangeably or back and forth. Please note that the table ingestion plan can only be modified once per week.
3. How can Basic Logs be used by customers?
Basic logs are available accessible via limited KQL for 8 days after ingestion for interactive queries, the query API, and the new Search UX. Data from Basic Logs can be used for investigation, IOC search, ad hoc queries, and as part of Logic App playbook automation. Beyond the initial 8 days, Basic Logs can be configured as Archived Logs and are accessible via the new Search experience. Official documentation for Basic Logs use cases will be available soon.
4. What is the difference between a search job and a query on Basic logs?
Interactive queries utilize a limited KQL experience in the portal. Search utilizing a new search job concept that performs an asynchronous search job and returns the results to a table within the Log Analytics workspace.
5. Is the latency of Basic Log queries vs. analytic queries the same?
In most cases latency will not be noticeable but there is a chance that the query duration on a Basic Logs table might be slightly longer than an identical query on an Analytics Logs table.
6. If I have tables enabled for Basic Logs enabled in my workspace, does Sentinel charge for these logs?
Yes. Sentinel charges for tables enabled for Basic Log ingestion. Basic logs cannot be excluded from the Sentinel charge.
7. Can Basic logs be used for analytic rules?
No, Basic logs are not supported for neither Azure Monitor Alerts nor Microsoft Sentinel detections.
8. Do Basic Logs support the ingestion-time transformation?
Yes. For more information, see here.
9. What are some examples questions application and endpoint owners can ask themselves to determine if the logs should be ingested as basic logs?
- Are these logs being used to generate detections?
- Are these logs required for compliance reasons?
- Are these logs good to have for investigations?
- Are these logs good to have for enrichment?
- Do we currently have logs that we are ingesting at full price that should be changed to basic tier?
- Do we currently have data that we don't query often?
10. Do basic logs contribute to the commitment tier of the workspace?
No. If 300 GB is ingested as Analytics tier and 100 GB is ingested as Basic tier, the 100 GB will not count towards a commitment and the recommended commitment tier would be 300 GB. This is reflected on the Azure Pricing Calculator.
Archived Logs and Log Restore
1. Are there any restrictions as to which tables can be archived?
No. Any table in the workspace can be set to archive storage for up to 7 years.
2. Is there a way to configure Archived Logs so that any future tables created within a workspace are automatically archived?
Not at this time. Archived Logs is currently only configurable on existing tables as the setting applies on a per table basis.
3. Can logs be configured to be sent straight to Archive?
Currently, logs must be ingested using the Basic Logs or Analytics Log plans before the log data is archived.
4. Can different tables have different retention policies or types?
Yes, tables can be configured to have different retentions than the workspace and from each other. Please note that if a table is given a unique retention, the default workspace retention remains unchanged.
5. If I adjust my table retention and lower it, will the data between the new retention policy and the old one be lost?
If the tables are within the workspace are configured for archival, the data will just be moved to archive in order to maintain the total retention policy in place. Ex. 720 total days of retention, 365 days in the workspace and 365 days in archive. The workspace retention is changed to 180 days. Rather than the data being lost, the data between day 180 and day 365 will be marked for archival and moved to maintain the 720 total days of retention.
6. Is Azure Data Explorer (ADX) or Azure Blobs used for Archived Log retention?
No, by using Archived Logs, Search, and Log Restore, logs will no longer need to be exported to other services for long-term retention. The logs can be archived, searched, or restored when needed while the data remains within the workspace the whole time.
7. Is there a comparison between archiving logs in Archived Logs vs. ADX?
Here is a sample comparison for an actual customer with 3TB ingest a day:
8. What is the main difference between Search and Restore if both bring back logs and put them in a special table?
Search returns results based on criteria via the query that is used. Restore will return all logs between a start and end time for a period. Restore does not review criteria to match before bringing the data back in. Additionally, Search results are re-ingested as the tables are generated vs. restore tables rehydrating the data without re-ingestion.
9. Are the results of Search and Restore jobs at the user level or a global level?
They are available at a global level. Jobs generated by other users are visible.
10. Are all data tables migrated to Archive at the end of the workspace retention period?
No. Each table needs to be configured for archival as they are not enabled by default.
11. Is there a UI within Log Analytics to configure archival?
Yes, please see this blog: Simplified Log Analytics Table Management - Microsoft Tech Community
As mentioned, if there are any other questions, please add them to the discussion below and we will be happy to answer them!
Updated May 25, 2022
Version 10.0
Matt_Lowe
Microsoft
Microsoft
