What are near-real-time (NRT) analytics rules?
When you are faced with security threats, time and speed are of the essence. You need to be aware of threats as they materialize so you can analyze and respond quickly to contain them. Sentinel's near-real-time (NRT) analytics rules offer you faster threat detection.
Sentinel’s NRT rules were designed to be highly responsive by running queries at intervals just one minute apart.
How do they work?
NRT rules are designed to run once every minute and capture events ingested in the preceding minute, so as to be able to supply you with information as up-to-the-minute as possible.
The NRT rules are delayed by 2 minutes, due to the time it takes to ingest data to Sentinel (making events visible in the workspace).
It is essential for both scheduled and NRT rules that the data will be ingested into the workspace when the query is executed.
Since NRT rules track the ingestion time and not the event creation time (the TimeGenerated field), we can ignore the ingestion delay (the time between the event’s creation and its ingestion into the workspace).
NRT rules have many of the same features and capabilities as scheduled analytics rules. The full set of alert enrichment capabilities is available – you can map entities and surface custom details, and you can configure dynamic content for alert details. You can choose how alerts are grouped into incidents, you can temporarily suppress the running of a query after it generates a result, and you can define automation rules and playbooks to run in response to alerts and incidents generated from the rule.
At the moment NRT rules are limited by the KQL syntax they support (not supporting join, union, cross workspace..) as well as by the number of rules supported (up to 20 rules).
Comparison between Scheduled and NRT rules:
|
Criteria |
Scheduled query rule |
NRT query rule |
|
Built in delay |
5 minutes |
2 minutes |
|
Filtered by |
Time Generated |
Ingestion time |
|
Scheduling (frequency) |
5 minutes maximum, set by the user |
Fixed 1 minute. |
|
Syntax |
Full KQL |
Partial KQL support |
|
Quantity |
Up to 512 rules |
20 rules |
|
Tables |
Query number of tables |
Single table |
Sample use-case – Monitor break glass account access:
What is a break glass account?
A break glass account is an account that is used for emergency purposes to gain access to a system or service that is not accessible under normal controls. You, as a systems administrator should not only document all of your break glass accounts but also regularly audit those accounts to ensure that the correct people have access.
As recommended, we would like to make this account a cloud account, make it a global admin and monitor all sign ins made from this account.
We will define this user in our Azure AD.
We define a user name that will be easily recognized by other admins – “EmergencyAdmin” and set it as a global administrator.
Typically, any account that is used for emergency purposes needs to have the rights to be able to gain access to the system and subvert any controls or lockouts that are in place.
Now after our work here is done let’s take on the challenge of this extensive monitoring.
Before we start, we need to understand that if this account is compromised, we are in trouble! For that reason, we would like to monitor any access to this account, and every second counts.
The sign-in logs for Azure AD do have some latency, so NRT will be the fastest way to monitor these events.
Our NRT detection:
It is recommended that you monitor sign-in activities by these emergency accounts. We want to be alerted on any activity coming from these accounts.
Since there can be several accounts in our organization, we would like to manage them in a single place and to support adding/removing/updating the accounts.
That is why we will start by creating a watchlist that will be used to manage all our break glass account UPN.
Microsoft Sentinel watchlists enable the collection of data from external data sources for correlation with the events in your Microsoft Sentinel environment. Once created, you can use watchlists in your search, detection rules, threat hunting, and response playbooks.
Now we can proceed to define our rule, note that we are selecting the “NRT query rules”.
We will name the NRT detection and provide description, tactics, and severity.
Setting a static name to the detection would work for now but it would be even better to mention the actual account that is being accessed but we will address that later.
We will define the following query in our NRT rule:
We can see that we are not explicitly using the account UPN in the query which means that we can change the watchlist at any given time and all of our rules will be up to date.
We will use Alert details to provide the name of the account on the incident that will be created.
For more details on how to customize the alert details please review the blog post on how to reduce investigation time by using alert enrichment.
We are done! Now any access to this account will be monitored and an incident will be created.
Note that even when there is significant ingestion delay, our detection will not miss any events since we are looking at the ingestion time.
Why shouldn’t I just use NRT for everything?
We need to understand that there is no “silver bullet” for threat detection, but these new abilities added by the NRT rules will improve the SOC’s ability to detect and respond to threats.
When trying to correlate multiple events we want to look at the time the events were created and determine a sequence. Using only the ingestion time in this scenario does not provide the required results. However, using both Scheduled query rules, Incident creation rules and NRT query rules together will provide better results.