Ignore alerts if Entities Match previous within the last 24 hours

%3CLINGO-SUB%20id%3D%22lingo-sub-1784230%22%20slang%3D%22en-US%22%3EIgnore%20alerts%20if%20Entities%20Match%20previous%20within%20the%20last%2024%20hours%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1784230%22%20slang%3D%22en-US%22%3E%3CP%3EI%20have%20a%20Proofpoint%20TAP%20connected%20to%20Sentinel.%20When%20a%20User%20clicks%20on%20a%20Malicious%20link%20in%20an%20email%2C%20one%20of%20our%20remediation%20steps%20is%20to%20have%20the%20user%20change%20their%20password.%20I%20have%20encountered%20a%20situation%20whereby%20Proofpoint%20generated%20one%20alert%2C%20but%20Sentinel%20generated%20two%20of%20the%20same%20alerts%2C%20an%20hour%20apart%20and%20triggered%20a%20playbook%20twice%2C%20to%20reset%20a%20user's%20password%20on%20both%20occasions.%26nbsp%3BAs%20in%20the%20image%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22sammyredo_0-1602775238805.png%22%20style%3D%22width%3A%20706px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F226941i3AC7402E39566A38%2Fimage-dimensions%2F706x60%3Fv%3D1.0%22%20width%3D%22706%22%20height%3D%2260%22%20role%3D%22button%22%20title%3D%22sammyredo_0-1602775238805.png%22%20alt%3D%22sammyredo_0-1602775238805.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20am%20seeking%20to%20create%20a%20solution%2C%20where%20by%20if%20a%20new%20alert%20is%20generated%20and%20has%20the%20entities%20match%20a%20previously%20created%20alert%20within%2024%20hours%2C%20the%202nd%20would%20be%20ignored%20and%20would%20not%20trigger%20the%20playbook.%20If%20there%20is%20a%20dynamic%20way%20of%20preventing%20these%20duplication%20of%20alerts%2C%20that%20would%20be%20the%20preferred%20rout.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1784276%22%20slang%3D%22en-US%22%3ERe%3A%20Ignore%20alerts%20if%20Entities%20Match%20previous%20within%20the%20last%2024%20hours%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1784276%22%20slang%3D%22en-US%22%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F807427%22%20target%3D%22_blank%22%3E%40sammyredo%3C%2FA%3E%20Have%20you%20tried%20aggregating%20the%20alerts%20together%20in%20the%20Analytic%20rule%3F%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1784810%22%20slang%3D%22en-US%22%3ERe%3A%20Ignore%20alerts%20if%20Entities%20Match%20previous%20within%20the%20last%2024%20hours%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1784810%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F833862%22%20target%3D%22_blank%22%3E%40LodewykV%3C%2FA%3E%26nbsp%3B%26nbsp%3BI%20have%20configured%20to%20group%20the%20alerts%20if%20the%20entities%20match.%20I%20have%20a%20question%20about%20that%20function%20though.%20So%20I%20have%20configured%20my%20query%20to%20run%20every%205%20minutes.%20If%20I%20set%20to%20limit%20the%20group%20to%20alerts%20created%20within%201%20hour%20and%20After%20the%20first%20alert%20is%20generated%20the%20first%20query%20run%2C%20will%20the%20subsequent%20alerts%20be%20added%20to%20the%201st%2C%20and%20won't%20they%20trigger%20an%20automated%20playbook%3F%20I%20get%20that%20the%20alerts%20generated%20within%20the%20hour%20will%20be%20grouped.%20My%20question%20however%20is%20how%20will%20that%20affect%20the%20automation%3F%20The%20first%20query%20runs%20and%20generates%20an%20alert%20which%20triggers%20a%20playbook.%20Query%20runs%20again%20after%2045%20minutes%20and%20generates%20another%20alert%20with%20same%20entities%2C%20will%20that%20trigger%20the%20playbook%2C%20or%20it%20will%20just%20be%20added%20to%20the%20first%20alert%20and%20not%20trigger%20the%20playbook%3F%3C%2FP%3E%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22sammyredo_0-1602778421055.png%22%20style%3D%22width%3A%20581px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F226972i78FD94DCD537E587%2Fimage-dimensions%2F581x478%3Fv%3D1.0%22%20width%3D%22581%22%20height%3D%22478%22%20role%3D%22button%22%20title%3D%22sammyredo_0-1602778421055.png%22%20alt%3D%22sammyredo_0-1602778421055.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22sammyredo_1-1602778490706.png%22%20style%3D%22width%3A%20718px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F226973i1CF107F593F9FE4A%2Fimage-dimensions%2F718x569%3Fv%3D1.0%22%20width%3D%22718%22%20height%3D%22569%22%20role%3D%22button%22%20title%3D%22sammyredo_1-1602778490706.png%22%20alt%3D%22sammyredo_1-1602778490706.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1785311%22%20slang%3D%22en-US%22%3ERe%3A%20Ignore%20alerts%20if%20Entities%20Match%20previous%20within%20the%20last%2024%20hours%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1785311%22%20slang%3D%22en-US%22%3ECurrently%20there%20is%20only%20one%20automation%20possible%2C%20which%20runs%20everytime%20an%20alert%20is%20created.%20So%20even%20when%20the%20same%20incident%20has%20multiple%20alerts%2C%20it%20will%20run%20multiple%20times.%3CBR%20%2F%3E%3CBR%20%2F%3EThere%20is%20a%20new%20possibility%20in%20private%20preview%2C%20which%20will%20only%20trigger%20once%20per%20incident.%20This%20would%20be%20a%20solution%20for%20you%2C%20but%20it's%20not%20GA%20yet.%3C%2FLINGO-BODY%3E
Highlighted
Occasional Contributor

I have a Proofpoint TAP connected to Sentinel. When a User clicks on a Malicious link in an email, one of our remediation steps is to have the user change their password. I have encountered a situation whereby Proofpoint generated one alert, but Sentinel generated two of the same alerts, an hour apart and triggered a playbook twice, to reset a user's password on both occasions. As in the image

 

sammyredo_0-1602775238805.png

 

I am seeking to create a solution, where by if a new alert is generated and has the entities match a previously created alert within 24 hours, the 2nd would be ignored and would not trigger the playbook. If there is a dynamic way of preventing these duplication of alerts, that would be the preferred rout. 

 

10 Replies
Highlighted
@sammyredo Have you tried aggregating the alerts together in the Analytic rule?
Highlighted

@LodewykV  I have configured to group the alerts if the entities match. I have a question about that function though. So I have configured my query to run every 5 minutes. If I set to limit the group to alerts created within 1 hour and After the first alert is generated the first query run, will the subsequent alerts be added to the 1st, and won't they trigger an automated playbook? I get that the alerts generated within the hour will be grouped. My question however is how will that affect the automation? The first query runs and generates an alert which triggers a playbook. Query runs again after 45 minutes and generates another alert with same entities, will that trigger the playbook, or it will just be added to the first alert and not trigger the playbook?

sammyredo_0-1602778421055.png

 

 

sammyredo_1-1602778490706.png

 

Highlighted
Currently there is only one automation possible, which runs everytime an alert is created. So even when the same incident has multiple alerts, it will run multiple times.

There is a new possibility in private preview, which will only trigger once per incident. This would be a solution for you, but it's not GA yet.
Highlighted

@Thijs Lecomte You might be referring to "When Azure Sentinel incident creation rule was triggered"?

 

sammyredo_1-1602788385031.png

 

Thank you for the response 

 

Highlighted
Highlighted

@Thijs LecomteThank you! But is there a way to stop alerts from generating for the same entities  repeatedly, especially if the source is only generating one of such alerts?

Highlighted
If alert grouping doesn't work, you could try to join your current kql query with the security alert table to only show things that aren't in the security alert table
Highlighted

@Thijs Lecomte  Nice Lead.. Do you have an example you can show me please? This is what I have so far. 

let timeframe = ago(30m);
ProofPointTAPClicksPermitted_CL
where TimeGenerated >= timeframe
summarize StartTime = min(TimeGenerated), EndTime = max(TimeGenerated), count() by TimeGenerated, Sender = sender_s, SenderIPAddress = senderIP_s, Recipient = recipient_s, TimeClicked = clickTime_t, URLClicked = url_s
extend timestamp = StartTime, AccountCustomEntity = Recipient, IPCustomEntity = SenderIPAddress, URLCustomEntity = URLClicked, SenderCustomEntity = Sender
extend HostCustomEntity = Recipient
extend FileHashCustomEntity = URLClicked
extend SenderCustomEntity = Sender
Highlighted
1. create a list of last 24h alerts.
2. create your main query and use "entities !in~ (the_list_you_created)".

If you have more than one entities, you can extend a new column and concatanate them into one field both for creating the list and using it in the main query.
Highlighted
Look into the join operator, left anti
https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/joinoperator?pivots=azuredataexplor...
You should join on the 'HostCustomEntity, FileHashCustomEntity and SenderCustomEntity'