Azure Sentinel false positive incidents due to duplicate logs in SigninLogs

%3CLINGO-SUB%20id%3D%22lingo-sub-1249774%22%20slang%3D%22en-US%22%3EAzure%20Sentinel%20false%20positive%20incidents%20due%20to%20duplicate%20logs%20in%20SigninLogs%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1249774%22%20slang%3D%22en-US%22%3E%3CP%3EHello%2C%20I've%20been%20searching%20for%20a%20way%20to%20fix%20this%20for%20the%20last%202%20weeks%20but%20I%20couldn't%20find%20anything%20that%20works.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EWe%20have%20recently%20deployed%20Azure%20Sentinel%2C%20and%20we're%20getting%20frequent%20false%20positive%20incidents.%20While%20investigating%20that%2C%20I've%20noticed%20that%20in%20the%20SigninLogs%20table%2C%20some%20entries%20are%20duplicated%2C%20and%20this%20triggers%20some%20rules%2C%20for%20example%20rules%20related%20to%20%22Multiple%20failed%20authentication%22%20or%20%22Multiple%20password%20reset%20attempts%22.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI've%20checked%20the%20duplicated%20rows%20and%20they%20have%20the%20exact%20same%20values%20in%20all%20columns%2C%20so%20not%20exactly%20sure%20how%20to%20proceed%20from%20here.%20I'd%20like%20to%20get%20rid%20of%20the%20duplicates%20first%2C%20instead%20of%20having%20to%20apply%20a%20workaround%20to%20all%20the%20Analytics%20rules%20we%20have%20in%20place.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22alexandrub_0-1585061692882.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F179158i464753BC2738F245%2Fimage-size%2Fmedium%3Fv%3D1.0%26amp%3Bpx%3D400%22%20title%3D%22alexandrub_0-1585061692882.png%22%20alt%3D%22alexandrub_0-1585061692882.png%22%20%2F%3E%3C%2FSPAN%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22alexandrub_1-1585061752960.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F179159i2537F5E154D43003%2Fimage-size%2Fmedium%3Fv%3D1.0%26amp%3Bpx%3D400%22%20title%3D%22alexandrub_1-1585061752960.png%22%20alt%3D%22alexandrub_1-1585061752960.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI'd%20also%20like%20to%20mention%20that%20the%20rules%20we%20have%20enabled%20are%20the%20built-in%20ones%20provided%20by%20Microsoft.%3C%2FP%3E%3CP%3EOne%20example%20would%20be%20the%20one%20below.%3C%2FP%3E%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22alexandrub_2-1585061906046.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F179160i2AED787616BEA2A1%2Fimage-size%2Fmedium%3Fv%3D1.0%26amp%3Bpx%3D400%22%20title%3D%22alexandrub_2-1585061906046.png%22%20alt%3D%22alexandrub_2-1585061906046.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EAny%20ideas%20on%20how%20to%20proceed%20from%20here%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThank%20you!%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1249774%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAzure%20Log%20Analytics%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EAzure%20Sentinel%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1251911%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Sentinel%20false%20positive%20incidents%20due%20to%20duplicate%20logs%20in%20SigninLogs%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1251911%22%20slang%3D%22en-US%22%3E%3CP%3EHi%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F592901%22%20target%3D%22_blank%22%3E%40alexandrub%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E1.%20I%20can%20see%20the%20duplicates%20-%20so%20will%20ask%20about%20why%20that%20is.%26nbsp%3B%20Obviously%20you%20can%20use%20a%20%3CSTRONG%3Esummarize%3C%2FSTRONG%3E%20or%20%3CSTRONG%3Edistinct%3C%2FSTRONG%3E%20to%20remove%20them.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E2.%20That%20Incident%20query%20makes%20use%20of%20multiple%20%3CSTRONG%3Esummarize%3C%2FSTRONG%3E%20operators%20and%20(for%20me%2C%20doing%20a%20simple%20test)%20that%20removes%20the%20duplicates.%26nbsp%3B%20Just%20to%20confirm%2C%20the%20full%20unaltered%20query%20using%20your%20data%20shows%20duplicates%3F%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThanks%20Clive%26nbsp%3B%20%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1251950%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Sentinel%20false%20positive%20incidents%20due%20to%20duplicate%20logs%20in%20SigninLogs%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1251950%22%20slang%3D%22en-US%22%3EHello%2C%3CBR%20%2F%3E%3CBR%20%2F%3EThanks%20for%20your%20quick%20response.%20I%20would%20love%20to%20get%20rid%20of%20the%20duplicates%20in%20the%20table%20without%20having%20to%20use%20summarize%2Fdistinct%20to%20filter%20them%20out.%20From%20what%20I've%20read%20in%20the%20Azure%20documentation%2C%20I've%20noticed%20that%20it's%20quite%20difficult%20to%20avoid%20duplicates%20and%20using%20summarize%20and%2For%20distinct%20is%20a%20common%20workaround.%20Do%20you%20think%20this%20is%20the%20same%20case%20here%3F%3CBR%20%2F%3E%3CBR%20%2F%3EFor%20the%20second%20topic%2C%20I'm%20not%20sure%20what%20to%20answer%20here.%20I%20did%20notice%20that%20some%20Incident%20queries%20use%20summarize%2C%20but%20I'm%20not%20exactly%20sure%20if%20the%20built-in%20incident%20queries%20are%20validated%20in%20order%20to%20treat%2Fget%20rid%20of%20duplicates.%3C%2FLINGO-BODY%3E
Highlighted
New Contributor

Hello, I've been searching for a way to fix this for the last 2 weeks but I couldn't find anything that works.

 

We have recently deployed Azure Sentinel, and we're getting frequent false positive incidents. While investigating that, I've noticed that in the SigninLogs table, some entries are duplicated, and this triggers some rules, for example rules related to "Multiple failed authentication" or "Multiple password reset attempts".

 

I've checked the duplicated rows and they have the exact same values in all columns, so not exactly sure how to proceed from here. I'd like to get rid of the duplicates first, instead of having to apply a workaround to all the Analytics rules we have in place.

 

alexandrub_0-1585061692882.pngalexandrub_1-1585061752960.png

 

I'd also like to mention that the rules we have enabled are the built-in ones provided by Microsoft.

One example would be the one below.

alexandrub_2-1585061906046.png

 

Any ideas on how to proceed from here?

 

Thank you!

2 Replies

Hi @alexandrub 

 

1. I can see the duplicates - so will ask about why that is.  Obviously you can use a summarize or distinct to remove them. 

2. That Incident query makes use of multiple summarize operators and (for me, doing a simple test) that removes the duplicates.  Just to confirm, the full unaltered query using your data shows duplicates?

 

Thanks Clive   

Highlighted
Hello,

Thanks for your quick response. I would love to get rid of the duplicates in the table without having to use summarize/distinct to filter them out. From what I've read in the Azure documentation, I've noticed that it's quite difficult to avoid duplicates and using summarize and/or distinct is a common workaround. Do you think this is the same case here?

For the second topic, I'm not sure what to answer here. I did notice that some Incident queries use summarize, but I'm not exactly sure if the built-in incident queries are validated in order to treat/get rid of duplicates.