%3CLINGO-SUB%20id%3D%22lingo-sub-1030142%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Sentinel%20correlation%20rules%3A%20Active%20Lists%20out%3B%20makelist()%20in%2C%20the%20AAD%2FAWS%20correlation%20exam%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1030142%22%20slang%3D%22en-US%22%3E%3CP%3EKeep%20the%20great%20blog%20posts%20coming!%20%3CIMG%20class%3D%22lia-deferred-image%20lia-image-emoji%22%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Fhtml%2Fimages%2Femoticons%2Fsmile_40x40.gif%22%20alt%3D%22%3Asmile%3A%22%20title%3D%22%3Asmile%3A%22%20%2F%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1030204%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Sentinel%20correlation%20rules%3A%20Active%20Lists%20out%3B%20makelist()%20in%2C%20the%20AAD%2FAWS%20correlation%20exam%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1030204%22%20slang%3D%22en-US%22%3E%3CP%3EExcellent%20article%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F293879%22%20target%3D%22_blank%22%3E%40Ofer_Shezaf%3C%2FA%3E%26nbsp%3Band%20I%20especially%20liked%20the%20colour%20coding%3CBR%20%2F%3E%3CBR%20%2F%3Enote%3A%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fkusto%2Fquery%2Fmakelist-aggfunction%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fkusto%2Fquery%2Fmakelist-aggfunction%3C%2FA%3E%26nbsp%3Byou%20may%20want%20to%20use%20make_list()%20rather%20than%20makelist()%20for%20large%20scale%20scenarios%20(makelist%20has%20a%20128%20default%20limit).%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1030208%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Sentinel%20correlation%20rules%3A%20Active%20Lists%20out%3B%20makelist()%20in%2C%20the%20AAD%2FAWS%20correlation%20exam%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1030208%22%20slang%3D%22en-US%22%3E%3CP%3EI%20switched%20from%20make_list%2C%20which%20I%20normally%20use%2C%20to%20makelist%20to%20adjust%20to%20the%20github%20query%2C%20but%20was%20not%20ware%20of%20the%20difference.%20I%20will%20discuss%20with%20our%20research%20team%20if%20we%20need%20to%20modify%20the%20original.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1030635%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Sentinel%20correlation%20rules%3A%20Active%20Lists%20out%3B%20makelist()%20in%2C%20the%20AAD%2FAWS%20correlation%20exam%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1030635%22%20slang%3D%22en-US%22%3E%3CP%3EVery%20useful%20%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F293879%22%20target%3D%22_blank%22%3E%40Ofer_Shezaf%3C%2FA%3E%26nbsp%3Bthanks%20for%20sharing!%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1141332%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Sentinel%20correlation%20rules%3A%20Active%20Lists%20out%3B%20make_list()%20in%2C%20the%20AAD%2FAWS%20correlation%20exa%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1141332%22%20slang%3D%22en-US%22%3E%3CP%3EHi%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F293879%22%20target%3D%22_blank%22%3E%40Ofer_Shezaf%3C%2FA%3E%26nbsp%3B%2C%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThanks%20for%20a%20very%20good%20post.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20am%20also%20interested%20in%20%22%3CSPAN%3EA%20future%20post%20will%20discuss%20implementing%20lookup%20tables%20in%20Azure%20Sentinel.%22.%20I%20am%20very%20new%20to%20Azure%20Sentinel%20and%20want%20to%20integrate%20custom%20threat%20intelligence%20from%20our%20company's%20website.%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%3EIf%20I%20download%20the%20TI%20feeds%20from%20our%20website%20and%20paste%20it%20somewhere%20on%20my%20local%20machine%2C%20then%20how%20can%20I%20update%20those%20feeds%20in%20Active%20lists%20(or%20similar%20in%20Sentinel)%20and%20call%20them%20against%20rules.%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%3EThanks%20in%20Advance.%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%3CSPAN%3E%3CBR%20%2F%3ERegards%2C%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%3CSPAN%3EMitesh%20Agrawal%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1142176%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Sentinel%20correlation%20rules%3A%20Active%20Lists%20out%3B%20make_list()%20in%2C%20the%20AAD%2FAWS%20correlation%20exa%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1142176%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F539205%22%20target%3D%22_blank%22%3E%40MiteshAgrawal%3C%2FA%3E%26nbsp%3B%2C%20we%20already%20published%20this%20blog%20post%2C%20or%20at%20least%20the%20first%20in%20a%20series%2C%20%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fazure-sentinel%2Fimplementing-lookups-in-azure-sentinel-part-1-reference-files%2Fba-p%2F1091306%22%20target%3D%22_self%22%3Ehere%3C%2FA%3E.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1142263%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Sentinel%20correlation%20rules%3A%20Active%20Lists%20out%3B%20make_list()%20in%2C%20the%20AAD%2FAWS%20correlation%20exa%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1142263%22%20slang%3D%22en-US%22%3E%3CP%3EThanks%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F293879%22%20target%3D%22_blank%22%3E%40Ofer_Shezaf%3C%2FA%3E%26nbsp%3B.%20The%20post%20is%20very%20useful.%20My%20requirement%20can%20be%20met%20with%20the%20link%20you%20provided.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EOne%20more%20query%20I%20have%20is%20whether%20can%20I%20read%20data%20from%20a%20file%20in%20my%20local%20machine%20if%20I%20do%20not%20want%20to%20create%20a%20BLOB%20object%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ERegards%2C%3C%2FP%3E%3CP%3EMitesh%20Agrawal%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1029225%22%20slang%3D%22en-US%22%3EAzure%20Sentinel%20correlation%20rules%3A%20Active%20Lists%20out%3B%20make_list()%20in%2C%20the%20AAD%2FAWS%20correlation%20example%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1029225%22%20slang%3D%22en-US%22%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EWriting%20alert%20rules%20using%20KQL%20is%20powerful%2C%20but%20does%20not%20have%20to%20be%20complex.%20A%20good%20example%20would%20be%20rules%20that%20in%20traditional%20SIEM%20use%20Active%20Lists%20(or%20Reference%20Sets%2C%20depending%20on%20your%20SIEM).%20When%20presenting%20KQL%20rules%2C%20I%20am%20often%20asked%20how%20to%20implement%20Active%20Lists%20in%20Sentinel.%20While%20replicating%20Active%20Lists%20in%20Sentinel%20is%20a%20good%20topic%20for%20another%20blog%20post%2C%20I%20will%20focus%20here%20on%20avoiding%20Active%20Lists%20in%20the%20first%20place%20using%20Sentinel%20query%20based%20rules.%20You%20just%20don't%20need%20them%20in%20most%20cases.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CFONT%20color%3D%22%23FF00FF%22%3E%3CSTRONG%3E%3CEM%3EThis%20post%20of%20part%20of%20a%20series%20of%20blog%20posts%20on%20writing%20rules%20in%20Azure%20Sentinel%3A%3C%2FEM%3E%3C%2FSTRONG%3E%3C%2FFONT%3E%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3E%3CFONT%20color%3D%22%23FF00FF%22%3E%3CSTRONG%3E%3CFONT%20color%3D%22%23000080%22%3E%3CEM%3E%5B%23%24dp11%5D%3CEM%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fazure-sentinel%2Fazure-sentinel-correlation-rules-active-lists-out-make-list-in%2Fba-p%2F1029225%22%20target%3D%22_blank%22%20rel%3D%22noopener%22%3EAzure%20Sentinel%20correlation%20rules%20using%20lists%3C%2FA%3E%3C%2FEM%3E%3C%2FEM%3E%3C%2FFONT%3E%3CFONT%20color%3D%22%23000080%22%3E%3CEM%3E%3CEM%3E%26nbsp%3B(this%20post)%3C%2FEM%3E%3C%2FEM%3E%3C%2FFONT%3E%3C%2FSTRONG%3E%3C%2FFONT%3E%3C%2FLI%3E%0A%3CLI%3E%3CFONT%20color%3D%22%23FF00FF%22%3E%3CSTRONG%3E%3CEM%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fazure-sentinel%2Fazure-sentinel-correlation-rules-the-join-kql-operator%2Fba-p%2F1041500%22%20target%3D%22_blank%22%20rel%3D%22noopener%22%3EAzure%20Sentinel%20correlation%20rules%20using%20the%20join%20operator%3C%2FA%3E%3C%2FEM%3E%3C%2FSTRONG%3E%3C%2FFONT%3E%3C%2FLI%3E%0A%3CLI%3E%3CFONT%20color%3D%22%23FF00FF%22%3E%3CSTRONG%3E%3CEM%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fazure-sentinel%2Fimplementing-lookups-in-azure-sentinel-part-1-reference-files%2Fba-p%2F1091306%22%20target%3D%22_blank%22%20rel%3D%22noopener%22%3EImplementing%20Lookups%20in%20Azure%20Sentinel%3C%2FA%3E%26nbsp%3B%3C%2FEM%3E%3C%2FSTRONG%3E%3C%2FFONT%3E%3C%2FLI%3E%0A%3CLI%3E%3CFONT%20color%3D%22%23FF00FF%22%3E%3CSTRONG%3E%3CEM%3E%3CA%20id%3D%22link_7%22%20class%3D%22page-link%20lia-link-navigation%20lia-custom-event%22%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fazure-sentinel%2Fapproximate-partial-and-combined-lookups-in-azure-sentinel%2Fba-p%2F1393795%22%20target%3D%22_blank%22%20rel%3D%22noopener%22%3EApproximate%2C%20partial%20and%20combined%20lookups%3C%2FA%3E%3C%2FEM%3E%3C%2FSTRONG%3E%3C%2FFONT%3E%3C%2FLI%3E%0A%3CLI%3E%3CFONT%20color%3D%22%23FF00FF%22%3E%3CSTRONG%3E%3CEM%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fazure-sentinel%2Fhandling-sliding-windows-in-azure-sentinel-rules%2Fba-p%2F1505394%22%20target%3D%22_self%22%3EHandling%20sliding%20windows%20in%20Azure%20Sentinel%20rules%3C%2FA%3E%3C%2FEM%3E%3C%2FSTRONG%3E%3C%2FFONT%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId--1442972646%22%20id%3D%22toc-hId--1442972646%22%3EWhat%20are%20Active%20Lists%20and%20what%20they%20used%20for%3F%3C%2FH2%3E%0A%3CP%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EActive%20Lists%20are%20the%20ArcSight%20term%20for%2C%20well%2C%20tables.%20Unlike%20the%20main%20event%20repository%20tables%20which%20are%20immutable%20in%20a%20SIEM%2C%20Active%20Lists%20can%20be%20modified%2C%20for%20example%20by%20rules.%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3E%26nbsp%3B%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EActive%20Lists%20are%20mostly%20used%20for%20two%20purposes%3A%3C%2FSPAN%3E%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3ETo%20implement%20%22complex%22%20multi%20rule%20use%20cases%2C%20in%20which%20one%20rule%20updates%20the%20list%20and%20another%20one%20inspects%20it.%3C%2FSPAN%3E%3C%2FLI%3E%0A%3CLI%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EFor%20lookup%20tables%2C%20uploading%20reference%20information%20and%20looking%20it%20up%20as%20part%20of%20a%20rule.%3C%2FSPAN%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EIn%20this%20blog%20post%20I%20will%20focus%20on%20the%20former.%20Read%20-ERR%3AREF-NOT-FOUND-'Implementing%20Lookups%20in%20Azure%20Sentinel%22%20to%20learn%20about%20the%20latter.%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-1044540187%22%20id%3D%22toc-hId-1044540187%22%3EThe%20%22Failed%20AzureAD%20logons%20but%20success%20logon%20to%20AWS%20Console%22%20rule%3C%2FH2%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%222019-11-25%2011_39_51-Clipboard.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F158731i5FD9FACBE3D0502F%2Fimage-size%2Fmedium%3Fv%3D1.0%26amp%3Bpx%3D400%22%20title%3D%222019-11-25%2011_39_51-Clipboard.png%22%20alt%3D%222019-11-25%2011_39_51-Clipboard.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3ETo%20avoid%20a%20theoretical%20discussion%2C%20let's%20focus%20on%20a%20specific%20example.%20I%20picked%26nbsp%3B%3C%2FSPAN%3E-ERR%3AREF-NOT-FOUND-%3CSPAN%20data-preserver-spaces%3D%22true%22%3Ethis%20rule%3C%2FSPAN%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3E%26nbsp%3Bwhich%20-ERR%3AREF-NOT-FOUND-%40petebrayne%26nbsp%3Bfrom%20our%20research%20team%20wrote%20as%20it%20is%20a%20powerful%20use%20case%20yet%20simple%20enough%20for%20demonstration%20purposes.%20The%20rule%20is%20also%20built-in%20into%20Azure%20Sentinel.%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3E%26nbsp%3B%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EThe%20rule%20looks%20for%3A%3C%2FSPAN%3E%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EAn%20IP%20address%20from%20which%20there%20where%20many%20failed%20logins%20to%20AAD%2C%20presumably%20a%20brute%20force%20attempt.%3C%2FSPAN%3E%3C%2FLI%3E%0A%3CLI%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EA%20successful%20login%20to%20AWS%20console%20from%20the%20same%20IP%2C%20implying%20a%20potential%20breach.%3C%2FSPAN%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId--762914276%22%20id%3D%22toc-hId--762914276%22%3ETraditional%20Active%20List%20implementation%3C%2FH2%3E%0A%3CP%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EIn%20a%20traditional%20SIEM%2C%20this%20use%20case%20would%20usually%20require%20two%20rules%20and%20an%20Active%20list%20(see%26nbsp%3B%3C%2FSPAN%3E-ERR%3AREF-NOT-FOUND-%3CSPAN%20data-preserver-spaces%3D%22true%22%3Ehere%3C%2FSPAN%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3E%26nbsp%3Ban%20ArcSight%20implementation%20of%20a%20similar%20Brute%20Force%20rule%20or%20the%26nbsp%3B%3C%2FSPAN%3E-ERR%3AREF-NOT-FOUND-%3CSPAN%20data-preserver-spaces%3D%22true%22%3Eofficial%20ArcSight%20guide%3C%2FSPAN%3E%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EThe%20first%20identifies%20failed%20AAD%20logins%20and%20update%20the%20count%20of%20failed%20logins%20for%20an%20IP%20in%20an%20Active%20List.%3C%2FSPAN%3E%3C%2FLI%3E%0A%3CLI%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EThe%20second%20will%20identifies%20a%20successful%20AWS%20console%20login%20and%20checked%20if%20the%20IP%20address%20appears%20in%20the%20Active%20List%20and%20the%20count%20is%20above%20a%20threshold.%3C%2FSPAN%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EThis%20approach%20works%2C%20but%20it%20is%20far%20from%20trivial%20and%20is%20hard%20to%20maintain.%20It%20requires%20two%20unrelated%20rules%20and%20a%20separate%20Active%20List%20object.%20The%20actual%20time%20frames%20for%20the%20use%20case%20are%20actually%20the%20Active%20List%20record%20TTL%20rather%20than%20any%20rule%20property.%20Make%20the%20scenarios%20slightly%20more%20complex%20-%20by%20adding%20a%203rd%20event%20condition%20-%20and%20it%20becomes%20unmanageable.%26nbsp%3B%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-1724598557%22%20id%3D%22toc-hId-1724598557%22%3EAzure%20Sentinel%20make_list()%3C%2FH2%3E%0A%3CP%3ELet's%20look%20at%20Pete's%20rule.%20The%20first%20part%20take%20full%20advantage%20of%20the%20fact%20that%20query%20based%20rules%20can%20look%20back%20to%20create%20the%20list%20on%20demand%20without%20requiring%20a%20second%20rule%20and%20an%20intermediate%20object%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%3Elet%20signin_threshold%20%3D%205%3B%3CBR%20%2F%3Elet%20suspicious_signins%3CFONT%20color%3D%22%23FF00FF%22%3E%3CSTRONG%3E%20%3D%20%3C%2FSTRONG%3E%3C%2FFONT%3E%3CBR%20%2F%3ESigninLogs%3CBR%20%2F%3E%3CFONT%20color%3D%22%230000FF%22%3E%7C%20where%20TimeGenerated%20%26gt%3B%3D%20ago(1d)%3C%2FFONT%3E%3CBR%20%2F%3E%3CFONT%20color%3D%22%230000FF%22%3E%7C%20where%20ResultType%20!in%20(%220%22%2C%20%2250125%22%2C%20%2250140%22)%3C%2FFONT%3E%3CBR%20%2F%3E%3CFONT%20color%3D%22%230000FF%22%3E%7C%20where%20IPAddress%20!%3D%20%22127.0.0.1%22%3C%2FFONT%3E%3CBR%20%2F%3E%3CFONT%20color%3D%22%23008000%22%3E%7C%20summarize%20count()%20by%20IPAddress%3C%2FFONT%3E%3CBR%20%2F%3E%7C%20%3CFONT%20color%3D%22%23FF9900%22%3Ewhere%20count_%20%26gt%3B%20signin_threshold%3C%2FFONT%3E%3CBR%20%2F%3E%3CFONT%20color%3D%22%23CC99FF%22%3E%7C%20%3CFONT%20color%3D%22%23FF00FF%22%3Esummarize%20make_list(IPAddress)%3B%3C%2FFONT%3E%3C%2FFONT%3E%3C%2FPRE%3E%0A%3CP%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EWhat%20is%20this%20section%20doing%3F%3C%2FSPAN%3E%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EThe%20initial%26nbsp%3B%3C%2FSPAN%3E%3CSTRONG%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3E%22%3CFONT%20color%3D%22%230000FF%22%3Ewhere%3C%2FFONT%3E%22%3C%2FSPAN%3E%3C%2FSTRONG%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3E%26nbsp%3Bcloses%20filter%20out%20the%20relevant%20failure%20events%2C%20this%20would%20be%20similar%20to%20an%20ArcSight%20filter.%20Note%20that%20the%20time%20frame%20for%20the%20failed%20events%20is%20conveniently%20included%20in%20this%20clause%20using%20the%20term%26nbsp%3B%3C%2FSPAN%3E%3CEM%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3ETimeGenerated%20%26gt%3B%3D%20ago(1d).%3C%2FSPAN%3E%3C%2FEM%3E%3C%2FLI%3E%0A%3CLI%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EThe%20first%26nbsp%3B%3C%2FSPAN%3E%3CSTRONG%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3E%22%3CFONT%20color%3D%22%23008000%22%3Esummarize%3C%2FFONT%3E%22%3C%2FSPAN%3E%3C%2FSTRONG%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3E%26nbsp%3Bstatement%20counts%20the%20number%20of%20failed%20logins%20for%20each%20IP%20address%20and%20the%20following%20%22%3CFONT%20color%3D%22%23FF9900%22%3E%3CSTRONG%3Ewhere%3C%2FSTRONG%3E%3C%2FFONT%3E%22%20clause%20selects%20only%20those%20above%20a%20threshold.%20Note%20that%20the%20threshold%20can%20be%20dynamic%20implementing%20behavioral%20analytics%20using%20the%20technique%20described%26nbsp%3B%3C%2FSPAN%3E%3CA%20class%3D%22_e75a791d-denali-editor-page-rtfLink%22%20href%3D%22https%3A%2F%2Fcommunity.microfocus.com%2Ft5%2FArcSight-User-Discussions%2FRule-for-Successful-brute-force-login%2Ftd-p%2F1513168%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3Ehere%3C%2FSPAN%3E%3C%2FA%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3E.%20A%20fixed%20threshold%20is%20often%20just%20as%20good%2C%20and%20certainly%20easier%20for%20learning.%26nbsp%3B%3C%2FSPAN%3E%3C%2FLI%3E%0A%3CLI%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EThe%20second%26nbsp%3B%3C%2FSPAN%3E%3CSTRONG%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3E%22%3CFONT%20color%3D%22%23FF00FF%22%3Esummarize%3C%2FFONT%3E%22%3C%2FSPAN%3E%3C%2FSTRONG%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3E%26nbsp%3Bcreates%20the%20list.%20Using%20two%20summarize%20statements%20is%20an%20optimization%20that%20shaves%20execution%20time%20as%20the%20second%20summarize%20is%20more%20costly%20but%20will%20be%20used%20only%20on%20above%20threshold%20IP%20addresses.%26nbsp%3B%3C%2FSPAN%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EPretty%20straight%20forward%2C%20isn't%20it%3F%20We%20just%20created%20a%20list!%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId--82855906%22%20id%3D%22toc-hId--82855906%22%3EWrapping%20up%20the%20rule%3C%2FH2%3E%0A%3CP%3EThe%20second%20part%20of%20the%20rule%20only%20has%20to%20identify%20AWS%20logon%20failures%20(%3CFONT%20color%3D%22%230000FF%22%3EBlue%3C%2FFONT%3E)%20and%20then%20check%20if%20the%20IP%20address%20is%20in%20the%20list%20from%20step%201%20(%3CFONT%20color%3D%22%23FF00FF%22%3EMagenta%3C%2FFONT%3E).%20The%20rest%20(%3CFONT%20color%3D%22%23008000%22%3EGreen%3C%2FFONT%3E)%20is%20only%20used%20to%20enhance%20the%20information%20for%20the%20analyst%20use.%3C%2FP%3E%0A%3CPRE%3EAWSCloudTrail%3CBR%20%2F%3E%3CFONT%20color%3D%22%230000FF%22%3E%7C%20where%20TimeGenerated%20%26gt%3B%20ago(1d)%3C%2FFONT%3E%3CBR%20%2F%3E%3CFONT%20color%3D%22%230000FF%22%3E%7C%20where%20EventName%20%3D%3D%20%22ConsoleLogin%22%3C%2FFONT%3E%3CBR%20%2F%3E%3CFONT%20color%3D%22%230000FF%22%3E%7C%20extend%20LoginResult%20%3D%20tostring(parse_json(ResponseElements).ConsoleLogin)%20%3C%2FFONT%3E%3CBR%20%2F%3E%3CFONT%20color%3D%22%230000FF%22%3E%7C%20where%20LoginResult%20%3D%3D%20%22Success%22%3C%2FFONT%3E%3CBR%20%2F%3E%3CFONT%20color%3D%22%23FF00FF%22%3E%3CSTRONG%3E%7C%20where%20SourceIpAddress%20in%20(suspicious_signins)%3C%2FSTRONG%3E%3C%2FFONT%3E%3CBR%20%2F%3E%3CFONT%20color%3D%22%23008000%22%3E%7C%20extend%20Reason%20%3D%20%22Multiple%20failed%20AAD%20logins%20from%20IP%20address%22%3C%2FFONT%3E%3CBR%20%2F%3E%3CFONT%20color%3D%22%23008000%22%3E%7C%20extend%20MFAUsed%20%3D%20tostring(parse_json(AdditionalEventData).MFAUsed)%3C%2FFONT%3E%3CBR%20%2F%3E%3CFONT%20color%3D%22%23008000%22%3E%7C%20extend%20User%20%3D%20iif(UserIdentityUserName%20%3D%3D%20%22%22%2C%20UserIdentityType%2C%20UserIdentityUserName)%20%3C%2FFONT%3E%3CBR%20%2F%3E%3CFONT%20color%3D%22%23008000%22%3E%7C%20project%20TimeGenerated%2C%20Reason%2C%20LoginResult%2C%20EventTypeName%2C%20UserIdentityType%2C%20User%2C%20AWSRegion%2C%20SourceIpAddress%2C%20UserAgent%2C%20MFAUsed%3C%2FFONT%3E%3CBR%20%2F%3E%3CFONT%20color%3D%22%23008000%22%3E%7C%20extend%20timestamp%20%3D%20TimeGenerated%2C%20AccountCustomEntity%20%3D%20User%2C%20IPCustomEntity%20%3D%20SourceIpAddress%3C%2FFONT%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ESimple.%20Isn't%20it%3F%20Since%20the%20Azure%20Sentinel%20rule%20does%20not%20depend%20on%20a%20state%20machine%2C%20it%20is%20easier%20to%20build%2C%20test%20and%20maintain.%20%3CBR%20%2F%3E%3CBR%20%2F%3ENext%20time%20we%20will%20discuss%20the%20other%20use%20of%20Active%20Lists%3A%20lookups.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-1029225%22%20slang%3D%22en-US%22%3E%3CP%3EWriting%20alert%20rules%20using%20KQL%20is%20powerful%2C%20but%20does%20not%20have%20to%20be%20complicated.%20A%20good%20example%20would%20be%20rules%20that%20in%20traditional%20SIEM%20use%20Active%20Lists%20In%20this%20blog%20post%20I%20will%20describe%20how%20to%20avoid%20Active%20Lists%20entirely%20using%20Sentinel%20query-based%20rules.%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1029225%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EDetection%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EQuery%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Microsoft

 

Writing alert rules using KQL is powerful, but does not have to be complex. A good example would be rules that in traditional SIEM use Active Lists (or Reference Sets, depending on your SIEM). When presenting KQL rules, I am often asked how to implement Active Lists in Sentinel. While replicating Active Lists in Sentinel is a good topic for another blog post, I will focus here on avoiding Active Lists in the first place using Sentinel query based rules. You just don't need them in most cases.

 

This post of part of a series of blog posts on writing rules in Azure Sentinel:

 

What are Active Lists and what they used for?

Active Lists are the ArcSight term for, well, tables. Unlike the main event repository tables which are immutable in a SIEM, Active Lists can be modified, for example by rules.

 

Active Lists are mostly used for two purposes:

  • To implement "complex" multi rule use cases, in which one rule updates the list and another one inspects it.
  • For lookup tables, uploading reference information and looking it up as part of a rule.

In this blog post I will focus on the former. Read 'Implementing Lookups in Azure Sentinel" to learn about the latter.

 

The "Failed AzureAD logons but success logon to AWS Console" rule

2019-11-25 11_39_51-Clipboard.png

 

To avoid a theoretical discussion, let's focus on a specific example. I picked this rule which @petebrayne from our research team wrote as it is a powerful use case yet simple enough for demonstration purposes. The rule is also built-in into Azure Sentinel.

 

The rule looks for:

  • An IP address from which there where many failed logins to AAD, presumably a brute force attempt.
  • A successful login to AWS console from the same IP, implying a potential breach.

 

Traditional Active List implementation

In a traditional SIEM, this use case would usually require two rules and an Active list (see here an ArcSight implementation of a similar Brute Force rule or the official ArcSight guide

  • The first identifies failed AAD logins and update the count of failed logins for an IP in an Active List.
  • The second will identifies a successful AWS console login and checked if the IP address appears in the Active List and the count is above a threshold.

This approach works, but it is far from trivial and is hard to maintain. It requires two unrelated rules and a separate Active List object. The actual time frames for the use case are actually the Active List record TTL rather than any rule property. Make the scenarios slightly more complex - by adding a 3rd event condition - and it becomes unmanageable. 

 

Azure Sentinel make_list()

Let's look at Pete's rule. The first part take full advantage of the fact that query based rules can look back to create the list on demand without requiring a second rule and an intermediate object:

 

let signin_threshold = 5;
let suspicious_signins =
SigninLogs
| where TimeGenerated >= ago(1d)
| where ResultType !in ("0", "50125", "50140")
| where IPAddress != "127.0.0.1"
| summarize count() by IPAddress
| where count_ > signin_threshold
| summarize make_list(IPAddress);

What is this section doing?

  • The initial "where" closes filter out the relevant failure events, this would be similar to an ArcSight filter. Note that the time frame for the failed events is conveniently included in this clause using the term TimeGenerated >= ago(1d).
  • The first "summarize" statement counts the number of failed logins for each IP address and the following "where" clause selects only those above a threshold. Note that the threshold can be dynamic implementing behavioral analytics using the technique described here. A fixed threshold is often just as good, and certainly easier for learning. 
  • The second "summarize" creates the list. Using two summarize statements is an optimization that shaves execution time as the second summarize is more costly but will be used only on above threshold IP addresses. 

Pretty straight forward, isn't it? We just created a list!

 

Wrapping up the rule

The second part of the rule only has to identify AWS logon failures (Blue) and then check if the IP address is in the list from step 1 (Magenta). The rest (Green) is only used to enhance the information for the analyst use.

AWSCloudTrail
| where TimeGenerated > ago(1d)
| where EventName == "ConsoleLogin"
| extend LoginResult = tostring(parse_json(ResponseElements).ConsoleLogin)
| where LoginResult == "Success"
| where SourceIpAddress in (suspicious_signins)
| extend Reason = "Multiple failed AAD logins from IP address"
| extend MFAUsed = tostring(parse_json(AdditionalEventData).MFAUsed)
| extend User = iif(UserIdentityUserName == "", UserIdentityType, UserIdentityUserName)
| project TimeGenerated, Reason, LoginResult, EventTypeName, UserIdentityType, User, AWSRegion, SourceIpAddress, UserAgent, MFAUsed
| extend timestamp = TimeGenerated, AccountCustomEntity = User, IPCustomEntity = SourceIpAddress

 

Simple. Isn't it? Since the Azure Sentinel rule does not depend on a state machine, it is easier to build, test and maintain.

Next time we will discuss the other use of Active Lists: lookups.

7 Comments
Super Contributor

Keep the great blog posts coming! :smile:

Microsoft

Excellent article @Ofer_Shezaf and I especially liked the colour coding

note: https://docs.microsoft.com/en-us/azure/kusto/query/makelist-aggfunction you may want to use make_list() rather than makelist() for large scale scenarios (makelist has a 128 default limit).

Microsoft

I switched from make_list, which I normally use, to makelist to adjust to the github query, but was not ware of the difference. I will discuss with our research team if we need to modify the original.

Very useful @Ofer_Shezaf thanks for sharing!

Contributor

Hi @Ofer_Shezaf ,

 

Thanks for a very good post.

 

I am also interested in "A future post will discuss implementing lookup tables in Azure Sentinel.". I am very new to Azure Sentinel and want to integrate custom threat intelligence from our company's website.

 

If I download the TI feeds from our website and paste it somewhere on my local machine, then how can I update those feeds in Active lists (or similar in Sentinel) and call them against rules.

 

Thanks in Advance.


Regards,

Mitesh Agrawal

Microsoft

Hi @MiteshAgrawal , we already published this blog post, or at least the first in a series, here.

Contributor

Thanks @Ofer_Shezaf . The post is very useful. My requirement can be met with the link you provided. 

 

One more query I have is whether can I read data from a file in my local machine if I do not want to create a BLOB object?

 

Regards,

Mitesh Agrawal