Azure Sentinel how to clear Threat Intelligence Indicator table

%3CLINGO-SUB%20id%3D%22lingo-sub-3251559%22%20slang%3D%22en-US%22%3EAzure%20Sentinel%20how%20to%20clear%20Threat%20Intelligence%20Indicator%20table%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3251559%22%20slang%3D%22en-US%22%3E%3CP%3E%3CSPAN%3EIs%20there%20a%20way%20to%20do%20a%20bulk%20delete%20of%20all%20indicators%3F%20I%20have%20the%20DShieldScanningIPs%26nbsp%3B%20source%20with%20over%20100%20thousand%26nbsp%3BIP%26nbsp%3Band%20I'd%20like%20to%20delete%20them%20all%20but%20it%20appears%20I%20can%20only%20delete%20100%20of%20them%20at%20a%20time.%20This%20will%20take%20a%20while.%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-3251559%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAlerts%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EDashboards%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EData%20Collection%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ELog%20Data%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EMigration%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ESolutions%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EThreat%20Intelligence%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-3256622%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Sentinel%20how%20to%20clear%20Threat%20Intelligence%20Indicator%20table%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3256622%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F1328167%22%20target%3D%22_blank%22%3E%40william890%3C%2FA%3E%26nbsp%3BHow%20comfortable%20are%20you%20with%20making%20REST%20API%20calls%3F%26nbsp%3B%20The%20call%20to%20delete%20a%20single%20IOC%20is%3A%3C%2FP%3E%3CPRE%20class%3D%22lia-code-sample%20language-applescript%22%3E%3CCODE%3Ehttps%3A%2F%2Fmanagement.azure.com%2Fsubscriptions%2F%3CSUBSCRIPTION%3E%2FresourceGroups%2F%26gt%3BResourceGroup%26gt%3B%2Fproviders%2FMicrosoft.OperationalInsights%2Fworkspaces%2F%3CWORKSPACENAME%3E%2Fproviders%2FMicrosoft.SecurityInsights%2Fthreatintelligence%2Fmain%2Findicators%2F%3CINDICATORGUID%3E%3Fapi-version%3D2019-01-01-preview%3C%2FINDICATORGUID%3E%3C%2FWORKSPACENAME%3E%3C%2FSUBSCRIPTION%3E%3C%2FCODE%3E%3C%2FPRE%3E%3CP%3E%26nbsp%3BThere%20is%20also%20a%20%22queryIndicators%22%20call%20that%20will%20allow%20you%20to%20filter%20what%20you%20see%20by%20source%20and%20other%20properties.%26nbsp%3B%20Go%20to%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fgithub.com%2FAzure%2Fazure-rest-api-specs%2Ftree%2Fmain%2Fspecification%2Fsecurityinsights%2Fresource-manager%2FMicrosoft.SecurityInsights%2Fpreview%2F2019-01-01-preview%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3Eazure-rest-api-specs%2Fspecification%2Fsecurityinsights%2Fresource-manager%2FMicrosoft.SecurityInsights%2Fpreview%2F2019-01-01-preview%20at%20main%20%C2%B7%20Azure%2Fazure-rest-api-specs%20(github.com)%3C%2FA%3E%26nbsp%3Bfor%20more%20information%20on%20it.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-3255690%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Sentinel%20how%20to%20clear%20Threat%20Intelligence%20Indicator%20table%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3255690%22%20slang%3D%22en-US%22%3EFunny%20enough%20I%20have%20this%20exact%20problem%2C%20DShield%20throws%20so%20many%20FP%20when%20mapping%20to%20signin%20events%20etc.%20I%20couldn%E2%80%99t%20find%20a%20way%20to%20bulk%20delete%20sadly%2C%20after%20searching%20high%20and%20low.%3CBR%20%2F%3E%3CBR%20%2F%3EI%20ended%20up%20editing%20the%20query%20to%20basically%20!%3D%20DShield%20and%20then%20wait%20for%20the%20retention%20to%20kick%20in%20and%20remove.%3CBR%20%2F%3E%3CBR%20%2F%3EWill%20be%20interested%20if%20someone%20comes%20with%20an%20answer%20to%20bill%20delete%20though!%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-3251608%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Sentinel%20how%20to%20clear%20Threat%20Intelligence%20Indicator%20table%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3251608%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F46875%22%20target%3D%22_blank%22%3E%40Gary%20Bushey%3C%2FA%3E%26nbsp%3BNo%2C%20I%20want%20to%20get%20rid%20of%20all%20data%20from%20Threat%20Intelligence%20from%20a%20specific%20source%20(in%20this%20case%20%22%3CSPAN%3EDShieldScanningIPs%3C%2FSPAN%3E%22)%20which%20is%20no%20longer%20useful%20for%20me.%20I%20still%20have%20other%20sources%20data%20that%20I%20want%20to%20keep.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-3251598%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Sentinel%20how%20to%20clear%20Threat%20Intelligence%20Indicator%20table%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3251598%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F1328167%22%20target%3D%22_blank%22%3E%40william890%3C%2FA%3E%26nbsp%3BIf%20it%20is%20old%20data%20you%20want%20to%20get%20rid%20of%20and%20you%20always%20only%20want%20to%20keep%20the%20newer%20data%20you%20can%20set%20data%20type%20retention%3A%26nbsp%3B%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Flogs%2Fmanage-cost-storage%23retention-by-data-type%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3EManage%20usage%20and%20costs%20for%20Azure%20Monitor%20Logs%20-%20Azure%20Monitor%20%7C%20Microsoft%20Docs%3C%2FA%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThis%20way%20you%20can%20set%20your%20table%20to%20only%20hold%2030%20days%20for%20example%2C%20while%20the%20other%20tables%20will%20still%20retain%2090%20days%3C%2FP%3E%3C%2FLINGO-BODY%3E
New Contributor

Is there a way to do a bulk delete of all indicators? I have the DShieldScanningIPs  source with over 100 thousand IP and I'd like to delete them all but it appears I can only delete 100 of them at a time. This will take a while.

4 Replies

@william890 If it is old data you want to get rid of and you always only want to keep the newer data you can set data type retention:  Manage usage and costs for Azure Monitor Logs - Azure Monitor | Microsoft Docs

 

This way you can set your table to only hold 30 days for example, while the other tables will still retain 90 days

@Gary Bushey No, I want to get rid of all data from Threat Intelligence from a specific source (in this case "DShieldScanningIPs") which is no longer useful for me. I still have other sources data that I want to keep.

Funny enough I have this exact problem, DShield throws so many FP when mapping to signin events etc. I couldn’t find a way to bulk delete sadly, after searching high and low.

I ended up editing the query to basically != DShield and then wait for the retention to kick in and remove.

Will be interested if someone comes with an answer to bill delete though!

@william890 How comfortable are you with making REST API calls?  The call to delete a single IOC is:

https://management.azure.com/subscriptions/<Subscription>/resourceGroups/>ResourceGroup>/providers/Microsoft.OperationalInsights/workspaces/<workspacename>/providers/Microsoft.SecurityInsights/threatintelligence/main/indicators/<indicatorGUID>?api-version=2019-01-01-preview

 There is also a "queryIndicators" call that will allow you to filter what you see by source and other properties.  Go to azure-rest-api-specs/specification/securityinsights/resource-manager/Microsoft.SecurityInsights/prev... for more information on it.