Forum Discussion
Backing up Sentinel and the Security subscription
A lot of people ask about how Security Operations can effectively back up all of the Sentinel related objects. One option is to use GitHub or Azure DevOps pipelines to get a daily backup. I've been doing this for a very long time and it seems like a good forum to share that code.
The trick behind it has been to use PowerShell to derive the current API versions for Azure objects. Once you do that, you can recursively download the whole subscription to a repo and then scripts can renerate reports using markdown and yaml. I've been backing up my subscription reliably since 2021.
The default project creates reports for all the Sentinel related elements.
Markdown lets the object reports be drilled down into...
And KQL is presented as YAML for readability.
It's actually easy to deploy all the backedup JSON files through REST if needed but for most of us, being able to have readable KQL and Git History of changes in files is probably all we need.
This project is completely written in PowerShell with no compiled modules & anyone is freely welcome to it.
I've written more about it here:
Daily Azure / Sentinel Backup (and Reporting) with GitHub
... and the source code and install documentation can be found here:
https://github.com/LaurieRhodes/PUBLIC-Subscription-Backup
I hope this is of use to the community! 🙂
Best Regards
Laurie
- Laurie_Rhodes thanks for sharing it. Is there a way to save the KQL functions stored in Sentinel in Github ?
Hi abon13 - In depends on what you mean!
Maybe the first question is what is actually "in" Sentinel?If we want Sentinel's notebooks, we are really installing a Machine Learning Workspace. Playbooks are Logic Apps. The substance of "Sentinel" is a Log Analytics workspace although all these things come together as an end-to-end solution we refer to as Sentinel.
Others would argue the opposite, that Sentinel really is a particular "solution" object that gets deployed onto a Log Analytics Workspace. It's this solution that contains the core aspects of SIEM that can't be built using other Azure services.
In that context, I've had to think about your questions as to what KQL "functions" are "in" Sentinel.
- The KQL queries (functions / parsers) we write are saved to Query Packs and Query Packs can be saved to GitHub. We see them as functions in Sentinel... but they aren't really in Sentinel at all. I do back these up currently.
- I've commented out the saving of hunting "bookmarks" with the PowerShell backup script - which are potentially KQL "functions" - just because it's too sensitive but they can be added back in if you wish.
However, if you are referring to the functions that exist within the Microsoft SecurityInsights Solution under the "Microsoft Sentinel" heading...
I don't think they are exposed for API access. All of these are managed and maintained by Microsoft and that management is part of the value of Sentinel. Because they are managed and kept up to date there is little point of backing them up as you'll never be able to write / restore into that Microsoft managed component anyway.
If you are really up for an interesting challenge, most of the functions are ASIM parsers and you can script and deploy them directly from the Azure-Sentinel GitHub. There is a big data use case for doing this with Azure Data Explorer and firms aiming to ingest hundreds of GB, or many TB of Security Logs a day for years of storage (like the m-21-31 Federal expectation in the US). This path is complex with a high overhead beyond the scope of benefit for most firms... but it's very cool when its working and it lets SecOps ingest everything while still passing high value data to Sentinel. 🙂
- Laurie_Rhodes This helps. My question was mainly around the ones that gets saved to Query Packs
