In the ever-evolving landscape of cybersecurity, both automation and infrastructure-as-code (IaC) have become indispensable. SIEM solutions, traditionally known for their complex configurations and manual deployments, are no exception. Microsoft Sentinel's repository support is taking a significant leap forward in empowering security teams to embrace SIEM-as-code practices.
Some of the added value that organizations can capitalize on immediately are:
- Reduction of human error
- Faster deployment and configurations
- Consistent operational process leading to increased security value
Organizations with simple environments, more complex environments, or MSSPs servicing multiple environments, all have use cases that can be satisfied with repositories.
This blog post will delve into Microsoft Sentinel repositories, explaining what they are, how they work, and some best practices that you can begin to take advantage of today.
What are Microsoft Sentinel Repositories?
Microsoft Sentinel Repositories allow you to create and manage your custom content from an external source control repository for continuous integration / continuous delivery (CI/CD). Currently, Microsoft Sentinel supports Azure DevOps and GitHub. Repositories thus provide a way to automate Microsoft Sentinel deployment and operations of your custom content to your Microsoft Sentinel environments.
This means you can:
- Control versioning: Track changes to your Microsoft Sentinel configurations and enable easy rollbacks.
- Collaborate: Foster teamwork by allowing multiple security engineers to contribute to and review Microsoft Sentinel content.
- Automate deployments: Implement CI/CD pipelines to automate the deployment of Microsoft Sentinel content to your environment.
- Centralized management: Store and manage all your Microsoft Sentinel configurations in a single, accessible location.
- Control approvals: Ensure that only approved changes in code are pushed to the platform, reducing mistakes that lead to a spike in alerts generated or unintentionally missing detection of events.
- Run full change audits: Track all changes within the platform along with changes in code.
Updates made to the content in your external repositories are synced to your Microsoft Sentinel workspace and overwrite any changes you make to that content through the Microsoft Sentinel portal. The external repository becomes your "single source of truth" for custom content in the connected workspaces.
Please note:
If you make changes outside of the repository, it is important to ensure that the content in your repository is updated accordingly and that your deployments are happening as you expect them to.
Use cases Microsoft customers find useful:
Customers commonly use analytics rules as follows:
- Interactive/GUI: Customers use the portal to manage analytics rules
- API: Customers use Microsoft APIs to update and create analytic rules
- Orchestration tools: Customers use external tools or SOAR tools, like Azure Logic Apps
- Content-as-code: Customers use our repository support, which can automatically manage content in Microsoft Sentinel
According to our customers: Managing a large number of clients often means repetitive clicks and interface navigation. Instead, the preferred approach is to have an automated pipeline that seamlessly executes tasks tailored to each client.
Maintaining versions of content:
Organizations can track changes of content: who and when the content was changed and the ability to compare the changes. This will help Security Operations teams to maintain the desired consistency of a set catalogue of analytics, playbooks, and workbooks across workspaces.
Use cases for managing multiple business units or clients using repositories
The following section presents use cases that our users of repositories find useful for managing their environments through code.
One repository for all, and one repository per business unit
There are times when an organization is in need of replicating the same set of Microsoft Sentinel content across multiple environments, whether it's the newest set of detections related to a popular cybersecurity incident, or it's a set of popular hunting queries.
Having one generic content repository for all environments allows organizations to connect multiple environments to a centrally managed workspace. This allows the organization to add generic content to that repository without having to deploy the content manually into each business unit's repository.
To complement this central repository, organizations can create a specific repository for each department or environment that needs tailored content and can include their specific content there. Those organization's repositories only need to be modified when a business unit needs to modify their specific content. This results in each organization's workspace having two connections, one to the centrally managed repository with generic content, and another one that's managed specifically for the different departments or environments it services. Here is a high-level diagram for this option:
This structure is best for organizations that have many environments with different needs/requirements but can also work well for MSSPs that have a balance of both content-for-all, and content tailored to specific business units.
Tip:
This same structure can be applied at a branch level if that's preferable to having multiple repositories. In this case, you'd have one repository with a main branch for all, and a specific branch per business unit.
When naming your folders, use different prefixes like "Global-[customname]" or "BU1-[customname]" to help track where content should be pushed or deployed.
One repository for all with custom folders per business unit
Some organizations prefer not to manage multiple repositories and/or prefer to group their content based on the shared data sources as opposed to splitting it for each entity that needs to be served. In those cases, another structure to consider is having all your content in one repository and connecting all the different entities or business units to that same repository. Organizations can then specify which folders each entity's connection deploys from by customizing each connection's workflow or pipeline definitions. Below is a high-level diagram for this option, and for more information, see the Microsoft Sentinel documentation:
This structure offers the flexibility and maintenance of a single repository, but requires more upfront work during the onboarding stage to ensure that each connection is properly mapped to the folder(s) it should be deploying content from.
Organizations can opt to structure their folders based on the content they have. For example, a Microsoft Entra ID Analytics folder can map to any connections that need Microsoft Entra ID content. Another approach is naming your folders based on the business unit(s) that this folder serves (such as 'Business Unit X folder'). This folder should store all of the business unit's content and be mapped to their connection(s).
Please note:
Note that anytime you update an existing connection from the portal wizard, such as to include more or fewer content types, any workflow/pipeline customization is reset.
One repository per business unit or entity
If your business units don't have any overlap in their Microsoft Sentinel content needs, or the idea of a repository where multiple business units share the same content isn't appealing, simply creating a repository per business unit might be the best method to use for managing your CICD pipelines. This allows for full separation of content across your business units and can best serve BUs with very specific needs across their workspaces. Here is a high-level diagram for this option:
These are just a few examples of folder structures that have worked for many of Microsoft's customers and partners using repositories for content-as-code, and there are certainly many more that can be used to best optimize your scenarios.
Testing and developing content prior to production
Organizations have found it useful to be able to control what environments to push content to and test before pushing to production. The challenge that organizations ran into was creating an analytics rule that might unintentionally create many alerts.
To further customize your CI/CD pipelines with any structure you use, you can use configuration files with each repository branch to prioritize the deployment of high-priority content, exclude any content you want to avoid deploying, or map parameter files to their respective content files. Learn more repositories configuration files in this article.
Getting started with content-as-code
Prerequisites
To start using Microsoft Sentinel repositories, there are some prerequisites which are:
- A Microsoft Sentinel workspace.
- A Git repository (GitHub or Azure DevOps).
- Appropriate permissions to connect the repository to Sentinel.
Overview of the process
- Connect a repository: You link your chosen Git repository to your Microsoft Sentinel workspace.
- Organize content: Structure your repository with folders and files representing your Microsoft Sentinel content.
- Configure deployment: Define the deployment settings, such as the branch to use and the deployment scope.
- Automate or manually deploy: Trigger deployments manually or set up automated pipelines to push changes to Microsoft Sentinel.
- Monitor and manage: Track deployment status and manage your repository connections within the Microsoft Sentinel interface.
Navigate to your tenant at security.microsoft.com. Scroll to Microsoft Sentinel and click on Content Management, then Repositories.
Add a new connection
Give a name and description to your connection
Sign into GitHub to connect to Sentinel
Provide authorization to Microsoft Sentinel to pull data from your GitHub account
The Future of SIEM-as-Code
Microsoft Sentinel repositories represent a significant step towards the future of managing your SIEM deployments with code. By embracing DevOps principles and providing native integration with Git repositories, Microsoft Sentinel empowers security teams to automate their workflows and achieve greater agility.
As the threat landscape continues to evolve, the ability to rapidly deploy and adapt security configurations will become increasingly critical. Microsoft Sentinel repositories provide the foundation for a more automated, collaborative, and efficient security operations model.
I'd like to thank Ofer Shezaf & Brian Brekkan for their contributions and guidance on this blog post.
Microsoft
