Home
%3CLINGO-SUB%20id%3D%22lingo-sub-657097%22%20slang%3D%22en-US%22%3EImporting%20Sigma%20Rules%20to%20Azure%20Sentinel%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-657097%22%20slang%3D%22en-US%22%3E%3CP%3EThe%20notebook%20accompanying%20this%20article%20is%20a%20quick%20and%20dirty%20Sigma%20rule%20to%20Log%20Analytics%20converter.%20It%20uses%20functionality%20from%20the%20%3CSTRONG%3E%3CEM%3Esigmac%3C%2FEM%3E%3C%2FSTRONG%3E%20tool%20to%20do%20the%20conversion.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH1%20id%3D%22toc-hId-1960502040%22%20id%3D%22toc-hId-1960502040%22%3EWhat%20are%20Sigma%20Rules%3F%3C%2FH1%3E%0A%3CP%3E%3CSTRONG%3ESigma%3C%2FSTRONG%3E%20rules%20are%20an%20implementation-independent%20way%20of%20specifying%20detection%20signatures%20for%20attacker%20activity.%20Each%20rule%20specifies%20a%20data%20source%20and%20a%20set%20of%20conditions%20that%20need%20to%20be%20met%20to%20satisfy%20a%20detection%20condition.%20A%20simple%20example%20might%20be%20a%20rule%20that%20specifies%20Windows%20security%20events%20as%20the%20source%20and%20process%20name%20equal%20to%20%E2%80%9Cmalware.exe%E2%80%9D.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ESigma%20provides%20converters%20to%20render%20these%20rules%20into%20queries%20that%20can%20be%20deployed%20in%20a%20security%20operations%20environment.%20For%20example%2C%20the%20previous%20logic%20can%20be%20converted%20into%20the%20equivalent%20rule%20for%20Splunk%2C%20ArcSight%2C%20LogAnalytics%2FAzure%20Sentinel%2C%20and%20many%20others.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSTRONG%3ESigmac%3C%2FSTRONG%3E%20is%20the%20python%20command%20line%20tool%20that%20performs%20the%20conversion%2C%20allowing%20you%20to%20specify%20the%20target%20format%20(%3CEM%3Ebackend%3C%2FEM%3E%20in%20%3CSTRONG%3ESigma%3C%2FSTRONG%3E%20terminology)%20and%20configuration%20and%20mapping%20files%20to%20help%20with%20the%20conversion.%20This%20notebook%20doesn%E2%80%99t%20do%20significantly%20more%20than%20the%20tool%20but%20it%20does%20allow%20interactive%20browsing%20and%20execution%20of%20the%20rules%20in%20an%20Azure%20Sentinel%20environment.%3C%2FP%3E%0A%3CH1%20id%3D%22toc-hId--591654921%22%20id%3D%22toc-hId--591654921%22%3EThis%20Notebook%3C%2FH1%3E%0A%3CP%3EThe%20notebook%20can%20be%20found%20%3CA%20href%3D%22https%3A%2F%2Fgithub.com%2FAzure%2FAzure-Sentinel%2Fblob%2Fmaster%2FNotebooks%2FSigmaRuleImporter.ipynb%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehere%3C%2FA%3E.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThe%20notebook%20uses%20%3CSTRONG%3Esigmac%3C%2FSTRONG%3E%20functionality%20to%20do%20conversions%20into%20KQL%20(Log%20Analytics%2FAzure%20Sentinel%20native%20query%20language)%20but%20has%20the%20following%20additional%20features%3A%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3EDownloads%20the%20full%20set%20of%20contributed%20sigma%20rules%20from%20the%20sigma%20GitHub%20repo%3C%2FLI%3E%0A%3CLI%3EAllows%20browsing%20and%20display%20of%20rules%20organized%20by%20the%20repo%20folder%20structure%3C%2FLI%3E%0A%3CLI%3EThe%20browser%20shows%20both%20the%20original%20sigma%20syntax%20and%20the%20converted%20KQL%20syntax%3C%2FLI%3E%0A%3CLI%3EYou%20can%20optionally%20add%20filter%20clauses%20for%20date%20range%20and%20hostname%20that%20are%20added%20to%20the%20output%20query.%3C%2FLI%3E%0A%3CLI%3EYou%20can%20execute%20each%20rule%2C%20specifying%20a%20time%20range%20if%20needed%2C%20and%20see%20the%20results%20in%20the%20notebook.%3C%2FLI%3E%0A%3CLI%3EThe%20conversion%20copies%20rule%20metadata%20into%20comments%20in%20the%20target%20KQL.%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-954641909%22%20id%3D%22toc-hId-954641909%22%3EDownloading%20Sigma%20Rules%3C%2FH2%3E%0A%3CP%3EEnter%20a%20path%20where%20you%20want%20to%20download%20the%20sigma%20rules%20and%20code%20to.%20This%20defaults%20to%20a%20folder%20named%20%E2%80%9Csigma%E2%80%9D%20in%20the%20current%20directory.%20The%20folder%20will%20be%20created%20if%20it%20does%20not%20exist.%20If%20the%20folder%20does%20exist%20the%20download%20and%20extraction%20will%20overwrite%20any%20files%20with%20same%20name%20%E2%80%93%20so%20if%20you%20have%20edited%20files%20here%2C%20save%20them%20somewhere%20else%20before%20executing%20the%20cell.%3C%2FP%3E%0A%3CP%3EIf%20the%20download%20is%20successful%20you%20should%20see%20a%20folder%20structure%20like%20the%20one%20show%20above.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F116666iA0E01FF9D5841EAA%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20alt%3D%22Repodownload.png%22%20title%3D%22Repodownload.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId--1597515052%22%20id%3D%22toc-hId--1597515052%22%3EConvert%20the%20Rules%3C%2FH2%3E%0A%3CP%3EThe%20next%20few%20cells%20import%20the%20required%20classes%20from%20the%20sigmac%20source%20code%2C%20define%20a%20few%20helper%20functions%20and%20then%20tries%20to%20covert%20each%20rule%20in%20turn%20and%20store%20the%20results%20in%20a%20python%20dictionary.%20Any%20rules%20that%20cannot%20be%20converted%20result%20in%20the%20string%20%E2%80%9CNot%20convertible%E2%80%9D%20in%20the%20output%20field.%20For%20limitations%20in%20conversion%20see%20the%20Limitations%20section%20at%20the%20end%20of%20this%20article.%3C%2FP%3E%0A%3CP%3EYou%20can%20see%20that%20the%20convertible%20rules%20are%20currently%20predominantly%20Windows%20rules%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F116667i0ADF9A1DE6BDB4FB%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20alt%3D%22Conversion.png%22%20title%3D%22Conversion.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-145295283%22%20id%3D%22toc-hId-145295283%22%3EBrowsing%20Rules%3C%2FH2%3E%0A%3CP%3EYou%20can%20select%20a%20category%20(these%20are%20derived%20from%20the%20folder%20structure)%20and%20view%20the%20converted%20KQL%20if%20conversion%20was%20possible.%26nbsp%3B%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F116668iE8342582F4CE3AF5%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20alt%3D%22RuleBrowser.png%22%20title%3D%22RuleBrowser.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThere%20are%20checkboxes%20to%20add%20option%20filters%20for%20specific%20date%20ranges%20and%20hostname.%20This%20is%20shown%20in%20the%20following%20screen%20shot.%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F116669i9297CBDDD8B84773%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20alt%3D%22RuleBrowserDateFilter.png%22%20title%3D%22RuleBrowserDateFilter.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EConverted%20rules%20can%20be%20individually%20selected%20and%20copied%20from%20the%20lower%20pane%20and%20saved%20in%20text%20files.%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-1888105618%22%20id%3D%22toc-hId-1888105618%22%3EExecuting%20a%20Rule%3C%2FH2%3E%0A%3CP%3EBefore%20you%20can%20run%20a%20converted%20rule%2C%20you%20need%20to%20authenticate%20to%20Azure%20Sentinel%20and%20run%20a%20few%20more%20cells%20to%20define%20functions%20used%20for%20executing%20the%20query.%3C%2FP%3E%0A%3CP%3EIf%20you%20have%20previously%20run%20any%20of%20the%20Azure%20Sentinel%20notebooks%2C%20you%20may%20already%20have%20a%20config.json%20file%20containing%20your%20Workspace%20and%20Tenant%20IDs.%20If%20not%2C%20paste%20your%20workspace%20ID%20into%20the%20text%20box%20supplied.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F116670iDFFFB07A62925E33%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20alt%3D%22WorkspaceId.png%22%20title%3D%22WorkspaceId.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EExecuting%20the%20next%20cell%20and%20follow%20the%20prompts%20to%20authenticate.%26nbsp%3BNow%20you%20are%20ready%20to%20execute%20the%20converted%20queries.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EGo%20back%20to%20the%20browser%20and%20the%20cell%20below%2C%20which%20contains%20an%20%3CSTRONG%3EExecute%20query%3C%2FSTRONG%3E%20button.%20Select%20a%20query%20in%20the%20browser%20(obviously%2C%20non-convertible%20rules%20won%E2%80%99t%20run%20in%20Azure%20Sentinel)%20and%20click%20the%20%3CSTRONG%3EExecute%20Query%3C%2FSTRONG%3E%20button.%20Unless%20your%20environment%20is%20very%20badly%20infected%2C%20you%20will%20probably%20see%20no%20results%20for%20the%20majority%20of%20these%20queries.%20If%20you%20have%20opted%20to%20include%20a%20date%20filter%20you%20can%20use%20the%20date%20range%20selector%20widget%20immediately%20below%20the%20Execute%20query%20cell%20to%20choose%20the%20date%20range.%20If%20you%20do%20not%20include%20a%20filter%20the%20query%20will%20run%20over%20the%20default%20time%20range%20for%20Log%20Analytics%2C%20which%20is%2024%20hours.%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F116671i371CDDF6DB444EC9%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20alt%3D%22RunQuery.png%22%20title%3D%22RunQuery.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EAlthough%20you%20may%20see%20no%20results%20in%20your%20environment%2C%20executing%20the%20queries%20does%20at%20least%20ensure%20that%20the%20conversion%20has%20been%20successful.%20You%20might%20want%20to%20simulate%20some%20of%20the%20activity%20in%20the%20rules%20to%20ensure%20that%20you%20are%20getting%20results%20as%20expected.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ESome%20things%20to%20look%20out%20for%20include%20the%20use%20of%20incorrectly%20escaped%20backslash%20(%E2%80%9C%5C%E2%80%9D)%20in%20file%20paths.%20Braces%20(%E2%80%9C%7B%E2%80%9C%20and%20%E2%80%9C%7D%E2%80%9D)%20embedded%20in%20strings%20may%20also%20conflict%20with%20the%20replaceable%20parameter%20syntax.%3C%2FP%3E%0A%3CH1%20id%3D%22toc-hId--467537838%22%20id%3D%22toc-hId--467537838%22%3ELimitations%3C%2FH1%3E%0A%3CP%3EOnly%20a%20subset%20of%20the%20Sigma%20rules%20are%20convertible%20currently.%20Failure%20to%20convert%20could%20be%20for%20one%20or%20more%20of%20these%20reasons%3A%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3Eknown%20limitations%20of%20the%20converter%3C%2FLI%3E%0A%3CLI%3Emismatch%20between%20the%20syntax%20expressible%20in%20Sigma%20and%20KQL%3C%2FLI%3E%0A%3CLI%3Edata%20sources%20referenced%20in%20Sigma%20rules%20do%20not%20yet%20exist%20in%20Azure%20Sentinel%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CH1%20id%3D%22toc-hId-1275272497%22%20id%3D%22toc-hId-1275272497%22%3ESaving%20All%20Converted%20Files%3C%2FH1%3E%0A%3CP%3EThe%20final%20section%20in%20the%20notebook%20allows%20you%20to%20bulk%20save%20all%20successfully%20converted%20KQL%20files.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EEnter%20a%20path%20(or%20leave%20the%20default)%20and%20each%20converted%20file%20will%20be%20saved%20into%20a%20sub-directory%20corresponding%20to%20the%20category%20of%20the%20original%20rule%20(multi-part%20paths%20from%20the%20original%20rules%20are%20collapsed%20into%20dot-separated%20folder%20names).%3C%2FP%3E%0A%3CP%3EWarning%3A%20when%20you%20execute%20this%20cell%20any%20existing%20file%20with%20a%20matching%20name%20will%20be%20overwritten.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH1%20id%3D%22toc-hId--1276884464%22%20id%3D%22toc-hId--1276884464%22%3EReferences%3C%2FH1%3E%0A%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fgithub.com%2FNeo23x0%2Fsigma%2Fwiki%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3ESigma%20GitHub%3C%2FA%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-657097%22%20slang%3D%22en-US%22%3E%3CP%3EBrowse%2C%20convert%20and%20test%20Sigma%20rules%20for%20use%20in%20Azure%20Sentinel.%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F116673i8D88D11CFE32E2F3%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20alt%3D%22Teaser.png%22%20title%3D%22Teaser.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-657097%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EDetection%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3Enotebooks%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EQuery%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Microsoft

The notebook accompanying this article is a quick and dirty Sigma rule to Log Analytics converter. It uses functionality from the sigmac tool to do the conversion.

 

What are Sigma Rules?

Sigma rules are an implementation-independent way of specifying detection signatures for attacker activity. Each rule specifies a data source and a set of conditions that need to be met to satisfy a detection condition. A simple example might be a rule that specifies Windows security events as the source and process name equal to “malware.exe”.

 

Sigma provides converters to render these rules into queries that can be deployed in a security operations environment. For example, the previous logic can be converted into the equivalent rule for Splunk, ArcSight, LogAnalytics/Azure Sentinel, and many others.

 

Sigmac is the python command line tool that performs the conversion, allowing you to specify the target format (backend in Sigma terminology) and configuration and mapping files to help with the conversion. This notebook doesn’t do significantly more than the tool but it does allow interactive browsing and execution of the rules in an Azure Sentinel environment.

This Notebook

The notebook can be found here.

 

The notebook uses sigmac functionality to do conversions into KQL (Log Analytics/Azure Sentinel native query language) but has the following additional features:

  • Downloads the full set of contributed sigma rules from the sigma GitHub repo
  • Allows browsing and display of rules organized by the repo folder structure
  • The browser shows both the original sigma syntax and the converted KQL syntax
  • You can optionally add filter clauses for date range and hostname that are added to the output query.
  • You can execute each rule, specifying a time range if needed, and see the results in the notebook.
  • The conversion copies rule metadata into comments in the target KQL.

 

Downloading Sigma Rules

Enter a path where you want to download the sigma rules and code to. This defaults to a folder named “sigma” in the current directory. The folder will be created if it does not exist. If the folder does exist the download and extraction will overwrite any files with same name – so if you have edited files here, save them somewhere else before executing the cell.

If the download is successful you should see a folder structure like the one show above.

 

Repodownload.png

 

Convert the Rules

The next few cells import the required classes from the sigmac source code, define a few helper functions and then tries to covert each rule in turn and store the results in a python dictionary. Any rules that cannot be converted result in the string “Not convertible” in the output field. For limitations in conversion see the Limitations section at the end of this article.

You can see that the convertible rules are currently predominantly Windows rulesConversion.png

 

Browsing Rules

You can select a category (these are derived from the folder structure) and view the converted KQL if conversion was possible. RuleBrowser.png

 

There are checkboxes to add option filters for specific date ranges and hostname. This is shown in the following screen shot.RuleBrowserDateFilter.png

 

Converted rules can be individually selected and copied from the lower pane and saved in text files.

Executing a Rule

Before you can run a converted rule, you need to authenticate to Azure Sentinel and run a few more cells to define functions used for executing the query.

If you have previously run any of the Azure Sentinel notebooks, you may already have a config.json file containing your Workspace and Tenant IDs. If not, paste your workspace ID into the text box supplied.

 

WorkspaceId.png

 

Executing the next cell and follow the prompts to authenticate. Now you are ready to execute the converted queries.

 

Go back to the browser and the cell below, which contains an Execute query button. Select a query in the browser (obviously, non-convertible rules won’t run in Azure Sentinel) and click the Execute Query button. Unless your environment is very badly infected, you will probably see no results for the majority of these queries. If you have opted to include a date filter you can use the date range selector widget immediately below the Execute query cell to choose the date range. If you do not include a filter the query will run over the default time range for Log Analytics, which is 24 hours.

RunQuery.png

 

Although you may see no results in your environment, executing the queries does at least ensure that the conversion has been successful. You might want to simulate some of the activity in the rules to ensure that you are getting results as expected.

 

Some things to look out for include the use of incorrectly escaped backslash (“\”) in file paths. Braces (“{“ and “}”) embedded in strings may also conflict with the replaceable parameter syntax.

Limitations

Only a subset of the Sigma rules are convertible currently. Failure to convert could be for one or more of these reasons:

  • known limitations of the converter
  • mismatch between the syntax expressible in Sigma and KQL
  • data sources referenced in Sigma rules do not yet exist in Azure Sentinel

Saving All Converted Files

The final section in the notebook allows you to bulk save all successfully converted KQL files.

 

 

Enter a path (or leave the default) and each converted file will be saved into a sub-directory corresponding to the category of the original rule (multi-part paths from the original rules are collapsed into dot-separated folder names).

Warning: when you execute this cell any existing file with a matching name will be overwritten.

 

References

Sigma GitHub