Forum Discussion

JMSHW0420's avatar
JMSHW0420
Iron Contributor
Dec 16, 2024

ARM template for deploying a workbook template to Microsoft Sentinel

Hello,

I am attempting to deploy an ARM Template (execution using PowerShell) for any Analytic Rule to a Microsoft Sentinel instance.

I have been following this link: https://learn.microsoft.com/en-us/azure/azure-monitor/visualize/workbooks-automate#next-steps.

I am struggling with ensuring the Workbook is deployed to the Microsoft Sentinel workbook gallery and NOT the Azure Monitor one.

The link includes a sample ARM template where you can add <templateData> (JSON code), which represents the workbook you wish to deploy.

I get it working to deploy to the Azure Monitor workbook gallery but not for it to be present in the Microsoft Sentinel one.

Jason 

  • Hi Jason,

    You are quite right.  It's moving quite a long way from the original question but altering the Id string is all that is required to deploy an object to a different subscription.

    In Azure, objects can clip together like Lego blocks based on the "provider" element.  The Id string refences where something is deployed to and it tells Azure what type of object is being deployed (although most of the time that is also shown as the type property).  You dont actually need the name property either - that's always the last element if the Id.  

    I'm familiar with the argument for deploying everything as code.  These content templates are vesioned and are being updated constantly - it will be one headache to keep the versioning in your code templates up-to-date and if they go down the path of not allowing SOC engineers to create detection rules or update content through the portal it will be a tragedy.  If your client also wants data connectors also deployed through code that becomes an almighty world of pain and also a security risk as your build pipeline has to be super-privileged with god rights in all systems.

    From an Engineer's perspective, seeing Azure as REST objects is really helpful in understanding how it really works and there are still rare times when REST can get around issues with ARM.  I use REST all the time as I'm troubleshooting in my sandpit but I still use Bicep (or ARM) when delivering solutions for customers as it's a standard solution that is officially supported by Microsoft.  Once you have a JSON object you can find the type of object with Microsoft's Bicep documentation and it's almost a straight copy and paste of values against properties.

  • JMSHW0420's avatar
    JMSHW0420
    Iron Contributor

    Hi Laurie_Rhodes,

    I have marked your last reply as the Solution.

    I know REST API, but you have made me look at it differently from what I needed here. 

    I will keep you informed (briefly) on how I proceed.

    Laurie, I like your blog/website. How do I log in to it, mate?

    I want to contribute with constructive comments when necessary.

    Jason

    • Laurie_Rhodes's avatar
      Laurie_Rhodes
      Brass Contributor

      Thanks for the feedback Jason :)

      I don't really have the time to manage a website with content moderation so I keep it locked and just focussed on my technical notes that I think may be of benefit to others.  I'm glad you found it of use!  

      • JMSHW0420's avatar
        JMSHW0420
        Iron Contributor

        Hi Laurie_Rhodes,

        When I try to execute your Push-Azureobject, I get the following error:

        .

        PS C:\WINDOWS\system32> $file = "C:\Users\Jason\OneDrive\Desktop\_ARM-PS-TEMPLATES\WorkspaceUsage.json"

        PS C:\WINDOWS\system32> Get-jsonfile -Path $file | Push-Azureobject -authHeader $header -apiversions $AzAPIVersions
        Invoke-RestMethod : {"error":{"code":"BadRequest","message":"Unable to translate bytes [A3] at index 943 from specified code page to Unicode."}}
        At line:58 char:5
        +     Invoke-RestMethod -Uri $uri -Method PUT -Headers $authHeader -Bod ...
        +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
            + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

        .

        .

        .

        I understand this likely relates to “unknown, unrecognized, or unrepresentable characters” in the JSON file and needs to process an Encoding dependency; how have you dealt with this in your environment or the PowerShell scripting for the AZRest module? 

  • Laurie_Rhodes's avatar
    Laurie_Rhodes
    Brass Contributor

    Absolutely it doesn't work!

    I've been looking a bit more deeply into what these workbook "templates" are when displayed in Sentinel.  They are "Content Templates"
    https://learn.microsoft.com/en-us/rest/api/securityinsights/content-templates/list?view=rest-securityinsights-2024-09-01&tabs=HTTP

    Creating your own set of content templates isn't well documented.  It really deserves quite a long blog post - but I have been able to get a basic example working with REST.

    I'm actually happy to get something working at all - so I'm sure refinement is needed.  But this is the path you should be looking at... 

    {
      "id": "/subscriptions/xxxxxxxx-xxxxxxxx-xxxxxxx-xxxxx/resourceGroups/xxxxxxxxxxxxxxxxxx/providers/Microsoft.OperationalInsights/workspaces/xxxxxxxxxxxxxx/providers/Microsoft.SecurityInsights/contenttemplates/LaurieTest",
      "name": "LaurieTest",
      "type": "Microsoft.SecurityInsights/contenttemplates",
      "properties": {
        "packageKind": "Standalone",
        "packageId": "sentinel-LaurieTest",
        "packageVersion": "1.0.0",
        "contentSchemaVersion": "3.0.0",
        "contentProductId": "LaurieTest",
        "contentId": "sentinel-LaurieTest",
        "displayName": "Laurie Test",
        "contentKind": "Workbook",
        "version": "1.0.0",
        "source": {
          "kind": "LocalWorkspace",
          "name": "Standalone"
        },
        "author": {
          "name": "Laurie Rhodes"
        },
        "support": {
          "tier": "Community",
          "name": "Laurie Rhodes",
          "email": "email address removed for privacy reasons",
          "link": "https://laurierhodes.info"
        },
        "MainTemplate": {
          "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
          "contentVersion": "1.0.0",
          "parameters": {
            "workspace": {
              "type": "string",
              "metadata": {
                "description": "The name of the workspace where the workbook will be deployed"
              }
            }
          },
          "resources": [
            {
              "type": "Microsoft.Insights/workbooks",
              "apiVersion": "2021-08-01",
              "name": "[guid('sentinel-LaurieTest')]",
              "location": "[resourceGroup().location]",
              "kind": "shared",
              "properties": {
                "displayName": "Laurie Test Workbook",
                "serializedData": "{\"version\":\"Notebook/1.0\",\"items\":[{\"type\":1,\"content\":{\"json\":\"## Welcome to the Test Workbook\\n---\\nThis is a test workbook to verify template deployment.\"},\"name\":\"text - 1\"},{\"type\":1,\"content\":{\"json\":\"### Sample Query Section\\nBelow is a basic query to verify data access:\"},\"name\":\"text - 2\"},{\"type\":3,\"content\":{\"version\":\"KqlItem/1.0\",\"query\":\"SecurityAlert\\n| summarize count() by AlertName\\n| top 10 by count_\",\"size\":0,\"timeContext\":{\"durationMs\":2592000000},\"queryType\":0,\"resourceType\":\"microsoft.operationalinsights/workspaces\"},\"name\":\"query - 1\"}],\"styleSettings\":{},\"defaultResourceIds\":[],\"fallbackResourceIds\":[],\"$schema\":\"https://github.com/Microsoft/Application-Insights-Workbooks/blob/master/schema/workbook.json\"}",
                "version": "1.0",
                "sourceId": "[resourceId('Microsoft.OperationalInsights/workspaces', parameters('workspace'))]",
                "category": "sentinel"
              }
            },
            {
              "type": "Microsoft.OperationalInsights/workspaces/providers/metadata",
              "apiVersion": "2022-01-01-preview",
              "name": "[concat(parameters('workspace'),'/Microsoft.SecurityInsights/Workbook-sentinel-LaurieTest')]",
              "dependsOn": [
                "[resourceId('Microsoft.Insights/workbooks', guid('sentinel-LaurieTest'))]"
              ],
              "properties": {
                "description": "Test of a community content template",
                "parentId": "[resourceId('Microsoft.Insights/workbooks', guid('sentinel-LaurieTest'))]",
                "contentId": "sentinel-LaurieTest",
                "kind": "Workbook",
                "version": "1.0.0",
                "source": {
                  "kind": "Community",
                  "name": "Laurie Rhodes",
                  "sourceId": "sentinel-LaurieTest"
                },
                "author": {
                  "name": "Laurie Rhodes",
                  "email": "email address removed for privacy reasons"
                },
                "support": {
                  "tier": "Community",
                  "name": "Laurie Rhodes",
                  "email": "email address removed for privacy reasons",
                  "link": "https://laurierhodes.info"
                }
              }
            }
          ]
        }
      }
    }

     

    • JMSHW0420's avatar
      JMSHW0420
      Iron Contributor

      Hi Laurie_Rhodes ,

      I appreciate all your help.

      Can I ask just two more things, please?

      • For the above JSON, can you provide the FULL script you created?
      • Second, this is an example (link) of an existing Workbook Template (templateData) for Sentinel Costs I am trying to deploy. Based on the Test Workbook you deployed, what is the FULL JSON structure I require?


      Thanks again for your input.

      Jason

      • Laurie_Rhodes's avatar
        Laurie_Rhodes
        Brass Contributor

        Hi Jason,

        What you are doing is really quite unusual.  The workbook you want to deploy is already in Content Hub so anyone with the right permissions can search for it and enable it there.  

        What you can do if deploying this content through code is a necessity is manually enabled the workbook through content hub and then after it succeeds look at the deployments on your resource group and grab the IDs for the Content Package and Content Template. 

         

        Ultimately everything in cloud uses REST API - it doesn't matter if it's Azure, AWS or GCP.  ARM just provides a template for REST and Bicep is templating for ARM.  With Bic ep and ARM looking at the JSON representation of an object lets you see what the property names and values are that you need in a template.

        What I posted before is the full representation of a JSON object for creating content that includes a workbook.  If you want to work with REST in getting and pushing objects into Azure, I have published a series of scripts for doing that here.  

        Where possible I'd prefer to use Bicep now for creating templates - but you can create ARM templates from Bicep.
        Microsoft.SecurityInsights/contentPackages - Bicep, ARM template & Terraform AzAPI reference | Microsoft Learn

        Content Hub is intended to be used through the portal... I would just enable the workbook you want that way.  Thats how we have to update the content when new versions come out anyway and if we must use the portal for content management, I don't see the criticality of managing that content through code.

        However, if you really must use code, I've included the JSON from my deployments below (with alteration of the Id that points to my subscription).  It's too much data for the chat thread to accept so I've uploaded them to GitHub here.

  • Laurie_Rhodes's avatar
    Laurie_Rhodes
    Brass Contributor

    Hey Jason,

    You'll need to change your default category to "sentinel".  With the example you linked, "category": "[parameters('workbookType')]" specifies the the category is set from the parameter workbookType.

    Also make sure that the workbook is deployed to the same Resource Group as Sentinel.  

    • JMSHW0420's avatar
      JMSHW0420
      Iron Contributor

      Hello Laurie,

      Thank you very much for your reply.

      Can I check that you mean an actual workbook instance?

      Where I am struggling is with the direct Template and deploying that...

      So, I am using this...

      {
          "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
          "contentVersion": "1.0.0.0",
          "parameters": {
              "resourceName": {
                  "type": "string",
                  "defaultValue": "my-workbook-template",
                  "metadata": {
                      "description": "The unique name for this workbook template instance"
                  }
              }
          },
          "resources": [
              {
                  "name": "[parameters('resourceName')]",
                  "type": "microsoft.insights/workbooktemplates",
                  "location": "[resourceGroup().location]",
                  "apiVersion": "2019-10-17-preview",
                  "dependsOn": [],
                  "properties": {
                      "galleries": [
                          {
                              "name": "A Workbook Template",
                              "category": "Deployed Templates",
                              "order": 100,
                              "type": "workbook",
                              "resourceType": "Azure Monitor"
                          }
                      ],
                      "templateData": <PASTE-COPIED-WORKBOOK_TEMPLATE_HERE>
                  }
              }
          ]
      }

      ...and then adding the JSON content for a Workbook Template at "templateData"

      So, is it possible to add the Template to the Microsoft Sentinel Gallery is what I am asking?

      Thank you for being so helpful. Much appreciated.

      Jason 

      • Laurie_Rhodes's avatar
        Laurie_Rhodes
        Brass Contributor

        Hey Jason,

        Thanks for the clarification.  You are quite right, what I was referring to was deploying individual workbooks by template - as long as the category is "sentinel", the example on the page you referenced works in deploying custom sourced workbooks.

        But this isnt what you were getting at.  Your question was (I believe) about populating the templates section of the Sentinel workbooks blade.  I don't have the right answer for you unfortunately.

        From the documentation, something like this I believed would have worked (it doesn't):

        {
            "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
            "contentVersion": "1.0.0.0",
            "parameters": {
                "workbookDisplayName": {
                    "type": "string",
                    "defaultValue": "Sentinel Workbook Template",
                    "metadata": {
                        "description": "The friendly name for the workbook that is used in the Gallery or Saved List. Needs to be unique in the scope of the resource group and source"
                    }
                },
                "workbookDescription": {
                    "type": "string",
                    "defaultValue": "A sample Sentinel workbook template",
                    "metadata": {
                        "description": "Description for the workbook"
                    }
                },
                "workbookSourceId": {
                    "type": "string",
                    "metadata": {
                        "description": "The id of resource instance to which the workbook will be associated"
                    }
                },
                "workbookId": {
                    "type": "string",
                    "defaultValue": "438ebb27-5c19-4dbd-9a88-9424f8a65361",
                    "metadata": {
                        "description": "The unique guid for this workbook template"
                    }
                }
            },
            "resources": [
                {
                    "name": "[parameters('workbookId')]",
                    "type": "Microsoft.Insights/workbookTemplates",
                    "location": "[resourceGroup().location]",
                    "apiVersion": "2020-11-20",
                    "properties": {
                        "priority": 1,
                        "galleries": [
                            {
                              "name": "Microsoft Sentinel Example",
                              "category": "Microsoft Sentinel",
                              "order": 100,
                              "type": "workbook",
                              "resourceType": "microsoft.operationalinsights/workspaces"
                            }
                        ],
                        "templateData": {
                            "strings": {
                                "title": "[parameters('workbookDisplayName')]",
                                "description": "[parameters('workbookDescription')]"
                            },
                            "workbook": {
                                "version": "Notebook/1.0",
                                "items": [
                                    {
                                        "type": 1,
                                        "content": {
                                           "json": "{\"name\": \"Welcome Section\", \"type\": \"markdown\", \"content\": \"## Welcome to your Sentinel workbook!\"}"
                                        }
                                    }
                                ]
                            }
                        }
                    },
                    "tags": {
                        "hidden-title": "[parameters('workbookDisplayName')]",
                        "hidden-link": "[parameters('workbookSourceId')]"
                    }
                }
            ],
            "outputs": {
                "workbookTemplateId": {
                    "type": "string",
                    "value": "[resourceId('Microsoft.Insights/workbookTemplates', parameters('workbookId'))]"
                }
            }
        }

        I'm surprised that the specifics about the properties that are needed to specify the Microsoft Sentinel gallery don't seem to be documented anywhere.  These seem to be case sensitive too.

                        "galleries": [
                            {
                              "name": "Microsoft Sentinel Example",
                              "category": "Microsoft Sentinel",
                              "order": 100,
                              "type": "workbook",
                              "resourceType": "microsoft.operationalinsights/workspaces"
                            }
                        ],

        That page shows eleven combinations for different types of galleries but nothing for Sentinel.
         https://learn.microsoft.com/en-us/azure/azure-monitor/visualize/workbooks-automate

        I had thought that using hidden titles and links might have solved this - but it was of no help.  Hopefully someone from Microsoft can provide some deeper insights!  I'm really curious now too! :)

Resources