Usage reporting for Azure Sentinel

Update:

3rd March 2021: This workbook is now available in the Azure Sentinel portal as a Template or you can still find it in my GitHub (see below) 

16th September 2020: There has been an issue with a name convention I used in this workbook, you will need to download v1.4.4 and above as fix to the issue.   Please upgrade if you see this error: "union named column name: TableName already exists"

 

From the above screenshot you can see you can select your Subscription and Workspace(s).  
The first part of the report, shows a Workspace or many if you select <unset> in the workspace drop-down. 

This report is using Azure resource Graph (ARG) data, so it retrieves data like the retention and licence used.  You can also see (if known) who last set the licence and what licence scheme you're on.  If you have Sentinel assigned to the the workspace, you can probably adjust your retention from 30days to 90days for free, so I make a note of that.  Please do check before you make changes.

 

Download and Install:

Please download the Workbook from my Github  read the import instructions here readme 

•  v1.4.0 Added Checks Tab for Daily, Weekly and Monthly suggested checking routines.  Thanks to Rod Trent, the Workbook aligns to his and the community suggested checks, Daily, Weekly & Monthly  https://secureinfra.blog/2020/03/19/suggested-daily-weekly-and-monthly-tasks-for-azure-sentinel/
• v1.4.4 Quick fix: There has been a recent clash, with the prime query table name I used in this workbook and therefore I have renamed "TableName" to "TableName1". 
  
  
• v1.4.6 In case you are interested I skipped releasing v1.4.5, this release has Moved Price to the Costs Analysis Tab (all pricing is now in the same place).  I added some table data, description and links to the Latency grid. Filter on Queries in Weekly report and Workspace audit filters.  + many more tweaks.
  I have also moved back to using 'Usage' in the workbook name, you can call it anything you like of course.  I prefer 'Workspace Usage Report'.  You can deploy from json as usual , however from this release I added a [Deploy to Azure button] in the GitHub so you can deploy the latest version with ease (thanks @paul collins!).
• v1.4.8 [Cost Analysis] is now a Tab with Sub Menus to aid load times and readability.
  New features: Syslog Cost Analysis, CEF Cost Analysis, in the [OverView] sub menu, there are now reports on capacity / price per Subscription, Resource Group and Tags (Tags, needs needs more work in the next version).
  The Azure Sentinel tab, has reports for Usage vs. Capacity Reservation and recommendations for the reservation settings you are on, for Log Analytics and Azure Sentinel.

Tab 1: Workspace Info 

The report then shows all the Tables you have (and a daily average in the the chart title).  


Next I have included the Table Size and Table entries reports from another workbook.  These are useful to see any pattern changes over the time period.

 

In the latest release (from v1.4.6) I have included an "Advanced Details" section.  Examples are LAQuery audit information - who ran a query, when and how much resource did it consume?   You can see which worked and failed and any poor performers - maybe ones with high CPU time can be improved on?  Note, you do this data source to be enabled, there is a link to the docs displayed.

 

Tip; If you toggle "Help" to YES - there are 3 hidden queries near the top of the page, that display some helpful troubleshooting data (if it exists).

Tab 2: Latency 

The latency report is similar to the info one in Tab1.  Here I show the Average, Minimum and Maximum latency information for each Table.  You can press the column heading to sort the results.

 

I have added some extra table information in a new column, showing information about many of the tables (not all...I'll add more later as well as some links to the docs).  I do need to revisit this section to make sure its 100% up to date on a regular basis.

 

Next we show (please select a Computer from the list), this shows it's Heartbeat data, this view is based on the default Agent Health workbook (see Azure Monitor Workbooks), but in the right-hand graph this shows the latency info, for both the Computer and the Agent (they can be different)

 

Tab 3: Cost Analysis (formally just called Costs)

This tab, as the name suggests give you some insights into Costs, I have moved the PRICE feature to this tab now, as it makes more sense to be here.  March 2021, there are now a few extra Tabs below this one, for new data types, like Table Analysis, Syslog and CEF - but also to break the workbook up to improve load times.
For costs you need to put in a default value: 4.0 is used.  Follow the tip, or open help for more info on this feature.   Azure Sentinel (if you are in the Azure Sentinel costs tab) has its own Price option, just so you can see the specific costs for this service, default is 2.0.

 

The first graph, looks a little like the Overview one in the first tab - but this is showcasing the Table sizing metrics and Table pricing.  The Table price is an estimate on the Price figure you entered above - "4.0" being the default.  This gives you at 'at a glance' view of the Table size, is it billable and also a estimate of cost (based on the price you provided). 
Please use the Azure Pricing Calculator for a estimate.

Next is a capacity trend, projecting forward 90days to give you a hint as to the ingestion trajectory you are on.

Note: the longer time span you select the better the slope will be, 30days+ ideally, however its a slow query and longer time span will slow it down more!  

From v1.4.6, in the grid below the graph (bottom left); I show the estimate price now and at the end of the trend line, plus the data capacities to match.
I also allow you to click the y-axis (red line) and in the grid (bottom right) you will see the data for the date selected - this example is for 5th December.  This can answer the question - how much data will I have on Dec 5th and what is the estimated price?

 

 

The next set of graphs, breakdown Top 10 costs by Table and by Resource, as well as Top 20 cost per EventID.  These can be very useful to spot a busy Computer or EventID that you may have.   
I have added new graphs to show the data change for the last few weeks, so you see how many GB's have been added or removed, and a % of change per week.

 

 

There is a section for Azure Security Center.   This is a good way to spot computers that are sending a lot of data from that solution.  Also is can show how much is sent by all ASC attached computers vs. the allowed allowance.  ASC allows for up to 500MB/day to be sent by a pool of computers.  You can see from my chart, I'm sending 1.7GB but I have the licence and headroom to send much more.  there is more on this topic in the built-in Helpfile.

Tab 4: Azure Sentinel 

The first display looks at the workspace used by Sentinel (and thanks to Paul Collins) shows when Azure Sentinel was added, and therefore how many days its been attached.   This is useful, especially if you are new to Azure Sentinel - as the free trial is 31days, so this can be a quick check to see how may days you have used.

 

This tab also shows some details from Azure Activity logs as a tile view.    The bottom graph just shows specific Tables that Sentinel uses in the Log Analytics workspace.

 

I have also added some view of the newly released preview of Watchlists.

Then Threat Intelligence metrics - which types and count etc...

You can also see a View of Solution and Tables, this shows all Tables and the Solution name they are under (even non Azure Sentinel ones) - you can then select to see extra info.  And the final grid, is a list of the enabled Connectors that Azure Sentinel is using.

 

Tab 5: regular Checks

This gives you some Daily, Weekly or Monthly checks, based on the efforts of Rod Trent and the Azure Sentinel community.  Please read his blog for more details.  Note, I have tried to provide as many of these checks as a visualization, however some I cant (yet) do. 

https://azurecloudai.blog/2020/05/19/azure-sentinel-daily-task-data-connectors/

 

Summary:

 

This workbook has been many months in the making, and thanks to many people for testing and suggesting features.
Special thanks to GaryBushey and a few others for being loyal testers and providing great feedback.