Ingestion Cost Alert Playbook

[Inwafula]

Managing cost for cloud services is an essential part of ensuring that you get maximum value for your investment in solutions running on this computing platform. Azure Sentinel is no different. To help you exercise greater control over your budget for Azure Sentinel  this playbook will send you an alert should you exceed a budget that you define for your Azure Sentinel Workspace within a given time-frame. The solution is also available for deployment from Github on this link .

Overview of the Logic App

The Logic App is activated by a Recurrence trigger whose frequency of execution can be adjusted to your requirements

Since the Logic App is being deployed from an ARM template you will need to make connections to Azure Monitor, Office 365 and Teams before the Logic App can work in your environment. You can expect to see windows like the one below. Click “Add new” to create a connection to Azure Monitor and subsequently perform a similar operation for the “Send and e-mail” and “Send a Teams message” steps.

The below variables are all declared as integers and can be used to specify values that suit your requirements.

The query below will then run against your workspace and computes the aggregated cost of billable tables in your Azure Monitor Workspace.

Below is the query being executed in the step above in text format which you can use for validation directly in the Log Analytics query window. Ensure to replace the variables below with actual numbers if running the query within the Log Analytics query window.

let price_per_GB = price_per_GB;

let how_many_days = how_many_days;

let total_funding = total_funding;

let threshold_per_day = toreal(total_funding) / toreal(how_many_days);

Usage

| where TimeGenerated > startofday(ago(1d))

| where IsBillable == true

| summarize AggregatedValue= sum(Quantity) * price_per_GB / 1000 by bin(TimeGenerated, 1d)

| where AggregatedValue > threshold_per_day

In this step, the aggregated value obtained from the previous step is compared against the budget value you set and should it exceed the amount then the logic branches to the left and sends out an e-mail or a Microsoft Teams message. If you are still within budget, then the logic branches to the right and no message is sent.

In the final step below sends out an e-mail to the specified recipient list and a message is posted in a Microsoft Teams channel of your choice

Additional information on cost alerts can be found in this document

[AlphaBetaGamma]

Am I blind or some of the logic app actions which are shown in above snips are not available when deployed the playbook from Github? (for eg, foreach loop which is showing here in the above post, but when I tried to deploy the playbook from the above GitHub link, I cannot see some logic app actions)

Also, the kql query is different from the above snips with the query in the deployed logic app. Can someone please help me if I'm missing any understanding here?

[Luizao_f]

@Inwafula  This feature is very cool. I have a problem with that and maybe you can help me.

We have some customers in the process of aligning about closing a partnership to obtain Azure Sentinel, however, now and then we are asked about the average expense that they may have through the acquisition of the solution so that the customer “projects” what average consumption they will have.

I did some research on the web and found some content about average consumption of EPS by solutions and average log size of them, however, I did not feel "firm" with such information.

I know it is subjective and depends on each solution / technology, among other variables, however, I would like to know if someone has / indicates some type of material / spreadsheet or something like that I can help “price” a new customer for Azure acquisition Sentinel.

Ex: Knowing on average that the customer has X equipment and technologies, as well as the EPS and average size of the logs of these artifacts, they arrive at an approximate value of X, we can “stipulate” that they will be ingested in Sentinel “X GB”, which in turn , can result in “X” values.

[Gary Bushey]

It appears the "Deploy to Azure" button in GitHub is deploying the wrong project. It creates the playbook for detecting if a service is no longer sending data.

[Peter_Piper]

@Gary Bushey   and @AlphaBetaGamma I usually deploy to Azure using the raw ARM and "Deploy a Custom Template" and selecting "Build your own template in the editor" within the Azure Portal. Hasn't failed me yet. 

[Inwafula]

@Gary Bushey thanks for catching this, my apologies. Let me fix that. Meanwhile you can use the method proposed by Peter

[Inwafula]

Hi @AlphaBetaGamma yes the issue is because the Deploy button points to a different Logic App. This will be resolved soonest and I'll post a confirmation here but in the meantime please use the route suggested by @Peter_Piper above

[Inwafula]

@AlphaBetaGamma  and @Gary Bushey  the Deploy button has now been fixed. You can retry

[Inwafula]

 Hi @Luizao_f thanks for the feedback. Your follow up question requires an in-depth response. Permit me to respond after the holidays -:)

[Inwafula]

Happy New Year @Luizao_f  

First off, Microsoft provides estimates using public facing tools with this disclaimer: Resulting data usage and prices are estimates only.  Actual data usage and prices may vary depending upon many factors, including customer environment, date of purchase, currency of payment, and type of agreement with Microsoft, Region of data ingestion & retention & Pricing model.  Accurately determining before  actual usage is quite a daunting task because our cost model is primarily based on GBs ingested from sources that are both Microsoft & non-Microsoft so the more accurately you determine that the more closer you get to your consumption estimate. The best way to get an idea of future costs is to run a limited pilot with target data sources. However, the public-facing resources we currently have available to help you estimate usage are per below:

Cost drivers:

Azure Sentinel Ingestion: Azure Sentinel Pricing | Microsoft Azure

Log Analytics Ingestion: https://azure.microsoft.com/en-us/pricing/details/log-analytics/

Log Analytics retention: https://azure.microsoft.com/en-us/pricing/details/log-analytics/

Automation-Logic Apps: https://azure.microsoft.com/en-us/pricing/details/logic-apps/

Bring Your Own ML: https://azure.microsoft.com/en-us/pricing/details/machine-learning-studio/ & https://azure.microsoft.com/en-us/pricing/details/databricks/

UEBA-5-10% over and above cost of selected data sources

(Not a Microsoft resource) - Log point provide a public-facing calculator here but you would use it based on their own disclaimer terms SIEM Storage Calculator | LogPoint

For historical Azure Sentinel costs, this workbook would be useful: Usage reporting for Azure Sentinel - Microsoft Tech Community

If you are a Microsoft partner I would suggest you get in touch with your Microsoft rep, they would have access to some internal resources that can help you better size future consumption. Hope this helps