Adopting Sentinel into an existing Azure Cloud environment

[damianborrowell]

I'm reasonably new to Azure as a whole, and I've been really enjoying playing with Sentinel so far, but the time has come to deploy it in a more formal manner. I've got an open question about the Log Analytics workspace that is proving more difficult than anticipated to get a clear understanding of.

We have multiple subscriptions in a single tenant, and we don't use O365 or M365 (although we do use AAD). The vast majority of our Azure usage is either PaaS services (databases, AKS, etc), or running in-house developed applications using App Services. Each of these applications is already set up with Application Insights, and although its across a few LA workspaces, we are actively using Application Insights and Azure Monitor today.

What I'm trying to figure out is: how do I make these logs accessible by Sentinel? I'm not looking for any sort of automated functionality, but I'd very much like to be able to query against and correlate with these logs when doing an investigation -- and being able to write my own automated queries against this would be extremely handy (so they're not all mixed up in the normal Monitor alerting). And while I don't necessarily care to get all of this logging data into Sentinel, I do care about a good chunk of it -- and it may be easier to pull it all in than to try to segregate (it's reasonably low volume today).

Reading between the lines, it seems as though if we have a single Log Analytics workspace that is used by Sentinel and AI, all our in-house application logs would be ingested into (or otherwise queryable by) Sentinel. However, I'm not positive this is true.

If this is not the case: how do I get logs from our App Services into Sentinel? Do I need to look at using CEF or the HTTP Data Collector API? Or is there some other connector in Sentinel that will ingest this logging data from an existing AI instance/LA workspace?

If this is the case: how does the interaction work for the workspace? I recall reading something a few days back that indicated the LA workspace for Sentinel should be dedicated to Sentinel, but didn't give an explanation why (and I can't recall where I read this). If we create a new LA workspace to use with Sentinel, will we still be able to use this workspace to create more traditional infrastructure/development dashboards and alerts?

Thanks! I hope this is reasonably clear, as I'm still a bit uncertain around all the various Azure components, and their interactions and dependencies.

[#2]

@damianborrowell 

Hello, having App Insights in a workspace with Azure monitor logs is a preview feature: https://docs.microsoft.com/en-us/azure/azure-monitor/app/create-workspace-resource#:~:text=%20Worksp...

Typically today we have one or more workspaces for Logs and that is then associated with Azure Sentinel.  

Reading between the lines, it seems as though if we have a single Log Analytics workspace that is used by Sentinel and AI, all our in-house application logs would be ingested into (or otherwise queryable by) Sentinel. However, I'm not positive this is true.

Azure Sentinel doesn't do ingestion (Log Analytics does that); Sentinel provides security analytics / insights onto that data.  If Azure Sentinel sees data from Logs and AI it will analyze it / allow you to query it.

I recall reading something a few days back that indicated the LA workspace for Sentinel should be dedicated to Sentinel

That could be a option, however people sometimes separate Operational logs (and maybe application logs) as they can have low security value and you maybe you don't want Azure Sentinel to charge you to analyze those sources (per GB/day).  e.g.
Sentinel makes use of a lot of the PaaS log sources and logs from ASC (SecurityEvents) etc...  data in the Perf table may not have such high security value (but it will to the Operation team), that's why you often see Perf in another workspace.

If we create a new LA workspace to use with Sentinel, will we still be able to use this workspace to create more traditional infrastructure/development dashboards and alerts?

Yes, the underlying query language and capability is still there.  So you can build an Azure Dashboard on the data (Log Analytics or Azure Sentinel), or use KQL to query, or build a Workbook to visualise the data - Workbooks are delivered by Azure Monitor but you see them in many portal blades such as Azure Sentinel.

Modules 2-4 will help you from the Azure Sentinel training: https://techcommunity.microsoft.com/t5/azure-sentinel/become-an-azure-sentinel-ninja-the-complete-le...

[#3]

Thanks, @CliveWatson! This is super helpful.

I think part of what's tripping me up is Monitor vs Log Analytics vs Application Insights. Based on the Azure Monitor Overview, I was under the impression that Monitor was the "umbrella" that housed all the underlying services. Given what you're saying above, and now that I'm a bit more familiar with the portal, it seems that Monitor is more of a "family" of related services rather than a distinct resource that we would be provisioning or managing.

That being said, it seems that we should be able to create a single Log Analytics workspace, configure Sentinel to use this workspace, and migrate all our Application Insights resources into this workspace as well. This will likely require some RBAC to keep things sane and partitioned appropriately.

---

That could be a option, however people sometimes separate Operational logs (and maybe application logs) as they can have low security value and you maybe you don't want Azure Sentinel to charge you to analyze those sources (per GB/day).  e.g.
Sentinel makes use of a lot of the PaaS log sources and logs from ASC (SecurityEvents) etc...  data in the Perf table may not have such high security value (but it will to the Operation team), that's why you often see Perf in another workspace.

There is a distinct drawback in having all your operational logs analyzed by a SIEM, both in up-front analysis costs and in long-term query complexity and computation time. There is a reasonably small subset of operational logs that we would want to ingest into Sentinel, but I'm unclear as to what would be entailed in splitting up the logs appropriately so that we can have only the security-sensitive logs analyzed by Sentinel while still having the full set of operational logs available to our developers and infrastructure people.

There's sufficient "magic" in Azure that it is surprisingly challenging coming in with no Azure knowledge or experience, and trying to figure out how to implement a logging pipeline. Most of the documentation boils down to "send it to Log Analytics and you're done". I'll start looking at your suggested training modules and see if that helps clear it up; they're definitely topical, and I'm looking forward to Module 5 having more content as well!

Given that there doesn't appear to be any technical concerns (at our size) with having a single Log Analytics workspace that is used by Sentinel and our in-house applications, for the sake of simplicity, I suspect that's where we'll wind up (barring corrections from those training modules). And so long as we're keeping our dashboards, alerts, and Sentinel configuration defined in Powershell/Terraform, it should be reasonably straightforward to change things up in the future.