Microsoft Secure Tech Accelerator
Apr 03 2024, 07:00 AM - 11:00 AM (PDT)
Microsoft Tech Community
Using Azure Lighthouse and Azure Sentinel to Investigate Attacks Across Multiple Tenants
Published Dec 04 2019 03:26 AM 105K Views
Microsoft

Since this blog was first written much more comprehensive documentation on Azure Lighthouse usage with Azure Sentinel has been produced. Whilst this blog provides an overview it is strongly recommended that users refer to this documentation: Build a scalable security practice with Azure Lighthouse and Azure Sentinel | Azure Blog and Updates...

 

Azure Lighthouse provides capability for cross-tenancy management of Azure services for Managed Service Providers (MSPs) and organizations with multiple Azure tenants, all from a single Azure portal. Azure Lighthouse is integrated with Azure Sentinel allowing organizations to easily manage Azure Sentinel workspaces from across multiple tenants. In this blog we will show you how to configure Azure Lighthouse, and how to use its capabilities to investigate an attack as it targets several customers at once.

 

Full documentation on Azure Lighthouse can be found here. 

 

Enabling Azure Lighthouse for Sentinel requires configuration on both the managing tenant and customer tenant (for the purpose of this blog we will refer to customer tenant to describe the tenant to be managed, whether this is an external customer or separate internal tenant), we will address these elements separately.

Lighthouse Architecture DiagramLighthouse Architecture Diagram

 

Managing Tenant Configuration

Several items are needed to configure Lighthouse for Azure Sentinel:

  • The tenant ID of the managing tenant.
  • The ID of the Azure AD group in the managing tenant that you wish to have access to Azure Sentinel in the customer tenant. It is advised that a specific Azure AD group is created for this purpose.
    (Get-AzADGroup -DisplayName "<Group Name>").id​
  • The ID of the user role that you wish the AD group to have within the customers tenant. For this you can use built in Role Definitions such as Security Reader, or create a custom definition.
    (Get-AzRoleDefinition -Name “<Role Name>”).id

 

The core deployment of the Azure Lighthouse configuration will be made using this Azure Resource Manager (ARM) template - https://github.com/Azure/Azure-Lighthouse-samples/blob/master/templates/delegated-resource-managemen...

 

In order to ensure this ARM template deploys the required configuration it is accompanied by a parameters file which you must populate with the ID details listed above. A template for this parameters file can be found here:  https://github.com/Azure/Azure-Lighthouse-samples/blob/master/templates/delegated-resource-managemen...

 

To complete the parameters file, update the values as follows:

  • mspOfferName – provide a display name for your managed service offering (this can be seen within the customer tenant).
  • mspOfferDescription – provide a description for your managed service offering (this can be seen within the customer tenant).
  • managedByTenantId – the tenant ID of the managing tenant.
  • authorizations
    • principalId – the ID string of the Azure AD Group in the managing tenant you want to grant access.
    • roleDefinitionId – the ID of the user role that you wish the AD group to have within the customers tenant.
    • principalIdDisplayName – provide a name for the group being granted access (this can be seen within the customer tenant and does not have to be the same as the AD group name).

Once complete your parameters file should look something like this:

 

Note: If you want to grant multiple groups access at differing role levels, you can include multiple entries under the authorizations section– see https://docs.microsoft.com/en-us/azure/lighthouse/how-to/onboard-customer for an example of this.

 

Once you have completed the parameters file you will need to provide the ARM template and parameters file to a user with the Owner role within the customer tenant.

 

Customer Tenant Configuration

After receiving the ARM template and parameters file from the managing tenant, the customer tenant  must deploy the configuration within the subscription that contains the Azure Sentinel workspace to be managed.

The first step in provisioning Lighthouse access within the customer tenant is to register the ManagedServices Resource in Azure. You can check if this is already registered via PowerShell with:

Get-AzResourceProvider -ProviderNameSpace Microsoft.ManagedServices

(If you are not using Azure Cloud Shell you will first need to run the Connect-AzAccount cmdlet to authenticate to Azure)

 

This should return an output similar to this:

 

If the RegistrationState does not read ‘Registered’  for all Workspaces in the output run the following command in PowerShell to register the service:

Register-AzResourceProvider -ProviderNamespace Microsoft.ManagedServices

Once the ManagedServices Resource is registered you can deploy the Lighthouse ARM template with the correct parameters provided by the managing tenant. First select the subscription that the Azure Sentinel workspace to be managed is part of using:

Select-AzSubscription -SubscriptionId ‘<Your Subscription ID>’

Then deploy the ARM template with:

New-AzDeployment -Name <deploymentName> `

                 -Location <AzureRegion>`

                 -TemplateFile <Path to ARM template> `

                 -TemplateParameterFile <Path to parameters file>`

                 -Verbose

(You can deploy in any region, if you are unsure you can use the region your Azure Sentinel workspace is deployed in.)

 

If deployed correctly the output should look like this:

 

To confirm if this deployment has been successful navigate to the Azure Portal and search for Service Providers. In here select Service provider offers where you should see the details of the configuration you just deployed.

 

Note:

There are also options to publish an MSP offering as a service via the Azure Marketplace and Cloud Partner Portal enabling customers to purchase and onboard the resources via the Azure portal rather than having to deploy an ARM template as described above. This process requires the MSP to publish several additional details about their service. For details regarding this process please see this documentation - https://docs.microsoft.com/en-us/azure/lighthouse/how-to/publish-managed-services-offers

 

Managing Tenants

Once the customer tenant has deployed the Lighthouse configuration the managing tenant can also validate the deployment by navigating to the Service Provider Portal and opening the Customers tab. Here you should see a list of customer tenants who have successfully deployed your Lighthouse configuration.

To view customer resources within the managing tenant’s Azure Portal open the portal and select the Directory + subscription filter at the top of the page, here you should see the customer tenant directories and subscriptions on-boarded for you to select:

 

 

Once selected the customer tenants Workspaces will appear in your Azure Sentinel workspaces list alongside your own workspaces.

 

Currently most features within Azure Sentinel are directly tied to a single workspace, however it is possible to query multiple tenant’s workspaces with one unified query in order to investigate and hunt for threats that might be impacting several tenants at once.  To do this you can use the union operator to merge a table from one workspace with another. For example, if you wanted to see a list of all security alerts from the last week from two workspaces you would use the following query:

union SecurityAlert, workspace('<Second Workspace Name>').SecurityAlert
| where TimeGenerated > ago(7d)

This can also be extended to any number of workspaces simply by adding more workspaces to the union query:

union SecurityAlert, workspace('<Second Workspace Name>').SecurityAlert, workspace(‘<Third Workspace Name>’).SecurityAlert
| where TimeGenerated > ago(7d)

The result set will appear as a single table, but you can use the TenantId field to identity which customer each item comes from.

 

Tracking an attack across multiple customer tenants

Exposed management endpoints on the internet are a regular attack target. Often attackers will scan the internet for exposed endpoints and then attempt to login using common credentials in the hopes of gaining access to at least some hosts. As a multi-customer monitoring team, it is common to see such attacks impacting several tenants simultaneously as commodity attackers indiscriminately attack any exposed interface they can find.

In this example we start with an alert in one of our customer’s workspaces for an attempted brute force attempt:

 

Upon inspecting the logs associated with this alert we can see all the attempts are coming from the same IP address:

 

To quickly assess if there is an impact from this activity we want to see if that IP address has successfully logged into any hosts in the environment. However, as we suspect this attack might be wider than just one of our customers, we can also simultaneously query our other workspaces for the same IP address.

union Syslog, workspace('<Customer2 Workspace Name>').Syslog
| where TimeGenerated > ago(7d)
| where ProcessName == "sshd"
| extend LogonIP = extract("([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3})", 1, SyslogMessage)
| where SyslogMessage contains 'Accepted'
| where LogonIP == "174.116.122.126"
| extend Customer = iif(TenantId == "<Customer2 Tenant ID>", "Customer2", "Customer1")
| project TimeGenerated, Customer, Computer, LogonIP, SyslogMessage 

The results of this query shows us that whilst there was no impact for the customer where we saw the Brute Force alert there were 2 successful logons for that IP address in another customer environment, highlighting these as events to investigate further.

 

Summary

A key principle to effective and scalable security operations is streamlining the security analysts’ job so that they can focus on threat detection and response. Azure Lighthouse’s integration with Azure Sentinel simplifies management of workspaces from multiple Azure tenants, whether they be external customer tenants or other tenants used internally within an organization, so that analysts can easily and effectively monitor multiple environments at once. In addition, the union operator within KQL allows analysts to query multiple environments at once, and track threats across them without having to repeat analysis.

 

If you want to know more about using Azure Sentinel in a multi-tenant or  Managed Service Provider environment @Ofer_Shezaf recently ran a really useful webinar on this subject. A recording of this can be found on YouTube, and the slides are available here.

 

Special thanks must also go to @Yaniv Shasha from the Azure Sentinel CxE team for his contribution to this blog and his work with Azure Lighthouse.

29 Comments
Copper Contributor

Lighthouse has so much promise for MSPs, but I get push-back from clients who need to maintain PCI compliance. The fear is that delegating access control to an external partner by grating rights to their groups means that they are relying on the partner to ensure that rouge actors don't gain access. It also would presumably make auditing more difficult.

Has Microsoft addressed this concern anywhere in a whitepaper or article? I can't seem to find anything.

Microsoft

@JoshuaK - PCI-DSS has robust provisions for handling service providers with access to in scope data, if a customer has PCI relevant data in Sentinel then Microsoft will already by a PCI service provider for them for which Microsoft has accreditation - https://docs.microsoft.com/en-us/microsoft-365/compliance/offering-pci-dss

Any other MSP with access to that data would also have to go through an accreditation and audit process but this is a well established path for QSAs.

 

As for malicious actors gaining access via a partner it is a threat vector to consider as attackers have targetted service providers to gain access to customers, however it with robust auditing of MSP security controls this risk can be managed. In addition the Lighthouse configuration allows you to assign granular controls over what an MSP has access to meaning it could be limited to read-only access to Sentinel data, limiting the scope of impact in a scenario where an MSP might be breached.

Bronze Contributor

Is there any way to easily deploy playbooks to different tenants at once? That way you can keep a baseline for all customers

Microsoft

@Thijs Lecomte : you can use ARM. See here and here.

Copper Contributor

how will the billing work when you have a Sentinel configured in CSP tenant and configured to ingest the data coming from customer tenants? I think as a CSP provider we can offer Security as a Service to the ISV customers but just wondering about how the billing would work.

Microsoft

@JoshuaK : Lighthouse is arguably better than the method it replaces. Keep in mind that service provisioning implies providing access to a contractor; the question is only how.

Until now, to provide access to a subcontractor, you had to add each employee of the contractor as a guest to your tenant. This fundamentally breaks any identity governance scheme and makes contractual protection harder. With lighthouse, the contractor is in charge of managing a group of which the end customer set permissions for. This more closely aligns with the way the contracts work. It also means that onboarding and offboarding contractor employees servicing the customer would follow the process better.

Microsoft

@badkid :

 

A clarification - there is no CSP tenant. There is a CSP subscription within the customer tenant.

 

As to your question: you may recall that a subscription is the billing unit in Azure. The CSP model implies that the MSSP is paying for the CSP subscription, allowing the MSSP to "resell", by billing the customer and adding any premium based on added services, rather than having Microsoft bill the customer directly.

 

~ Ofer

Copper Contributor

@Ofer_Shezaf When I say CSP tenant, I meant: we being the Microsoft Cloud Solution Provider, we have our own subscription where sentinel is configured. If we configure the sentinel with the customer data source (say customer Log Analytics workspace - hosted on customers subscription), we as a CSP provider has to bare the cost of sentinel data consumption rather than customer pays for the sentinel. Is that the correct understanding?

Microsoft

@badkid : Yes, this is correct. This is the goal of the CSP model: enabling partners to bear the cost and charge the customer themselves. Note that Lighthouse does not require a CSP subscription and works with a regular subscription at the customer tenant.

Copper Contributor

we are using lighthouse for an enterprise that has multiple tenants. in this case we would like to query all log workspace. eg. union syslog *. is this posisble. we have 30 + tenants and this would be helpful to create a central workspace.

Microsoft

@Brendon Lobo : It is possible. That said, while 30+ tenants are sometimes needed, in general, it seems like a tenant reduction is due. 

Copper Contributor

what is the syntax to run such query? yes light house is helping with step 1 of discovery and management. once we have tenant related settings migrated we will move subscriptions. stating that light house and multi tenant is giving us an opportunity to monitor but give freedom.

Microsoft

@Brendon Lobo : the syntax uses the workspace keyword as demonstrated in this blog post. You can union more than two tables. To make this easier for individual analysts or content creators, you can encapsulate the union part of the query in a function

Copper Contributor

We are considering using Lighthouse to manage multiple customers Sentinels and we are trying to evaluate what would be the additional cost in our subscription ? what level of data comes from the managed Sentinels ? I realize there is no cost to the service itself but what about the data coming from the managed instances ?

Microsoft

@JF_68 the cost is incurred at the subscription where the Sentinel Workspace resides. Do if you have Tenant A with a Sentinel Workspace, and then you use Lighthouse to give Tenant B access to that workspace the costs associated with the Sentinel Workspace and the data remain with Tenant A.
When you provide access using Lighthouse you are not actually moving any of the data or resources you are simply giving a section of another tenant access to the resource.

 

Hope this helps.

Copper Contributor

little confusing :) Sorry! Let's say I have a subscription called "Provider" which hosts the Sentinel Workspace. Now, I provide access to that access to customer subscription say "Customer1" which sends the logs to the Sentinel workspace hosted under "Provider" subscription. My understanding is, Sentinel cost will be incurred in "Provider" subscription and NOT in "Customer1" subscription. I think now the challenge is, how to do the cross charging to the customer. Tags will solve the costing issue but still, as a CSP provider we need to figure out how to consolidate the billing for the charges across Customer1 subscription and Provider subscription. HOpe that helps!

 

Cheers

Deepak

Copper Contributor
Deepak, I think the simple answer here is that services which incur costs go into customer subscriptions rather than provider ones. Lighthouse is just delegated access to the data in a customer subscription for providers. In your example above, Sentinel is deployed to Customer 1 and Provider is simply given access to the Sentinel data stored inside Customer 1's subscription. Tooling allows Provider to aggregate and report off of data across multiple customers without storing their own copy of that data.
Brass Contributor

@Pete Bryan One question. Assuming azure light house is in place. How do I know which incident in azure sentinel belongs to which tenant/subscription? I see tags option that is only the way to configure and identify? I also recall the preview feature 

Azure Sentinel Cross-Tenant/Cross-Workspace Incidents View. Assuming this will solve my requirement. Please confirm.
Microsoft

@Pavan_Gelli 

 

You are correct: the solution as described here does not consolidate incidents to a single workspace, hence no need to identify the workspace for an incindet. The corss workspace incidents screen now in private preview, which Lighthouse extends to be corss-tenant, indeed clearly dispays the worksapce and tenant for each incident.

Copper Contributor

With Sentinel in a Lighthouse setup where does the data storage retain? Is data retained in the MSP Managing tenant as well as the child tenant or will the data be retained in the child tenant? will that increase the ingestion and retention costs as well as potential security concerns around data leaving a customer tenant. We had some issues in our AWS environment when we tried to multi tenant a solution but customers pushed back because of the Data moving from a customer account to one of our managed accounts

Microsoft

@LeeAMurphy : the data stays at the "child" tenant and therefore does not affect pricing.

Brass Contributor

Hi @Pete Bryan : As I tried to click on slide link, it gives me a warning that its a malicious website. Could you please update the link to some secure repository?

Microsoft

@Deepanshu_Marwah : I am not sure which linked you rererred to, but I went through the links and found one that included a safelink, probably copied from outlook. I updated it. Let me know 

Brass Contributor

Thanks @Ofer_Shezaf its working alright now

Iron Contributor

Thanks for the great info; sharing with my Linkedin Network

Copper Contributor

@Ofer_Shezaf

 

We have several customer's tenants on-boarded into Lighthouse.  In our Lighthouse tenant, we can see all incidents in Sentinel.  However, we can't see the Owner of the incident.  Error is "Missing required permissions to list tenant users."  Is it possible to see who was assigned an incident within Lighthouse, ore can this only be seen within the customer's tenant?

 

 

Users.png

Copper Contributor

Hi @Ofer_Shezaf 

 

It looks like the list "delegatedRoleDefinitionIds" is now required in order to deploy the ARM template to manage resources. Which values should it contain  if roleDefinitionId is already set to 'ab8e14d6-4a74-4a29-9ba8-549422addade'?

Solved: this had to be removed from the template from Github.

 

Note: for your example, Logs Analytics Reader role also has to be assigned :)

 

Thank you

Brass Contributor

Good afternoon people.

I'm managing some clients with MSSP service and we're validating the integration via lighthouse.

The point we made is that we don't want to create a rule for every customer, but rather an X rule that is useful for all customers.

 

How could I do this through the resources that lighthouse offers me?

 

Would the solution be to create a rule in my Sentinel tenant and use the "union" command for each customer? If so, wouldn't that bring a lot of performance cost in the survey if there are many customers? Is there another better way?

I will be grateful for the response.

Copper Contributor

 I was able to do this, but now I have another issue currently:

I have set up the delegation with Sentinel Responder, Playbook Operator and Logic App operator role. The service provider is able to modify/assign/update incidents but unable to run the playbooks from under the incidents with the error:

Caller is missing required playbook triggering permissions on playbook resource <playbook-name-here>, or Microsoft Sentinel is missing required permissions to verify the caller has permissions

 

Can you tell me how to solve this ?

Version history
Last update:
‎Sep 21 2020 01:04 PM
Updated by: