This blog is part one of a three-part series focused on facilitating programmatic data pulls from Microsoft APIs.
Data collection and analysis is one of the most important aspects of a reactive Incident Response investigation or a proactive threat hunt exercise. Often one of the major contributing factors to the success of these investigations is not only how comprehensive your data collection is, but how fast the data is collected, what data is highlighted, and how consistent the data collection is across different investigations.
This blog series highlights ways you can leverage an Azure AD application registration and OAuth authentication to allow API access to alerts, incidents, and data in Microsoft 365 Defender and Microsoft Defender for Endpoint. This API access can enable programmatic Advanced Hunting queries and data pulls to improve hunting consistency, efficiency, speed, and completeness. While we will provide examples for Microsoft 365 Defender APIs, Microsoft Defender for Endpoint APIs, and Microsoft Graph API endpoints, the general guidance is suitable for any service that have APIs exposed.
To facilitate programmatic data pulls from Microsoft APIs there are 5 main steps, which we will cover in this blog series:
This blog will provide examples for leveraging PowerShell and C# code in Azure Functions. The same C# code can be used for a desktop app or a web API according to your need.
Registering an app in Azure AD allows us to facilitate programmatic access to the Microsoft Security APIs, since Azure AD is the identity authority authorizing access to the tenant data. Using an OAuth application can also simplify cross-tenant authentication, in the case where a third-party Incident Response provider, like the Microsoft Detection and Response Team (DART), or another external party needs access to the tenant security information in a secure, programmatic manner.
Microsoft 365 Streaming API supports events streaming to an Azure Event Hub or an Azure Storage account. The Streaming API is often used in situations in which an organization would need to send all live, ongoing events to a location for continuous, permanent monitoring or consumption. For example, it can be used when an organization is integrating Microsoft 365 data into a Security Information and Event Management (SIEM) solution, such as Microsoft Sentinel.
However, streaming APIs may not be the best option for a targeted incident response investigation. The Streaming API does not allow for specific, custom Advanced Hunting queries to be executed nor does it allow for time targeting, both of which can be critical for finding the data or malicious events. Additionally, streaming all events will come with storage, Compute, and networking costs depending on your processing method. Finally, if the OAuth application is leveraged to provide access to a third-party or different organization for the purposes of an investigation, streaming all events may provide way more data than desired and may make it significantly more challenging to find “the needle in the haystack”.
The first thing that must be done is create an OAuth application that will be granted the permissions necessary to the desired resources. For the purposes of this blog series, we will focus on Azure AD.
To leverage an OAuth application there are a few things that must be done:
The OAuth application can be created as either a single-tenant or multi-tenant application. Which option you leverage will depend on your circumstances, but for the purposes of this content we will provide an example of a single-tenant application. The single-tenant OAuth app is designed to be used just for a single Azure Active Directory tenant, whereas a multi-tenant OAuth app can be granted permission to multiple tenants.
The Azure account used in this section must have permission to manage applications in Azure AD. Any of the following Azure AD roles include the required permissions:
Here's a detailed set of steps to create a basic OAuth application to suit our needs:
Depending on the intended purpose of the OAuth application you will be creating, for most API resources there are often either “Read” or “ReadWrite” permissions options. To adhere to the concept of least privilege we will configure the application with the minimum required permissions and choose read-only permissions.
For the sake of simplicity, you can configure permissions for multiple API endpoints in a single application. As you go through the steps below keep in mind that you should not configure permissions you do not intend to use
To complete these steps, you will need to grant tenant-wide admin consent which requires you to use an Azure AD user account with one of the following roles:
Here's a detailed set of steps to configure the new OAuth application permissions to suit our needs:
At this point you should have all the permissions you need assigned to the application. However, they are not quite effective yet – under “Status” they should all be showing as “Not granted for <tenant name>”. The next step is to remedy that by granting consent for the permissions for the tenant.
The permissions should now be effective for the tenant. Below is an example screenshot that depicts an application with permissions successfully configured for the tenant and shows an example set of permissions that can be valuable for Incident Response purposes.
After the application is configured as described above, everything is ready for the application to be used except for setting the authentication mechanism – you can configure either certificate-based authentication or a client secret (text string). Certificates are more secure and should be used where possible.
To complete the steps in this section you will need to use an Azure AD account that is configured as an owner of the application if you are not using an administrative Azure AD role. These steps should also be ideally performed on a privileged access workstation to further reduce the risk of credential theft.
If you choose to use a certificate for authenticating to the application then we recommend that these two steps, “Creating a certificate” and “Uploading the certificate”, are done at the same time and by the same person. That will eliminate the need to transmit any key material via email or chat, because the certificate will only need to be stored in memory or temporarily on disk on a single system. If you choose to use a secret for authentication instead of a certificate you can go directly to “1.3.2 Secrets”.
The creation of certificate should be done by the process recommended by your organization – For this blog we will demonstrate creating a certificate in an Azure Key Vault.
The certificate generated in the above step needs to be uploaded to the OAuth app created above.
An alternate authentication mechanism is through client secrets in the app. If you choose to use a secret for authenticating to the application then we recommend that these two steps, “Create a secret” and “Store the key material”, are done at the same time and by the same person. Doing this will eliminate the need to transmit any key material via email or chat, because the will only need to be stored in memory or temporarily on disk on a single system. If you choose to use a certificate for authentication instead of a secret, you will need to complete the previous section “1.3.1 Certificates”. NOTE: As mentioned above certificate-based authentication is the recommended approach as certificates are more secure and should be used where possible.
These steps will be used to generate a client secret in the OAuth application.
The key value generated in the previous step, in this case a secret, should be stored securely. While it may be tempting to hard code the secret in a PowerShell script or C# Azure function, that is not secure and should be avoided. There are several methods securely store it. In this case, we will demonstrate using an Azure Key Vault. If you do not already have an Azure Key Vault created, read the quickstart article on creating an Azure Key Vault with the Azure CLI.
Here we will leverage the Azure CLI to store the secret in a key vault. To complete these steps, you will need to use an Azure AD account that has permissions to write secrets, such as “Key Vault Secrets Officer” or “Key Vault Certificate Office”. See the following article for additional details: Grant permission to applications to access an Azure key vault using Azure RBAC for additional details. These steps should also be ideally performed on a privileged access workstation to further reduce the risk of credential theft.
az keyvault secret set --vault-name "<keyVaultName>" --name "<nameYouWillUseToRetrieveTheSecretInTheFuture>" --value "<SecretValue>"
az keyvault secret show --vault-name "<keyVaultName>" --name "<nameYouWillUseToRetrieveTheSecretInTheFuture>"
In this first blog of a planned three-part series, we introduced a method by which an organization can leverage an Azure AD OAuth application to collect data as part of an Incident Response investigation. We discussed 5 broad steps to this methodology:
The overall objective of this method is to enable programmatic access for data collection and analysis purposes, which will in turn improve collection speed and consistency. Once collection of Incident Response-pertinent data is automated and stored someplace where it can be easily accessed, the sky is the limit – rules can be written to highlight specific events, data can be correlated with external sources, and so on.
In addition to an overview of the methodology we also detailed “step 1” of the methodology, how to prepare an Azure AD OAuth application registration. Examples were provided for a few Microsoft security APIs, but the method is applicable to any API that is exposed and supports OAuth authentication.
In the next blog in the series, we plan to cover steps 2 and 3, how to get an access token for the API resource and how to provide that access token to the API resource. In the final blog in the series, we plan to cover steps 4 and 5, how to review and parse the results and storing the results for analysis.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.