This blog is part two of a three-part series focused on facilitating programmatic data pulls from Microsoft APIs.
In part one of this series we discussed how an organization can leverage an Azure AD application registration and OAuth authentication to allow API access to alerts, incidents, and data in Microsoft 365 Defender and Microsoft Defender for Endpoint. This API access can enable programmatic Advanced Hunting queries and data pulls to improve hunting consistency, efficiency, speed, and completeness.
We specifically provided a high-level overview of the general methodology that can be used, broken down into 5 main steps, and provided details on how one can prepare an Azure AD OAuth app registration. In this second part of the series, we will cover steps 2 and 3, while in part three we will cover steps 4 and 5.
This blog will provide examples for leveraging PowerShell and C# code in Azure Functions. The same C# code can be used for a desktop app or a web API according to your need. Specifically our examples here will focus on the new Microsoft Graph APIs that were released for Public Preview that are documented in Use the Microsoft Graph security API - Microsoft Graph beta and discussed in the blog about the new Microsoft 365 Defender APIs in Microsoft Graph available in public preview.
After you create your OAuth app in the tenant you wish the access, successfully configure the necessary permissions, and establish a way to authenticate (i.e., setup a certificate or a secret), the second step is to authenticate against the OAuth app to get an access token you can use to access the API resource.
In part one of the series when we covered step “1.3 Configure Credentials”, which discusses how you can configure either a certificate or a secret for authentication. We also recommended leveraging an Azure Key Vault to securely store the key material.
Regardless of how you stored the credentials, however, the next thing you need to do is ensure your application accesses the stored credentials. Possible methods include using PowerShell to retrieve a certificate or secret from an Azure Key Vault, using C# to access a certificate or a secret from an Azure Key Vault, or even manually passing in a secret during code execution as a parameter.
One method is to leverage PowerShell to directly retrieve a secret using a command from the Az.KeyVault module.
What you will need:
Once you have this, ultimately this is the command you will need to get executed:
Connect-AzAccount
Get-AzKeyVaultSecret -VaultName "<keyVaultName>" -Name "<nameYouWillUseToRetrieveTheSecretInTheFuture>"
If you store the output to a variable named “$appSecret”, you can then access the data you need via “$appSecret.SecretValueText”. Here is a screenshot example:
Figure 1: Get Secret using Get-AzKeyValultSecret
A second method is to leverage PowerShell to call the Azure CLI to retrieve a secret.
What you will need: The name under which you stored the secret in the Azure Key Vault. Once you have this, ultimately this is the command you will need to get executed:
az keyvault secret show --vault-name "<keyVaultName>" --name "<nameYouWillUseToRetrieveTheSecretInTheFuture>"
However, to script it out in PowerShell you can do something like this:
There is a detailed PowerShell script example below. The “keyVaultName” variable is the name of the key vault and “secretName” is the name under which you stored the secret value.
$keyvaultName="<keyVaultName>"
$secretName="<nameYouWillUseToRetrieveTheSecretInTheFuture>"
Try {
$azLogin=$false
Write-Warning 'Attempting "az login". The default web browser will be opened at https://login.microsoftonline.com/common/oauth2/authorize. Please continue the login in the web browser. If no web browser is available or if the web browser fails to open, use device code flow with `az login --use-device-code`.'
az login *> $null
$azLogin=$true
}
Catch {
$_ | Write-Verbose
Write-Debug 'Error detected for "az login".'
Write-Error -ErrorAction Stop 'Error detected for "az login". Cannot continue.'
$azLogin=$false
}
Try {
If ($azLogin -eq $false) {Write-Error -ErrorAction Stop 'It does not look like "az login" was successful, so exiting.'}
$appSecret=(az keyvault secret show --vault-name "$keyvaultName" --name "$secretName" | ConvertFrom-Json).Value
If ([string]::isnullorempty($appSecret) -or $appSecret -eq "<App Secret>") {Write-Error -ErrorAction Stop 'The value for "`$appsecret" was either null or default value, so exiting.'}
}
Catch {
$_ | Write-Verbose
Write-Debug "Error detected when trying to retrieve the appsecret from the Azure Key Vault."
Write-Error -ErrorAction Stop "Error detected when trying to retrieve the appsecret from the Azure Key Vault. Cannot continue."
}
When executed interactively, the code above will open up a web browser to perform the authentication using the currently logged-in user. If you use the example above, elsewhere in your script you would access the secret value with “$appsecret”.
If you choose key vault certificates as the authorization mechanism, the code below can be used as an example for how to get the certificate from the Azure Key Vault using C#.
using System;
using System.Threading.Tasks;
using Azure.Identity;
using Azure.Security.KeyVault.Certificates;
using System.Security.Cryptography.X509Certificates;
namespace Sample
{
public class CertificateSample
{
public async Task<X509Certificate2> GetCert()
{
try
{
string defenderKeyVaultUri = "<CertificateKeyVaultURI>";
string certName = "<CertName>";
var certificateClient = new CertificateClient(new Uri(defenderKeyVaultUri), new DefaultAzureCredential());
X509Certificate2 cert = await certificateClient.DownloadCertificateAsync(certName);
return cert;
}
catch (Exception ex)
{
throw new Exception("Error getting certificate", ex);
}
}
}
}
Once we have what we need to authenticate as the OAuth app, the next step is to generate a request to the authentication endpoint to get an access token.
In order to get an access token you will need several pieces of information. If you are using a Microsoft Security API to access data in an Azure Tenant, you will need:
Depending upon which resource you need, you may most often need the “Graph” API endpoints.
If you’re not sure if you need to use the Authentication Endpoint for “Azure AD Government” you can follow the instructions at Azure AD authentication & national clouds - Microsoft Entra | Microsoft Docs > Frequently asked ques....
To summarize, you must do the following:
Figure 2: Example of identifying tenant type
Ultimately to use PowerShell & an application secret to get an access token regardless of whether you want to access Microsoft Graph, Microsoft 365 Defender, or Microsoft Defender for Endpoint APIs the important piece of code is going to boil down to this PowerShell code:
$resourceAppIdUri = "https://$resourceBaseUri/.default"
$oAuthUri = "https://$oAuthBaseUri/$TenantId/oauth2/v2.0/token"
$authBody = [Ordered] @{
scope = $resourceAppIdUri
client_id = $ClientId
client_secret = $ClientSecret
grant_type = 'client_credentials'
}
$authResponse = Invoke-RestMethod -Method Post -Uri $oAuthUri -Body $authBody -ErrorAction Stop
In the above example, for which I’ll give more complete code shortly:
If you take the code snippet above you can flesh it out into a full-fledged PowerShell function that:
The below PowerShell code would get an access token specifically for Microsoft Graph:
Function Get-GraphAccessToken() {
[cmdletbinding()]
param(
[parameter(Position=0)][string]$TenantId,
[parameter(Position=1)][string]$ClientId,
[parameter(Position=2)][string]$ClientSecret,
[ValidateSet("AzureADPublic","AzureADGovernment")]
[parameter()][string]$IdentityAuthority="AzureADPublic",
# NOTE: $IdentityAuthority CAN BE "AzureADPublic" EVEN IF THE apiCloudEnvironment IS "GCC"
[ValidateSet("Commercial","GCC","GCCHigh")]
[parameter()][string]$apiCloudEnvironment="Commercial"
)
switch ($IdentityAuthority) {
"AzureADPublic" {$oAuthBaseUri="login.microsoftonline.com"}
"AzureADGovernment" {$oAuthBaseUri="login.microsoftonline.us"}
"default" {$oAuthBaseUri="login.microsoftonline.com"}
}
switch ($apiCloudEnvironment) {
# Reference: https://docs.microsoft.com/en-us/graph/deployments
"Commercial" {$resourceBaseUri="graph.microsoft.com"}
"GCC" {$resourceBaseUri="graph.microsoft.com"}
"GCCHigh" {$resourceBaseUri="graph.microsoft.us"}
"default" {$resourceBaseUri="graph.microsoft.com"}
}
$resourceAppIdUri = "https://$resourceBaseUri/.default"
$oAuthUri = "https://$oAuthBaseUri/$TenantId/oauth2/v2.0/token"
$authBody = [Ordered] @{
scope = $resourceAppIdUri
client_id = $ClientId
client_secret = $ClientSecret
grant_type = 'client_credentials'
}
$authResponse = Invoke-RestMethod -Method Post -Uri $oAuthUri -Body $authBody -ErrorAction Stop
return $authResponse
}
When executed, it will look something like this:
Figure 3: Result of getting an access token
As you can see in the example above, I stored the results in a variable named “$graphAccessToken”. This value will be used in future steps.
In this C# example we are using the certificate retrieved from the Key Vault to get access token. Use this method if you use certificate as the authorization method.
using System;
using System.Threading.Tasks;
using Azure.Identity;
using Azure.Security.KeyVault.Certificates;
using Microsoft.Identity.Client;
using System.Security.Cryptography.X509Certificates;
namespace Sample
{
public class TokenSample
{
/// <summary>
/// Function to get token
/// </summary>
/// <param name="authority">https://login.microsoftonline.com/</param>
/// <param name="resourceuri">Resource URI.For eg - MDE Resource URI: https://api.securitycenter.microsoft.com/</param>
/// <returns>Token string</returns>
/// <exception cref="Exception"></exception>
public async Task<string> GetToken(string authority, string resourceuri)
{
try
{
//OAuth app secret from keyvault
string defenderKeyVaultUri = "<CertificateKeyVaultURI>";
string certName = "<CertName>";
//TenantId - customer tenant id
string TenantId = "<tenantid>";
//ClientId - OAuth App id
string ClientId = "<clientid>";
var certificateClient = new CertificateClient(new Uri(defenderKeyVaultUri),new DefaultAzureCredential());
X509Certificate2 cert = await certificateClient.DownloadCertificateAsync(certName);
string[] scopes = new string[] { $"{resourceuri}.default" };
IConfidentialClientApplication app = ConfidentialClientApplicationBuilder.Create(ClientId)
.WithCertificate(cert)
.WithAuthority(new Uri($"{authority}{TenantId}/"))
.Build();
var authResult = await app.AcquireTokenForClient(scopes).ExecuteAsync();
var token = authResult.AccessToken;
return token;
}
catch (Exception ex)
{
throw new Exception("Error getting token", ex);
}
}
}
}
If you are curious about the details of the access token, one way you can validate the parameters is to decode the access token using PowerShell. While explanation of the inner workings of the access token are outside the scope of this blog, you can read more information about the structure of the access token at RFC 7519: JSON Web Token (JWT) (rfc-editor.org). For Azure Active Directory tokens you see full explanations of the exact property meanings at Microsoft identity platform access tokens - Microsoft Entra | Microsoft Docs.
The function below will accept an input parameter of the access token. If you set the output of the example PowerShell function shown in step 2.1.2 to a variable named “$token”, for example, the access token would be a property named “$token.access_token”.
Function Parse-AccessToken([parameter(Position=0)][string]$oauthToken) {
$tokenArray=$oauthToken.Split(".").Replace("-", "+").Replace("_", "/")[0..1]
$tokenHeader=$tokenArray[0]
$tokenPayload=$tokenArray[1]
# Add "=" until string is valid base64 length
while ($tokenHeader.Length % 4) {$tokenHeader += “=” }
while ($tokenPayload.Length % 4) {$tokenPayload += “=” }
$tokenHeader=[System.Text.Encoding]::ASCII.GetString([system.convert]::FromBase64String($tokenHeader)) | ConvertFrom-Json
$tokenPayload=[System.Text.Encoding]::ASCII.GetString([system.convert]::FromBase64String($tokenPayload)) | ConvertFrom-Json
if ([string]::IsNullOrEmpty($tokenPayload.iat) -eq $false) {$tokenPayload.iat=[datetimeoffset]::FromUnixTimeSeconds($tokenPayload.iat)}
if ([string]::IsNullOrEmpty($tokenPayload.nbf) -eq $false) {$tokenPayload.nbf=[datetimeoffset]::FromUnixTimeSeconds($tokenPayload.nbf)}
if ([string]::IsNullOrEmpty($tokenPayload.exp) -eq $false) {$tokenPayload.exp=[datetimeoffset]::FromUnixTimeSeconds($tokenPayload.exp)}
return New-Object -TypeName PSCustomObject -Property @{"header"=$tokenHeader;"payload"=$tokenPayload}
}
The output of the function is an object with two properties: header and payload. The data we are most interested in is the “payload”, where we can see various pieces of information including the following:
Figure 4: Example of a parsed access token
Two pieces of data of particular interest are the token expiration date (exp) and the permissions (roles) assigned. Additionally, you can see that this access token is valid for the endpoint (aud, “audience”) https://graph.microsoft.com.
Figure 5: Example of the "roles" in a parsed access token
In the example above you can see this access token was granted the roles “SecurityEvents.Read.All”, “SecurityAlert.Read.All”, “SecurityINcident.Read.All”, and “ThreatHunting.Read.All”. We can use this to confirm that we have the “SecurityAlert.Read.All” permission necessary to use the List alerts_v2 - Microsoft Graph beta | Microsoft Learn API and the “SecurityIncident.Read.All” permission necessary to use the List incidents - Microsoft Graph beta | Microsoft Learn API, for example.
Once we have an access token, we can submit the access token to the API resource endpoint (i.e., the “audience”).
To submit the request to the API endpoint you basically only need to know five things, at least for a Microsoft security API:
Identifying these pieces of information is the most step that needs to be completed before we can successfully send a request to the API endpoint. The actual submission is often relatively straightforward.
If we put all the example pieces of information together from items 2 – 5 above, we would get this full, example URI:
https://graph.microsoft.com/beta/security/alerts_v2?$top=100&$skip=200
If we request data from that API URI we would get a list of 100 alerts from Microsoft 365 Defender, as accessed from Microsoft Graph, after the first 200 alerts were skipped.
The best way to use PowerShell to submit a request to an API is to use the “Invoke-WebRequest” command, where there are three main parameters that need to be set:
Assuming we are continuing to use our example of the List alerts_v2 - Microsoft Graph beta | Microsoft Learn API, the URI we will be using is the full example provided in step 3.1, and we have our access token stored in a variable named $graphAccessToken as grabbed in step 2.1.1, this PowerShell code sample will do the job:
$gurl = "https://graph.microsoft.com/beta/security/alerts_v2?$top=100&$skip=200"
$gheaders = @{
'Content-Type' = 'application/json'
Accept = 'application/json'
Authorization = "Bearer $($graphAccessToken.access_token)"
}
$gwebResponse = Invoke-WebRequest -Method Get -Uri $gurl -Headers $gheaders
That same format can be used for any Microsoft 365 Defender APIs, Microsoft Defender for Endpoint APIs, or Microsoft Graph API endpoints. When that code snippet is executed, the results will be stored in the “$gwebResponse” variable, with the actual data stored as JSON in the “content property (i.e., “$gwebResponse.Content”). The third and final part of this blog series will cover more details on how to retrieve and process the response data.
Below is a more fully-fledged script that increases the complexity of the code but drastically simplifies the execution:
param(
[parameter(Position=0)][string]$AccessToken
, [parameter()][ValidateRange(0,1000)][int]$top
, [parameter()][int]$skip
, [parameter()][string]$select
, [parameter()][string]$orderBy
, [parameter(ParameterSetName="All")][switch]$all
, [parameter(ParameterSetName="AdvancedFilter")][string]$filter
, [parameter(ParameterSetName="BasicFilter")][datetime]$createdAfter
, [parameter(ParameterSetName="BasicFilter")][datetime]$createdBefore
, [parameter(ParameterSetName="BasicFilter")][datetime]$updatedAfter
, [parameter(ParameterSetName="BasicFilter")][datetime]$updatedBefore
)
[array]$params=@()
[array]$filters=@()
switch ($PsCmdlet.ParameterSetName) {
"AdvancedFilter" {
if ([string]::IsNullOrEmpty($filter) -eq $false) {
$filters+=$filter
}
}
"BasicFilter" {
if ([string]::IsNullOrEmpty($createdAfter) -eq $false) {
$filters+="createdDateTime ge $($createdAfter.ToString("yyyy-MM-ddTHH:mm:ssZ"))"
}
if ([string]::IsNullOrEmpty($createdBefore) -eq $false) {
$filters+="createdDateTime lt $($createdBefore.ToString("yyyy-MM-ddTHH:mm:ssZ"))"
}
if ([string]::IsNullOrEmpty($updatedAfter) -eq $false) {
$filters+="lastUpdateDateTime ge $($updatedAfter.ToString("yyyy-MM-ddTHH:mm:ssZ"))"
}
if ([string]::IsNullOrEmpty($updatedBefore) -eq $false) {
$filters+="lastUpdateDateTime lt $($updatedBefore.ToString("yyyy-MM-ddTHH:mm:ssZ"))"
}
}
}
Try {
$at=Parse-AccessToken $AccessToken -ErrorAction Stop
$TenantId=$at.payload.tid
$apiBase=$at.payload.aud
}
Catch {
$TenantId=$null
$apiBase="https://graph.microsoft.com"
}
if ($filters.Length -gt 0) {$params+="`$filter=$($filters -join " and ")"}
if ($top -gt 0) {$params+="`$top=$top"}
if ($skip -gt 0) {$params+="`$skip=$skip"}
if ([string]::IsNullOrEmpty($select) -eq $false) {$params+="`$select=$select"}
if ([string]::IsNullOrEmpty($orderBy) -eq $false) {$params+="`$orderBy=$orderBy"}
$Operators=$params -join "&"
$gurlBase="$apiBase/beta/security/alerts_v2"
$gurl = "$gurlBase"+"?$Operators"
$gheaders = @{
'Content-Type' = 'application/json'
Accept = 'application/json'
Authorization = "Bearer $AccessToken"
}
$resultsGuid=$([guid]::NewGuid().Guid)
[array]$responseValues=@()
[array]$gresponseExpanded=@()
Invoke-WebRequest -Method Get -Uri $gurl -Headers $gheaders
A line-by-line review of the script is outside of the scope of this blog, but in summary:
Assuming the script above is saved to C:\temp\OauthAlertsV2.ps1, this command would return the id, title, and createdDateTime values for the most recent 2 alerts:
.\temp\OauthAlertsV2.ps1 -AccessToken $graphAccessToken.access_token -top 2 -select "id,title,createdDateTime" -orderBy "createdDateTime desc"
Figure 6: Example of a normal, unparsed response
The above screenshot shows what a normal, unparsed response would look like.
Below is the way to submit the same API request but using C#.
Create a re-usable class in your project for holding deserialized responses.
public class ApiResponse
{
[JsonProperty("@odata.context")]
public string context { get; set; }
[JsonProperty("@odata.nextLink")]
public string nextLink { get; set; }
[JsonProperty("value")]
public List<dynamic> Values { get; set; }
}
Create a class and members used to invoke the request. Pay particular attention that HttpClient is intended to be instantiated once in the constructor and reused throughout its lifetime.
private string requestBaseUri = "https://graph.microsoft.com/beta/security/alerts_v2";
private readonly HttpClient apiHttpClient;
private static DefenderApiClient()
{
apiHttpClient = new HttpClient();
}
Create a method for building the filter, posting the request, and returning a response.
public static ApiResponse GetApiResponse(
string token, // Security token retrieved earlier
DateTimeOffset startUtcDate, // Events from this UtcNow date ...
DateTimeOffset endUtcDate, // ... to this UtcNow date
int skip = 0, // skip X number of records "paging"
int top = 1000 ) // return the top Y records for that page
{
if( !ParamterValidation( startDate, endDate, skip, top, nextLink ) )
{
// Apply your organizations parameter validation logic
throw new ArgumentException( "Invalid parameter supplied!" );
}
// Filters and parameters for building the API request
var filters = new List<string>();
var paramters = new List<string>();
var updatedAfterFilter = $"lastUpdateDateTime ge {startUtcDate.ToString("yyyy-MM-ddTHH:mm:ssZ")}";
filters.Add(updatedAfterFilter);
var updatedBeforeFilter = $"lastUpdateDateTime lt {endUtcDate.ToString("yyyy-MM-ddTHH:mm:ssZ")}";
filters.Add(updatedBeforeFilter);
parameters.Add("$filter=" + string.Join(" and ", filters));
parameters.Add("$orderby=lastUpdateDateTime desc");
parameters.Add("$top=" + top);
if (skip > 0)
{
parameters.Add("$skip=" + skip);
}
var operators = string.Join("&", parameters);
var requestUri = requestBaseUri + "?" + operators;
var request = new HttpRequestMessage(HttpMethod.Get, requestUri);
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token);
try
{
var webResponse = await HttpClient.SendAsync(request);
var contentString = await webResponse.Content.ReadAsStringAsync();
if (webResponse.IsSuccessStatusCode)
{
dynamic response = JsonConvert.DeserializeObject<ApiResponse>(contentString);
}
}
catch (Exception ex)
{
throw new Exception(ex.Message, ex);
}
// Calling code levarages ApiResponse values, context, and nextLink for subsequent calls
return response;
}
In this second blog of a planned three-part series, we discussed steps 2 and 3 of the five broad steps for facilitating programmatic data pulls from Microsoft APIs, how to get an access token for the API resource and how to provide that access token to the API resource. The overview of the methodology and step 1, how to prepare an Azure AD OAuth application registration, was covered in part one.
At this point you should understand:
In our third and final blog in the series, we will discuss step 4 and step 5, how to review and parse the results and store the results for analysis.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.