Access Azure Sentinel Log Analytics via API (Part 1)
Published Mar 23 2020 09:44 PM 30.7K Views
Microsoft

rinure_0-1585016032000.png

In this blog, I want to discuss a use case that is not very well documented, "... how do I use the Log Analytics API to connect to my Sentinel's Workspace and query the data?" This is a common ask and once you know the steps, its quite simple. Here in part 1, I will show you step-by-step how to register an application within your Azure Active Directory, Add your application to your Azure Sentinel's Log Analytics Workspace, and finally test your newly registered application to query any data set within your Sentinel's ALA Workspace.

 

Steps:
1. Register an AAD Application
2. Give the AAD Application permissions to your (Sentinel) Log Analytics Workspace
3. Preparing to Call the API
4. Procedure Overview
5. Querying (Sentinel) Log Analytics API
6. Request Format (Trouble Shooting)

Register an AAD Application

 

This is step 1. You need to log into your Azure Active Directory and click on "App registrations". 

 

rinure_0-1585017448513.png

 

You will see at the top of the App registrations page, "+ New registration". Click to start a new application registration.

 

rinure_1-1585017630117.png

 

It is my recommendation that you give your newly registered application a name that fits the type of API connection. In this case, I called my app "LogAnalyticsAPI". You will need to remember the name of this app when we add this app to your Sentinel's Log Analytics Workspace.

 

rinure_0-1585263766710.png

 

 

For now, I have no need to choose Accounts in any organizational directory that is Multitenant or that uses a personal Microsoft Account. So you will see that I chose the first option. Also, I have chosen "Web" as my application type. Keep this option.

 

Click "Register".

 

Once the application has registered, you will be taken to the "Overview" page for your new application. It should look something like this:

 

rinure_2-1585263961321.png

 

From this point we want to set up the "API permissions" and the "Certificates & secrets". Let's start with the API permissions.

 

In order for this application to be able to access and query your Sentinel's Log Analytics Workspace, you need to configure the following permissions:

 

  • Log Analytics API

rinure_1-1585263894138.png

 

Make sure you choose the "Application permissions" so your application runs as a background service or daemon without a signed-in user. Also, make sure to check the box next to "Data.Read" under the Data dropdown.

 

rinure_2-1585018769837.png

 

Click "Add permissions".

 

Your applications current configuration page should now look something like this:

 

rinure_3-1585018962902.png

 

Now that we have the right configured permissions, you need to "Grant admin consent" in order for your application to work.

 

Click "Grant admin consent for...".

 

Click "Yes".

 

rinure_4-1585019118634.png

 

rinure_5-1585019165423.png

 

You have successfully granted admin consent for these permissions to work with this registered application.

 

Our next step is to move on to the "Certificates & secrets" so the application can authenticate.

 

Click "Certificates & secrets".

 

Click "+ New client secret".

 

rinure_7-1585019438907.png

 

rinure_4-1585264131743.png

 

As you can see I have created a client secret with the Description "LogAnalyticsAPISecret". I have also set this secret to never expire.

 

Click "Add".

 

 

rinure_0-1585264252346.png

 

You will want to copy the value of this secret. You will need it later when you test your API connection to your Sentinel's Log Analytics Workspace.

 

With our Secret in place and our registered application permissions set to use the Log Analytics API, we are now ready to move on to step 2.

 

Give the AAD Application permissions to your (Sentinel) Log Analytics Workspace

 

This step is to ensure that users that authenticated against your AAD Application will have the access to your Sentinel's Log Analytics workspace. Let's start by finding your Sentinel's Log Analytics Workspace resource within your Azure Portal.

 

rinure_0-1585020117690.png

 

In my case, my Sentinel's Log Analytics Workspace is called "SentinelAnalytics". What ever you have called your workspace, make sure to choose that workspace, because you need to add the application as an "Access control" so that your registered application has "read" access to your Sentinel's Log Analytics Workspace. We do that by applying our application through "Access control (IAM)".

 

rinure_2-1585020558656.png

 

Click "+ Add".

 

Click "Add role assignment".

 

rinure_1-1585264379050.png

 

We do not want to give our registered application any more permissions than is necessary, so by assigning the "Log Analytics Reader" role we are only giving our application the permission to read the data upon query. This way your application does not have any execution or write permissions giving way to compromise.

 

Click "Save".

 

rinure_2-1585264449112.png

 

Congrats! This concludes steps 1 & 2 and our application is now registered, configured with API permissions and added to our Log Analytics Workspace with the "Log Analytics Reader" role to read/query data from our Sentinel's ALA Workspace. We can now proceed to step 3. "Preparing to Call the API."

 

Preparing to Call the API

 

In this step we will use the following outline to execute an OAuth2 client credentials request. We need to acquire an access token however if we want to send requests to the Log Analytics API. To do this, we will use a POST request to the AAD Token endpoint using the following:

 

POST /YOUR_AAD_TENANT/oauth2/token HTTP/1.1
Host: https://login.microsoftonline.com
Content-Type: application/x-www-form-urlencoded
grant_type=client_credentials
&client_id=YOUR_CLIENT_ID
&redirect_uri=YOUR_REDIRECT_URI
&resource=https://api.loganalytics.io
&client_secret=YOUR_CLIENT_SECRET

 

A successful token request will have an access token. We will need this token in order to authenticate and make a query to our Log Analytics Workspace over API.

 

For testing purposes, I will be using POSTMAN. This is an open source application that provides an thick client interface for testing each stage of our API call. The first stage will be our POST for our access token. Lets look at how to configure our POSTMAN UI to get our access token.

 

Procedure Overview

 

The first thing I like to do is set up my global variables. There is a COG in the upper right hand corner of your POSTMAN. Use this to open your environment and set your global variables.

 

rinure_1-1585021980966.png

 

The 4 global variables you will need to set are:

  • grant_type = "client_credentials"
  • client_id = [YOUR APPLICATION (CLIENT) ID]
  • client_secret = [YOUR APPLICATION SECRET]
  • resource = [https://api.loganalytics.io]

They should look something like this:

 

rinure_0-1585021855829.png

 

Now, none of the information I have screenshot and shared in this blog post is active. After walking through these steps and making sure these tokens and steps were valid, I removed the application and its association to my Sentinel's Log Analytics Workspace. Hahaha, Give me some credit...

 

Querying (Sentinel) Log Analytics API

 

Ok, here is a screenshot of my global variables working to request the access token I need to make my Log Analytics API call... You will need to set your Request to POST and use the following URL:

 

https://login.microsoftonline.com/{your_directory_tenant_id}/oauth2/token

 

Then you will need to add your variables to the Body of your POST and choose the "x-www-form-urlencoded" radio button.

 

To add your variables simply click into the text box under KEY. Add the following:

 

grant_type {{grant_type}}
client_id {{client_id}}
client_secret {{client_secret}}
resource {{resource}}

 

Click the Blue "Send" Button.

 

You should see output that looks like:

 

{
"token_type": "Bearer",
"expires_in": "3599",
"ext_expires_in": "3599",
"expires_on": "1585026197",
"not_before": "1585022297",
"resource": "https://api.loganalytics.io",
"access_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IllNRUxIVDBndmIwbXhvU0RvWWZvbWpxZmpZVSIsImtpZCI6IllNRUxIVDBndmIwbXhvU0RvWWZvbWpxZmpZVSJ9.eyJhdWQiOiJodHRwczovL2FwaS5sb2dhbmFseXRpY3MuaW8iLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC9lYjgxYzRjOS0yNTQ2LTQzZjItOGM0My05YzIyOTVhZjRiODgvIiwiaWF0IjoxNTg1MDIyMjk3LCJuYmYiOjE1ODUwMjIyOTcsImV4cCI6MTU4NTAyNjE5NywiYWlvIjoiNDJOZ1lCQm5uY0sxWVpKdTZiMm93NVBrTGpsWEFRQT0iLCJhcHBpZCI6IjE0ZjRmNGFlLWQyMjQtNDUxOC1iNDg2LTk1Y2E3OTcyMTlmZSIsImFwcGlkYWNyIjoiMSIsImlkcCI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0L2ViODFjNGM5LTI1NDYtNDNmMi04YzQzLTljMjI5NWFmNGI4OC8iLCJvaWQiOiJlNDUyNTM5MS1lODFhLTRmMzYtOWUxYS02MjIyYTJlYWZiZDciLCJyb2xlcyI6WyJEYXRhLlJlYWQiXSwic3ViIjoiZTQ1MjUzOTEtZTgxYS00ZjM2LTllMWEtNjIyMmEyZWFmYmQ3IiwidGlkIjoiZWI4MWM0YzktMjU0Ni00M2YyLThjNDMtOWMyMjk1YWY0Yjg4IiwidXRpIjoiTmZyeWlFX2Z2a1c1WGVqbVdrUTZBQSIsInZlciI6IjEuMCJ9.PVCG1G62DX1Fb3xb2QEJ7oMroW0NOC9Dfc_d9MDP20QVHHZgiijMhiteKMiZ4MLGuIiaTOVfkmXkN5CGXhcXCXfA3QLStVtqD5uehLHk5qwhHwevoLPTrfIk2ftKggYBFKRdB3GfUEwtV0hkhyTObMkJ9ZJZ-rpXyz50VAHHPqlZZFFFEtls7ceV0XmP-MNC7vnZL91EG_LgEV_7wea5X4ua-BXSh6eQlHcVvCPEm9C3vcKdVgQ1zP3VIpZXyPyYRO_nH-16IyVZCy-lmuxl-2w0CW0M9PdmbLpo-GDQW9jRwmT4Z_5Uh_9q6G3BvtrRQrUPhvGIAehXr3PFK6oilw"
}

 

Here is a screenshot for verification of what you should be seeing within your POSTMAN.

 

rinure_0-1585022523874.png

 

OK! With our Access Token now in hand, we need to change to our second POST tab and choose our Token TYPE in the drop down, that type needs to be "Bearer Token".

 

rinure_0-1585023041848.png

 

Then we need to paste our Access Token into the Token space:

 

rinure_1-1585023121044.png

 

 

Then in your Request dropdown, choose POST and use the following URL:

 

https://api.loganalytics.io/v1/workspaces/{your_sentinel_workspace_id}/query

 

You can find your Sentinel's Workspace ID by opening your Log Analytic Workspaces and clicking on your Sentinel's Workspace and on the right hand side under "Workspace Id" copy the ID.

 

rinure_2-1585023319945.png

 

Now switch to the "Body" tab within your POSTMAN application and choose "raw" and enter the following query:

 

{
"query": "SecurityAlert | where TimeGenerated >= ago(1d)"
}

 

Your screen should look something like this:

 

rinure_3-1585023549280.png

 

Click the Blue "Send" button.

 

You should get a Status: 200 OK and output in JSON format that looks something like the this:

 

rinure_4-1585023678914.png

 

Awesome! Job well done. At this point, this concludes Part 1 of this series. You have successfully registered an application that uses the Log Analytics API to authenticate to our Sentinel's Log Analytics API and make a query to return all SecurityAlerts that Sentinel has generated for the last 1 day.

 

In Part 2, I will show you how to use PowerShell to bring all of these pieces together and send these Sentinel Alerts to a 3rd Party Destination.

 

Until next time!

 

12 Comments

Thank you for sharing this Awesome blogpost with the Community :cool:

Iron Contributor

Hi, I'm not suggesting this isn't useful - but was wondering if it's possible to do the reverse?

We'd like to Query the Graph API from KQL? Is this possible?

 

Regards,

Dave C

Bronze Contributor

Nice blog!

Why does the Log Analytics return it's data as it does?

 

It seperates the column names from the rows which makes it really difficult to parse it and select certain columns.

 

Is there a way to retrieve it in a different format?

Copper Contributor

Very good article :)

Well you can use the below example code (python) to have this visible in a CSV type format:

 

response = requests.request("POST", url, headers=headers, data = payload)
data = response.json()

list = []
for report in data['tables']:
    column = []
    for col in report['columns']:
        column.append(col['name'])
    list.append(column)
    for col in report['rows']:
        list.append(col)

for l in list:
    print (l)

I have used it for backupjobs.

 

,Sheel

Copper Contributor

Hello @RinUre , thanks for this post.

I just want to make sure I understood the AAD app concept.
This app provides a way to authenticate to login.microsoftonline.com and receive the authorization token that allows me to query the log analytics API - api.loganalytics.io within the scope of the permissions that was allocated when the App was created - is it correct?

Brass Contributor

Hi Rinure,

 

This was a very good post. When can we expect the part 2 of this article.

 

Thanks

Copper Contributor

Hi everyone,
Solved, Edit: I should have change the raw input to JSON type.
After receiving the token from login.microsoftonline.com I added it as an Authorization header to the request,
Now when I make the requests, I don't get any results (the same query retrieved results in log analytics GUI)
Anyone know why?

image.png

Bronze Contributor

@DavidSho you are missing a where statement

| where TimeGenerated

Copper Contributor

I get 

{
"error": {
"message": "The provided credentials have insufficient access to perform the requested operation",
"code": "InsufficientAccessError",
"correlationId": "fa778534-4514-43e3-9a05-fa1dfef7bd77"
}
}
on using the generated token to post request on 
https://api.loganalytics.io/v1/workspaces/{your_sentinel_workspace_id}/query
Copper Contributor

Hi Guys,

 

We are still seeing permissions error:

Has there been any changes since this article was published?

agandhwale_1-1617892386595.jpeg

 

Kind Regards,

Ajinkya

Copper Contributor

the same error I get 

{
"error": {
"message""The provided credentials have insufficient access to perform the requested operation",
"code""InsufficientAccessError",
"correlationId""****************************************"
}
}
 
Is there any other way?
Copper Contributor

Thank you very much, you helped a lot

Version history
Last update:
‎Mar 26 2020 04:14 PM
Updated by: