Microsoft Tech Community - Latest Blogs -

Introducing SOC Optimization API

Jeremy Tan — Thu, 27 Jun 2024 03:04:38 PDT

SOC optimization is a new feature designed to combine the power of out of the box content with the flexibility of the SIEM to help you optimize your SOC processes and coverage to your organization’s specific needs, priorities, threats and environment. The first phase of this new feature helps you gain deep insights into your data usage patterns and coverage gaps against specific threats. It provides actionable recommendations to tighten your ingestion rates for data that doesn't provide security value, leverage correctly the data the does and improve your current coverage based on the threat landscape. You can learn more about the feature with the following resources.

Documentation: SOC optimization overview ; Recommendation’s logic

Short overview and demo: SOC optimization Ninja show

In dept webinar: Manage your data, costs and protections with SOC optimization

In this blog, we will focus on the API usage for SOC optimization. That’s right, if you didn’t know, there is an API available for you to interact with programmatically.

The API

Having an API for the SOC optimization feature is crucial for several reasons. We aim to unlock the power of precision-driven security and empower security teams through API with flexibility in automation, integration, customization, scalability and real-time access to SOC optimization data.

Refer to the Swagger specification and examples to learn more about the API.

Use cases

There are numerous scenarios where the SOC optimization API can be utilized. Here are some key use cases:

You can build custom reports and dashboards, for example, with Workbooks, Power BI, and other reporting tools. The Sentinel Optimization workbook has been updated with recommendation data via the API.
Integrate with third-party tools such as SOAR, ITSM, or any other applications that need to integrate with recommendations programmatically.
The API allows real-time access to SOC Optimization data. Security teams can retrieve up-to-date recommendations, trigger evaluations if needed, and respond promptly to the suggestions. Recommendations are calculated every 24 hours, and with the API you’re always up to date.
For customers or MSSPs managing multiple environments, the API provides a scalable way to handle recommendations across multiple workspaces.
You can export the data from the API and store it externally for audit, archiving, or tracking trends.

“We consider this feature as a valuable source of data for us and the customers we protect, it speeds up many tasks for us and provides meaningful insights we can act upon. The API and the reporting that it enables improves our efficiency and accuracy and reduces manual effort for custom reporting, thus reducing our costs and providing a better fidelity of service. “
Clive Watson – Solution Director, Quorum Cyber

Available actions

Below is a summary of the API actions and sample their sample responses.

1. Get recommendations

Use this action to list all the recommendations in your workspace.

GET /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{workspaceName}/providers/Microsoft.SecurityInsights/recommendations

2. Get recommendation

This allows you to get a specific recommendation by id. The id can be obtained from the previous action.

GET /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{workspaceName}/providers/Microsoft.SecurityInsights/recommendations/{recommendationId}

3. Patch recommendation

This allows you to update the status of a recommendation. For example, mark a recommendation as in progress, completed, dismissed or reactivate a recommendation.

Supported values when configuring the state property are ‘Active’, ‘InProgress’, ‘Dismissed’, ‘CompletedByUser’ and ‘CompletedBySystem’

PATCH /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{workspaceName}/providers/Microsoft.SecurityInsights/recommendations/{recommendationId}

4. Reevaluate recommendation

Use this action to manually trigger the evaluation for a recommendation.

POST /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{workspaceName}/providers/Microsoft.SecurityInsights/recommendations/{recommendationId} /triggerEvaluation

Sample workbook

If you need a sample for reference or to get started, you can refer to the Microsoft Sentinel Optimization Workbook as mentioned earlier. Install the workbook from the content hub, save the template, and launch the workbook. You will find the ‘SOC Optimization’ tab that visualizes the data based on the SOC optimization API.

Expand the items under ‘Details’ to drill down into each optimization type. Below are sample screenshots for ‘data value optimization’ and ‘threat-based optimization’ from the workbook.

Edit the workbook to check the parameters defined for SOC optimization (at the top of the workbook) and see how each visualization is built.

Customer story

Below is the case study shared by Quorum Cyber:

“When we first saw the preview of SOC Optimization the idea resonated with work we already do for customers as their MSSP. We knew that customers would be excited by this level of detail and insight provided and they would ask us questions about this. We were impressed with the API provided from day one, it worked immediately with Sentinel Workbooks, and we were able to almost immediately incorporate new reports on a customer-by-customer basis.

However, we were really interested in how we could scale this knowledge and be proactive. This is where the API really helped us; we were able to use it to detect all customers with data and bring those insights centrally to report on them. This gave us a few advantages; one was that we could see where a customer had an issue and where we might need to assist. For example, a customer has a warning about zero usage of a table (which you can’t detect otherwise), this helps our on-boarding team and improves our ongoing management, as it’s good to know that a table may have been asked for but isn’t used, or that over time the usage or importance of this may have changed and maybe we can adjust accordingly.

Detection coverage is a key part of us being an MSSP, we look to provide threat-led analytic rules to our customers, so having insights from the API on what might be missing and areas to investigate is crucial and looking at that data across customers at scale has given us many invaluable insights. For example, one customer being recommended coverage is important but having many customers with the same recommendation might mean this is a crucial task and we need to adopt the recommendation faster.

What we also appreciated was the link back to the Microsoft Sentinel GitHub for each analytic and the counts of active vs. available, so not only did we know that there were, for example, eight of 10 active detections deployed, but we had the GUID of that detection to look it up. With that data were able to correlate that GUID to our own GitHub repository to match to any customization we have done to that use case.”

Summary

Get started with the SOC optimization API today. We hope that this detailed walkthrough will help you unlock your use cases via the API.

Here is a list of useful resources mentioned in the blog:

API: Swagger specification and examples

Documentation: SOC optimization overview ; Recommendation’s logic

Short overview and demo: SOC optimization Ninja show

In dept webinar: Manage your data, costs and protections with SOC optimization

Workbook: Sentinel Optimization workbook

Using Keycloak with Azure AD to integrate AKS Cluster authentication process

sasina — Thu, 27 Jun 2024 00:00:00 PDT

Introduction

Integrating Azure Kubernetes Service (AKS) with Keycloak through Azure Active Directory (Azure AD) as an intermediary leverages Azure AD’s support for OpenID Connect (OIDC) to handle authentication and authorization. This integration enhances security, streamlines user management, and simplifies the authentication process for users accessing the AKS cluster.

Integrating Azure Kubernetes Service (AKS) with Keycloak through Azure Active Directory (Azure AD) Use case

The integration of AKS with Keycloak using Azure AD is highly applicable in all these industries and beyond that prioritize security, scalability, and efficient user management, making it a best practice for organizations leveraging cloud-based Kubernetes environments.

Financial Services: Securely manage and authenticate thousands of users accessing sensitive financial data and applications hosted on Kubernetes clusters.
Healthcare: Protect patient data and ensure compliance with healthcare regulations while providing medical staff secure access to applications.
E-commerce: Manage a large user base accessing e-commerce platforms and ensure secure transactions and user data protection.
Technology and Software Development: Provide developers with secure access to development environments and resources hosted on Kubernetes clusters.
Education: Manage access to educational resources and applications for students, faculty, and staff in a secure manner.

Figure 1: Similar use case architecture.

Concept

To make this integration possible and effective you should have a clear understanding of the following components, the concept of using Azure AD as an intermediary, and the pre-requisites.

Understanding the Key Components

What is AKS?

Azure Kubernetes Service (AKS) is a managed Kubernetes service that simplifies deploying, managing, and operating Kubernetes clusters in the cloud.

What is Keycloak

Keycloak is an open-source identity and access management solution that provides features like single sign-on (SSO), identity brokering, and user federation.

What is Azure AD?

Azure Active Directory (Azure AD/Microsoft Entra ID) is Microsoft’s cloud-based identity and access management service, which helps users access external resources like Microsoft 365, the Azure portal, and thousands of other SaaS applications.

Overview of OIDC

OpenID Connect (OIDC) is an identity layer built on top of the OAuth 2.0 protocol, allowing clients to verify the identity of end-users based on the authentication performed by an authorization server.

Why Use Azure AD as an Intermediary?

Using Azure AD as an intermediary offers several benefits:

Enhanced Security: Leverages Azure AD’s robust security features.
Simplified Management: Centralizes authentication and authorization.
Seamless Integration: Provides easy integration with Azure services and applications.

Pre-requisites for Integration

Before starting the integration process, ensure you have:

Azure AD: Set up and configured with necessary permissions.
Keycloak: Configured with a realm ready to be integrated.
AKS Cluster: Either existing or a plan to create a new one with Azure AD integration.

Figure 2: Mian concept

According to the image shown in Figure 2: Main concept, the authentication processes taken place in this concept is Azure Active Directory (Azure AD/Microsoft Entra ID) at the center, and from the right Azure Kubernetes Service requested an authentication and Azure AD will perform OIDC Token Exchange to Keycloak at the left, then Keycloak will perform validation and revert back to Azure AD while, Azure AD respond back to AKS.

Hands-on (Technical Steps)

More technical in these steps and processes, you will find step-by-step guide for seamless Integration.

Register Keycloak as an Application in Azure AD

Create an App Registration in Azure AD:

Navigate to Azure AD and create a new app registration.

Figure 3: Register Keycloak as an Application in Azure AD -1

Figure 4: Register Keycloak as an Application in Azure AD -2

Save the Client ID and Client Secret from Azure AD. This information will be needed later in Keycloak.

Obtain Client ID and Client Secret
After the registration is complete, go to the app's overview page and copy the "Application (client) ID".

Navigate to "Certificates & secrets" and create a new client secret. Copy the value of the client secret as it will not be shown again.

Figure 5: Register Keycloak as an Application in Azure AD -3

Configure API Permissions:

Go to "API permissions" and add the required Microsoft Graph API permissions. Typically, you need `User.Read` and `openid`, `profile`, and `email` permissions.

Figure 6: Register Keycloak as an Application in Azure AD -4

Figure 7: Register Keycloak as an Application in Azure AD -5

On click Add a permission, the above similar pane will be displayed as shown and you will click on Add permission. Then, after Add permission, you will have similar configuration to the below image.

Figure 8: Register Keycloak as an Application in Azure AD -6

Configure Keycloak to Use Azure AD

Set Up Keycloak:

https://www.keycloak.org/downloads
https://quay.io/repository/keycloak/keycloak
Log in to the Keycloak admin console.
Select the realm you want to configure or create a new realm.

Figure 9: Configure Keycloak to use Azure AD

Add Azure AD as an Identity Provider:

In the left menu, go to "Identity Providers".

Click "Add provider" and select "OpenID Connect v1.0".
Fill in the details:
Alias: A friendly name for the provider (e.g., AzureAD).
Display Name: A display name for the login button.
Authorization URL: `https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/authorize`
Token URL: `https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/token`
Client ID: The Application (client) ID obtained from Azure AD.
Client Secret: The client secret obtained from Azure AD.
Save the configuration.

Import OIDC Metadata:

Go to the Azure AD app registration overview and find the "OpenID Connect metadata document" URL.

Figure 10: Configure Keycloak: In Keycloak, use this URL to import the metadata automatically, which fills out most of the configuration fields.

Integrate AKS with Azure AD

Enable Azure AD Integration When Creating AKS Cluster:

Use the following Azure CLI command to create an AKS cluster with Azure AD integration

     az aks create \

       --resource-group myResourceGroup \

       --name myAKSCluster \

       --node-count 1 \

       --enable-aad \

       --aad-admin-group-object-ids <admin-group-object-id> \

       --enable-oidc-issuer \

       --oidc-issuer-url "https://<keycloak-server>/auth/realms/<realm>"

Replace the placeholders with actual values:

`myResourceGroup`: The resource group name.
`myAKSCluster`: The AKS cluster name.
`admin-group-object-id`: The object ID of the Azure AD group that will have admin rights.
`oidc-issuer-url`: The URL of the Keycloak OIDC issuer.

Update Existing AKS Cluster:

If you already have an existing AKS cluster, use the following command to enable Azure AD integration:

     az aks update \

       --resource-group myResourceGroup \

       --name myAKSCluster \

       --enable-aad \

       --aad-server-app-id <server-app-id> \

       --aad-server-app-secret <server-app-secret> \

       --aad-client-app-id <client-app-id> \

       --aad-tenant-id <tenant-id>

Outcome

The benefits of successful integration are not limited to the followings:

Streamlined User Management: Centralized user management through Azure AD.
Enhanced Security: Leverages Azure AD’s security features to protect your AKS cluster.
Simplify Security: Using Azure AD’s security features and Keycloak settings to simplifies the authentication process for users

Conclusion

We saw in this article how integrating AKS with Keycloak using Azure AD as an intermediary provides a robust and secure authentication solution. In addition, to the above steps, you can use `kubectl` to log into the AKS cluster test the integration, redirecting to the Azure AD login page, and after successful authentication, you should be redirected back to Keycloak if configured correctly. Verify that users have the appropriate roles and permissions in both Azure AD and Keycloak to access the AKS cluster. For troubleshooting, ensure the Client ID, Secret, and URLs are correct, and verify all required permissions and alignment of Keycloak and Azure AD settings and use HTTPS for all communications and regularly review and update access controls and permissions for security. Maintaining the system by keeping all components updated and implementing monitoring tools to track the health and performance of the integration and also, advanced configurations include adjusting claims in Keycloak to meet specific requirements and configuring group memberships according to organizational structure which will be in the next article. By following the steps outlined in this article, you can ensure a seamless and efficient integration process to simplify and enhance app security.

Microsoft Copilot in Azure Series - Copilot Access Management

Pierre Roman — Thu, 27 Jun 2024 00:00:00 PDT

Hello folks!

Today, we’re diving into Microsoft Copilot in Azure. It’s like having a super-smart assistant in the cloud!

It’s an AI-powered tool that’s all about making your life easier when you’re working with Azure, when you’re navigating the Azure portal, or using the Azure mobile app.

Now, keep in mind, at the time of recording this, Copilot in Azure is still in preview. That means it’s like a sneak peek, and there are some extra terms you have to check out before you jump in.

This Copilot in Azure can be a real lifesaver. It knows a ton about Azure’s services and resources, it also has access to all the information in Azure Resource Graph.

It’s like having a cheat sheet for the cloud. You can ask it questions about your environment, and it’ll give you answers tailored to your own Azure resources, and your level of access.

That means that Microsoft Copilot in Azure will only report on the resources that you have access to. It can only take the actions that you have permission to perform, and it requires confirmation before making changes to your environment.

It complies with all existing access management rules and protections such as:

1- Azure role-based access control (Azure RBAC),

2- Privileged Identity Management,

3- Azure Policy,

4- and resource locks.

You can chat with Copilot right in the Azure portal or on the go with the Azure mobile app. It’s like texting with a friend who can do some of your homework for you. It’ll answer questions, run queries, and even perform tasks for you, all while keeping your organization’s policies in check.

Oh, to use Microsoft Copilot in Azure you might need to have a chat with your network or security admins about a WebSocket connections to https://directline.botframework.com .

It's required for Copilot in Azure to work.

Now, Let’s dive into controlling the Access to Microsoft Copilot in Azure. By default, everyone in your tenant gets to play with Copilot in Azure. But, as the Global Administrator, you’ve got the power to manage access. Want to give specific users or groups a sneak peek? No problem! the video below shows you how to control access.

Remember, Copilot in Azure only helps you with the stuff you’re already allowed to touch. So, if you’ve already got access to certain data and resources, “Cool, let’s get to work!” But if it’s locked away from you, or you don’t have rights, Copilot respects that – no sneaking in or anything. It’s all about keeping things above board.

And that’s a wrap on this episode of ITOpsTalk "Using Microsoft Copilot in Azure"! Stay tuned to this channel for more episodes and updates

Catch you in the cloud!

Cheers!

Pierre

Language in Azure AI prompt flow

YanlingX — Wed, 26 Jun 2024 15:24:08 PDT

Prompt flow in Azure AI Studio is a development tool designed to streamline the entire development cycle of AI applications powered by Large Language Models (LLMs). Last Ignite, we announced Azure AI Language prompt flow available on GitHub. Today, we are excited to announce that Azure AI Language tooling is now available in prompt flow natively. With that, you can explore, quickly start to use and fine-tune various natural language processing capabilities from Azure AI Language, reducing your time to valueand deploying solutions with reliable evaluation.

The Azure AI Language sample flows in Azure AI prompt flow gallery are good starting point for you. You can simply start by cloning one of the two sample flows:

Analyze Documents: This flow is designed to analyze and extract insights from textual document input, such as identifying named entities, redacting Personal Identifiable Information (PII), analyzing sentiments, summarizing main points and translating to other languages.
Analyze Conversations: This flow is designed for conversational input and particularly useful for contact center analytics or meeting review, such as summarizing customer issues and resolution, analyzing customer sentiment trend during calls, redacting PII, chaptering long meeting into segments making it easy to navigate and find topics of interest.

Then you will see a wizard that guides you to configure tools in your flow, and run, evaluate, and deploy your flow:

Graph view of your flow
Files in your flow
Azure AI Language tools in the “More tools” dropdown menu, which you can add capabilities that you need for your flow. There are more tools that you can add from LLM, Prompt, and Python menu.
Configure output
Configure steps (or tools) in the flow
Run, evaluate, and deploy your flow

What’s Next

We will continue enhancing the underlying capabilities by leveraging state-of-the-art SLMs and LLMs, and enriching prompt flow offerings to further ease your effort in utilizing the best service Azure AI offers.

Learn more about Azure AI Language in the following resources:

Azure AI Language homepage: https://aka.ms/azure-language
Azure AI Language product documentation: https://aka.ms/language-docs
Azure AI Language product demo videos: https://aka.ms/language-videos
Explore Azure AI Language in Azure AI Studio: https://aka.ms/AzureAiLanguage
Prompt flow in Azure AI Studio: https://learn.microsoft.com/en-us/azure/ai-studio/how-to/prompt-flow
PyPl package (includes general documentation): promptflow-azure-ai-language · PyPI
Azure AI Language prompt flow github examples (includes READMEs): promptflow/examples/flows/integrations/azure-ai-language at main · microsoft/promptflow · GitHub

Advanced Network Observability for your Azure Kubernetes Service clusters through Azure Monitor

DalanMendonca — Wed, 26 Jun 2024 13:15:57 PDT

Last year at BUILD we announced the public preview of the Network Observability add-on. Today we’re excited to share a massive upgrade to the network observability capabilities available to AKS customers.

What’s new?

We are excited to announce Advanced Container Networking Services (ACNS), a suite of services built to significantly enhance the operational capabilities of your Azure Kubernetes Service (AKS) clusters. Advanced Network Observability is the inaugural feature of the ACNS suite bringing the power of Hubble’s control plane to both Cilium and Non-Cilium Linux data planes. With Advanced Network Observability, customers can now pinpoint network-related issues with more precision and detect root causes faster. The offering provides pod level packet statistics, DNS statistics, L4 connections and enhanced debugging capabilities with network flow logs and DNS error tracking.

Data flow for Advanced Network Observability

How can Azure customers use the new offering?

ACNS is integrated deeply with existing Azure Monitor capabilities for monitoring AKS clusters. Using the new capabilities requires no extra configuration from customers. Customers can enable advanced network observability with a single command. The metrics are then ingested automatically by Azure Monitor managed service for Prometheus. Additionally, customers can visualize the metrics in pre-built dashboards in Azure Managed Grafana. Customers have access to 6 pre-built dashboards in Azure Managed Grafana covering a breadth of signals from cluster traffic to DNS and pod level flows.

Pod flows dashboard built into Azure Manged Grafana

How to get ACNS

To get started with ACNS capabilities, please see our onboarding documentation