How to Set Up Sentinel Data Connectors for Kubernetes and GitHub
A guide to configure and use Sentinel Connectors to collect logs and data from your Kubernetes clusters and GitHub CI/CD pipelines. Part 2 of 3 part series about security monitoring of your Kubernetes Clusters and CI/CD pipelines by @singhabhi and @Umesh_Nagdev Link to Part 1 Link to Part 3 Introduction In part 1 of this series, we discussed the type of log sources you should consider for monitoring the security of your Kubernetes environment. This blog will demonstrate how to connect some of the critical log sources using Sentinel Data Connectors. Sentinel Data Connectors are a set of tools that enable you to collect and analyze logs and data from various sources, such as cloud services, applications, devices, and networks. Sentinel Data Connectors can help you monitor the health, performance, and security of your Kubernetes clusters and GitHub CI/CD pipelines, as well as detect and respond to threats and incidents. In this document, we will show you how to set up Sentinel Data Connectors for three types of sources: Kubernetes clusters, GitHub CI/CD pipelines, and Defender for Containers alerts and Defender for Cloud recommendations. We will also explain how to use the connectors to view and query the collected data in Sentinel. Security monitoring use cases Let’s first highlight some security risks you would want to monitor with Sentinel: 1. Pod Security Monitoring: Log source: Defender of Containers Risks monitored: Detect unauthorized or suspicious pods running in the cluster. Monitor for privilege escalation attempts within pods. Track and alert on changes to pod security policies. 2. Network Security Monitoring: Log source: Defender of Containers Risks monitored: Identify and alert on unexpected network traffic patterns. Monitor for unauthorized ingress and egress traffic. Detect and investigate potential denial-of-service (DoS) attacks. 3. Container Image Security: Log source: Defender for Cloud - Defender Cloud Security Posture Management (DCSPM) Risks monitored: Scan container images for vulnerabilities before deployment. Monitor for unauthorized or unsigned images. Track changes to container image repositories. 4. Kubelet Activity Monitoring: Log source: Defender of Containers Risks monitored: Monitor kubelet logs for signs of compromise or unauthorized access. Detect abnormal activities related to node management. 5. API Server Security: Log source: Defender of Containers Risks monitored: Monitor Kubernetes API server logs for suspicious activities. Track and alert on failed authentication attempts. Detect unusual API server request patterns. 6. RBAC (Role-Based Access Control) Monitoring: Log source: AKS Diagnostics Logs, Azure AD logs, Azure Monitor Container Insights Risks monitored: Monitor changes to RBAC policies and roles. Detect and alert on unauthorized access attempts. Track role binding changes and escalations. 7. Secrets and ConfigMap Access Monitoring: Log source: Defender of Containers Risks monitored: Monitor for unauthorized access to Kubernetes secrets and ConfigMaps. Detect changes to sensitive configuration data. Track usage patterns of sensitive information. 8. Audit Logging: Log source: AKS Diagnostic Logs Risks monitored: Enable and monitor Kubernetes audit logs for cluster-wide activities. Correlate audit logs to identify security events and policy violations. Regularly review audit logs for anomalies and potential threats. 9. Compliance Monitoring: Log source: Defender for Cloud - Defender Cloud Security Posture Management (DCSPM) Risks monitored: Ensure compliance with security standards and policies. Monitor for deviations from security best practices. Generate reports on compliance status and potential risks. 10. Container Runtime Security: Log source: Defender of Containers Risks monitored: Monitor runtime activities of containers for abnormal behavior. Detect and alert on suspicious system calls within containers. Integrate with container runtime security tools for enhanced monitoring. 11. Incident Response and Forensics: Log source: Defender of Containers Risks monitored: Develop and test incident response plans for Kubernetes security incidents. Monitor for indicators of compromise (IoCs) and initiate investigations in Sentinel Collect and analyze forensics data in the event of a security incident in Sentinel 12. Cluster Health Monitoring: Log source: AKS Diagnostic Logs Risks monitored: Regularly monitor the overall health of the Kubernetes cluster. Detect and alert on abnormal resource consumption or performance issues. Ensure the availability of critical components and services. Prerequisites Before you can set up Sentinel Data Connectors, you need to have the following: Sentinel workspace. This is where you store and analyze the data collected by the connectors. Enable Sentinel on the Log Analytics Workspace where you are exporting all of the below mentioned log sources . Instructions on how to setup Sentinel Kubernetes cluster. This is the source of the data for the Kubernetes Cluster using Diagnostics logs. You can use any Kubernetes cluster that supports the Kubernetes API, such as Azure Kubernetes Service (AKS), Google Kubernetes Engine (GKE), or Amazon Elastic Kubernetes Service (EKS). We will showcase this with AKS. Instructions on how to deploy AKS GitHub account. This is the source of the code and manifests used for creating container images which are then deployed in your Kubernetes Clusters. You will need to configure DCSPM DevOps security for secure scanning of artifacts. Or if you are using a third-party scanning tool you will need to send the scan results to Sentinel. Container Registry. The images stored in the registry need to be scanned for vulnerabilities. You will need access to the scan logs this can be done via Defender for Cloud DCSPM Defender for Containers subscription. This is a service that provides security and compliance monitoring for your Kubernetes clusters. You need to enable Defender for Containers on your subscription where your Kubernetes cluster is located and configure it to send alerts to the Sentinel workspace. Instructions on how to enable Defender for Containers on a subscription. A Defender for Cloud DSPM subscription. This is a service that provides security and compliance recommendations for your cloud resources such as AKS, ACR, and Azure tenant. You need to enable Defender for Cloud DCSPM on your subscription with AKS cluster and configure it to send recommendations to the Sentinel workspace. Instructions on how to enable DCSPM on a subscription. How to Set Up Kubernetes Cluster Connector The Kubernetes Cluster connector allows you to collect logs and metrics from your Kubernetes cluster, such as cluster events, pod logs, node metrics, and container metrics. To ingest AKS logs into Sentinel, deploy the Azure Kubernetes Solution for Sentinel then, follow the steps below to enable the AKS data connector. Configure AKS data connector to ingest logs into Sentinel: In Microsoft Sentinel, go to the "Data connectors" page. Find and configure the "Azure Kubernetes Service (AKS)" connector. Launch the Azure Policy wizard under configuration to enable logging. Verify Integration: After configuration, verify that logs from your AKS cluster are flowing into Sentinel. Create Sentinel Workbooks and Queries (to be elaborated in part 3): Leverage Microsoft Sentinel workbooks and Kusto Query Language (KQL) queries to create visualizations and reports based on AKS logs. Customize the workbooks and queries based on your specific security and monitoring requirements. Set Up Alerts and Incidents (to be elaborated in part 3): Configure alerts within Microsoft Sentinel based on specific events or patterns detected in AKS logs. Set up incidents and response workflows to investigate and respond to security events. Monitor and Fine-Tune: Regularly monitor the integration, alerts, and logs to ensure that the AKS logs are being properly processed in MicrosoftSentinel. Fine-tune your configurations based on feedback, new security requirements, or changes to your AKS environment. How to Set Up GitHub connector To ingest logs into Sentinel, deploy the Microsoft Sentinel - Continuous Threat Monitoring for GitHub. Enable the two connector that are installed as part of this solution: GitHub Enterprise Audit Log connector: this connector collects GitHub audit logs which tracks changes to repository, user added/removed, pull request activities, etc. GitHub (using Webhooks) connector: to ingest you can ingest the scan data using a built in data connector for GitHub events. This connector can pull events related to code scanning alert, repository vulnerability alert (via Dependabot) and Secret Scanning Alert. How to Set Up Defender for Containers Alerts and Defender for Cloud Connector Sentinel has a buil-in data connector to ingest Defender for Cloud alerts and recommendation. You can find the details https://learn.microsoft.com/en-us/azure/sentinel/connect-defender-for-cloud#connect-to-microsoft-defender-for-cloud Setting up AKS data connector and additional logging for Sentinel Setup the Diagnostic Settings for the Azure Kubernetes Services to send the events to a Sentinel-enabled Log Analytics workspace. https://learn.microsoft.com/en-us/azure/aks/monitor-aks#aks-control-planeresource-logs. In our scenario we are using the following logs In addition, you will also need to enable Container Insights to get the Pod level data so you can run the search queries for risk related to Pod specifics like pods running in Default namespace. You can refer to this https://learn.microsoft.com/en-us/azure/azure-monitor/containers/kubernetes-monitoring-enable?tabs=cli#enable-container-insights resource to enable Container Insights. The logs will go to ContainerLogsV2 in Log Analytics Workspace https://learn.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-logs-schema#enable-the-containerlogv2-schema The following pic shows the ContainerLogv2 Schema as an example. You will need to for to Sentinel Content Hub and enable the following. This will give you Workbook, several hunting queries and a data connector to ingest AKS data. Your AKS cluster will populate the data in the following tables, which we will use to write custom search queries in the section below.Correlating Microsoft Defender for Cloud alerts in Sentinel
Incident load is one of the major pain points of modern SOC. One of the methods used to manage alert fatigue is using grouping. In this blog post we will demonstrate a concept for correlating Microsoft Defender for Cloud alerts using Microsoft Sentinel analytics rules. This can be applied to many scenarios made possible using the new ‘Sentinel entities’ entity mapping.Modernize Log Management with the Maturity Model for Event Log Management (M-21-31) Solution
Announcing the Microsoft Sentinel: Maturity Model for Event Log Management (M-21-31) Solution. This solution consists of (1) Workbook, (8) Analytics Rules, (4) Hunting Queries, and (3) Playbooks. The solution provides actionable insights into log management posture and intuitive steps for remediation to drive compliance across event logging maturity levels. The workbook serves as a starting point for designing and reporting event log management capabilities by providing visibility into current posture mapped against the four maturity tiers.
I'm Being Attacked, Now What?
Learn how to respond to attacks with the Microsoft Sentinel: Threat Analysis & Response Solution while applying SOAR remediations and implementing NIST SP 800-53 Controls with Microsoft Defender for Cloud. Evolve from reactive to proactive threat hunting with the MITRE ATT&CK blade. Check out the demo and get started today.
What’s new: Closer integration between Microsoft Sentinel and Microsoft 365 Defender
Over a year ago, we first announced the integration between Microsoft Sentinel and Microsoft 365 Defender as part of the Microsoft SIEM and XDR story. Combining the breadth of a SIEM with the depth of XDR to give security professionals the integrated toolset they need to fight against attacks that take advantage of today’s diverse, distributed, and complex environments. Today, we are happy to share several new preview updates
Integrating SIEM + XDR: Azure Sentinel and Azure Defender bi-directional incident sync
Azure Sentinel and Azure Defender now share the concept of an incident. This new capability enables you to update or close Azure Defender incidents from within Azure Sentinel and complements a similar feature already available for Microsoft 365 Defender incidents and Azure SentinelPlaybooks & Watchlists Part 1: Inform the subscription owner
Leveraging the power of Azure Sentinel playbooks and new Watchlists feature, responding to ASC alerts becomes easier then ever. Define an Azure Sentinel Watchlist of subscription owners and their contact email address, and keep them informed about possible attacks on their subscriptions automatically.
