Introducing Microsoft Security Store
Security is being reengineered for the AI era—moving beyond static, rulebound controls and after-the-fact response toward platform-led, machine-speed defense. We recognize that defending against modern threats requires the full strength of an ecosystem, combining our unique expertise and shared threat intelligence. But with so many options out there, it’s tough for security professionals to cut through the noise, and even tougher to navigate long procurement cycles and stitch together tools and data before seeing meaningful improvements. That’s why we built Microsoft Security Store - a storefront designed for security professionals to discover, buy, and deploy security SaaS solutions and AI agents from our ecosystem partners such as Darktrace, Illumio, and BlueVoyant. Security SaaS solutions and AI agents on Security Store integrate with Microsoft Security products, including Sentinel platform, to enhance end-to-end protection. These integrated solutions and agents collaborate intelligently, sharing insights and leveraging AI to enhance critical security tasks like triage, threat hunting, and access management. In Security Store, you can: Buy with confidence – Explore solutions and agents that are validated to integrate with Microsoft Security products, so you know they’ll work in your environment. Listings are organized to make it easy for security professionals to find what’s relevant to their needs. For example, you can filter solutions based on how they integrate with your existing Microsoft Security products. You can also browse listings based on their NIST Cybersecurity Framework functions, covering everything from network security to compliance automation — helping you quickly identify which solutions strengthen the areas that matter most to your security posture. Simplify purchasing – Buy solutions and agents with your existing Microsoft billing account without any additional payment setup. For Azure benefit-eligible offers, eligible purchases contribute to your cloud consumption commitments. You can also purchase negotiated deals through private offers. Accelerate time to value – Deploy agents and their dependencies in just a few steps and start getting value from AI in minutes. Partners offer ready-to-use AI agents that can triage alerts at scale, analyze and retrieve investigation insights in real time, and surface posture and detection gaps with actionable recommendations. A rich ecosystem of solutions and AI agents to elevate security posture In Security Store, you’ll find solutions covering every corner of cybersecurity—threat protection, data security and governance, identity and device management, and more. To give you a flavor of what is available, here are some of the exciting solutions on the store: Darktrace’s ActiveAI Security SaaS solution integrates with Microsoft Security to extend self-learning AI across a customer's entire digital estate, helping detect anomalies and stop novel attacks before they spread. The Darktrace Email Analysis Agent helps SOC teams triage and threat hunt suspicious emails by automating detection of risky attachments, links, and user behaviors using Darktrace Self-Learning AI, integrated with Microsoft Defender and Security Copilot. This unified approach highlights anomalous properties and indicators of compromise, enabling proactive threat hunting and faster, more accurate response. Illumio for Microsoft Sentinel combines Illumio Insights with Microsoft Sentinel data lake and Security Copilot to enhance detection and response to cyber threats. It fuses data from Illumio and all the other sources feeding into Sentinel to deliver a unified view of threats across millions of workloads. AI-driven breach containment from Illumio gives SOC analysts, incident responders, and threat hunters unified visibility into lateral traffic threats and attack paths across hybrid and multi-cloud environments, to reduce alert fatigue, prioritize threat investigation, and instantly isolate workloads. Netskope’s Security Service Edge (SSE) platform integrates with Microsoft M365, Defender, Sentinel, Entra and Purview for identity-driven, label-aware protection across cloud, web, and private apps. Netskope's inline controls (SWG, CASB, ZTNA) and advanced DLP, with Entra signals and Conditional Access, provide real-time, context-rich policies based on user, device, and risk. Telemetry and incidents flow into Defender and Sentinel for automated enrichment and response, ensuring unified visibility, faster investigations, and consistent Zero Trust protection for cloud, data, and AI everywhere. PERFORMANTA Email Analysis Agent automates deep investigations into email threats, analyzing metadata (headers, indicators, attachments) against threat intelligence to expose phishing attempts. Complementing this, the IAM Supervisor Agent triages identity risks by scrutinizing user activity for signs of credential theft, privilege misuse, or unusual behavior. These agents deliver unified, evidence-backed reports directly to you, providing instant clarity and slashing incident response time. Tanium Autonomous Endpoint Management (AEM) pairs realtime endpoint visibility with AI-driven automation to keep IT environments healthy and secure at scale. Tanium is integrated with the Microsoft Security suite—including Microsoft Sentinel, Defender for Endpoint, Entra ID, Intune, and Security Copilot. Tanium streams current state telemetry into Microsoft’s security and AI platforms and lets analysts pivot from investigation to remediation without tool switching. Tanium even executes remediation actions from the Sentinel console. The Tanium Security Triage Agent accelerates alert triage, enabling security teams to make swift, informed decisions using Tanium Threat Response alerts and real-time endpoint data. Walkthrough of Microsoft Security Store Now that you’ve seen the types of solutions available in Security Store, let’s walk through how to find the right one for your organization. You can get started by going to the Microsoft Security Store portal. From there, you can search and browse solutions that integrate with Microsoft Security products, including a dedicated section for AI agents—all in one place. If you are using Microsoft Security Copilot, you can also open the store from within Security Copilot to find AI agents - read more here. Solutions are grouped by how they align with industry frameworks like NIST CSF 2.0, making it easier to see which areas of security each one supports. You can also filter by integration type—e.g., Defender, Sentinel, Entra, or Purview—and by compliance certifications to narrow results to what fits your environment. To explore a solution, click into its detail page to view descriptions, screenshots, integration details, and pricing. For AI agents, you’ll also see the tasks they perform, the inputs they require, and the outputs they produce —so you know what to expect before you deploy. Every listing goes through a review process that includes partner verification, security scans on code packages stored in a secure registry to protect against malware, and validation that integrations with Microsoft Security products work as intended. Customers with the right permissions can purchase agents and SaaS solutions directly through Security Store. The process is simple: choose a partner solution or AI agent and complete the purchase in just a few clicks using your existing Microsoft billing account—no new payment setup required. Qualifying SaaS purchases also count toward your Microsoft Azure Consumption Commitment (MACC), helping accelerate budget approvals while adding the security capabilities your organization needs. Security and IT admins can deploy solutions directly from Security Store in just a few steps through a guided experience. The deployment process automatically provisions the resources each solution needs—such as Security Copilot agents and Microsoft Sentinel data lake notebook jobs—so you don’t have to do so manually. Agents are deployed into Security Copilot, which is built with security in mind, providing controls like granular agent permissions and audit trails, giving admins visibility and governance. Once deployment is complete, your agent is ready to configure and use so you can start applying AI to expand detection coverage, respond faster, and improve operational efficiency. Security and IT admins can view and manage all purchased solutions from the “My Solutions” page and easily navigate to Microsoft Cost Management tools to track spending and manage subscriptions. Partners: grow your business with Microsoft For security partners, Security Store opens a powerful new channel to reach customers, monetize differentiated solutions, and grow with Microsoft. We will showcase select solutions across relevant Microsoft Security experiences, starting with Security Copilot, so your offerings appear in the right context for the right audience. You can monetize both SaaS solutions and AI agents through built-in commerce capabilities, while tapping into Microsoft’s go-to-market incentives. For agent builders, it’s even simpler—we handle the entire commerce lifecycle, including billing and entitlement, so you don’t have to build any infrastructure. You focus on embedding your security expertise into the agent, and we take care of the rest to deliver a seamless purchase experience for customers. Security Store is built on top of Microsoft Marketplace, which means partners publish their solution or agent through the Microsoft Partner Center - the central hub for managing all marketplace offers. From there, create or update your offer with details about how your solution integrates with Microsoft Security so customers can easily discover it in Security Store. Next, upload your deployable package to the Security Store registry, which is encrypted for protection. Then define your license model, terms, and pricing so customers know exactly what to expect. Before your offer goes live, it goes through certification checks that include malware and virus scans, schema validation, and solution validation. These steps help give customers confidence that your solutions meet Microsoft’s integration standards. Get started today By creating a storefront optimized for security professionals, we are making it simple to find, buy, and deploy solutions and AI agents that work together. Microsoft Security Store helps you put the right AI‑powered tools in place so your team can focus on what matters most—defending against attackers with speed and confidence. Get started today by visiting Microsoft Security Store. If you’re a partner looking to grow your business with Microsoft, start by visiting Microsoft Security Store - Partner with Microsoft to become a partner. Partners can list their solution or agent if their solution has a qualifying integration with Microsoft Security products, such as a Sentinel connector or Security Copilot agent, or another qualifying MISA solution integration. You can learn more about qualifying integrations and the listing process in our documentation here.Microsoft Sentinel data lake is now generally available
Security is being reengineered for the AI era, shifting from static controls to fast, platform-driven defense. Traditional tools, scattered data, and outdated systems struggle against modern threats. An AI-ready, data-first foundation is needed to unify telemetry, standardize agent access, and enable autonomous responses while ensuring humans are in command of strategy and high-impact investigations. Security teams already anchor their operations around SIEMs for comprehensive visibility. We're building on that foundation by evolving Microsoft Sentinel into both the SIEM and the platform for agentic defense—connecting analytics and context across ecosystems. Today, we’re introducing new platform capabilities that build on Sentinel data lake: Sentinel graph for deeper insight and context; an MCP server and tools to make data agent ready; new developer capabilities; and a Security Store for effortless discovery and deployment—so protection accelerates to machine speed while analysts do their best work. We’ve reached a major milestone in our journey to modernize security operations — Microsoft Sentinel data lake is now generally available. This fully managed, cloud-native data lake is redefining how security teams manage, analyze, and act on their data cost-effectively. Since its introduction, organizations across sectors are embracing Sentinel data lake for its transformative impact on security operations. Customers consistently highlight its ability to unify security data from diverse sources, enabling enhanced threat detection and investigation. Many cite cost efficiency as a key benefit, with tiered storage and flexible retention, helping reduce costs. With petabytes of data already ingested, users are gaining real-time and historical insights at scale. "With Microsoft Sentinel data lake integration, we now have a scalable and cost-efficient solution for retaining Microsoft Sentinel data for long-term retention. This empowers our security and compliance teams with seamless access to historical telemetry data right within the data lake explorer and Jupyter notebooks - enabling advanced threat hunting, forensic analysis, and AI-powered insights at scale" Farhan Nadeem, Senior Security Engineer Government of Nunavut Industry partners also commend its role in modernizing SOC workflows and accelerating AI-driven analytics. “Microsoft Sentinel data lake amplifies BlueVoyant’s ability to transform security operations into a mature, intelligence-driven discipline. It preserves institutional memory across years of telemetry, which empowers advanced threat hunting strategies that evolve with time. Security teams can validate which data sources yield actionable insights, uncover persistent attack patterns, and retain high-value indicators that support long-term strategic advantage.” Milan Patel, CRO BlueVoyant Microsoft Sentinel data lake use-cases There are many powerful ways customers are unlocking value with Sentinel data lake—here are just a few impactful examples.: Threat investigations over extended timelines: Security analysts query data older than 90 days to uncover slow-moving attacks—like brute-force and password spray campaigns—that span accounts and geographies. Behavioral baselining for deeper insights: SOC engineers build time-series models using months of sign-in logs to establish a standard of normal behavior and identify unusual patterns, such as credential abuse or lateral movement. Alert enrichment: SOC teams correlate alerts with Firewall and Netflow data, often stored only in the data lake, reducing false positives and increasing alert accuracy. Retrospective threat hunting with new indicators of compromise (IOCs): Threat intelligence teams react to emerging IOCs by running historical queries across the data lake, enabling rapid and informed response. ML-Powered insights: SOC engineers use Spark Notebooks to build and operationalize custom machine learning models for anomaly detection, alert enrichment, and predictive analytics. The Sentinel data lake is more than a storage solution—it’s the foundation for modern, AI-powered security operations. Whether you're scaling your SOC, building deeper analytics, or preparing for future threats, the Sentinel data lake is ready to support your journey. What’s new Regional expansion In light of strong customer demand in public preview, at GA we are expanding Sentinel data lake availability to additional regions. These new regions will roll out progressively over the coming weeks. For more information, see documentation. Flexible data ingestion and management With over 350 native connectors, SOC teams can seamlessly ingest both structured and semi-structured data at scale. Data is automatically mirrored from the analytics tier to the data lake tier, at no additional cost, ensuring a single, unified copy is available for diverse use cases across security operations. Since the public preview of Microsoft Sentinel data lake, we've launched 45 new connectors built on the scalable and performant Codeless Connector Framework (CCF), including connectors for: GCP: SQL, DNS, VPC Flow, Resource Manager, IAM, Apigee AWS: Security Hub findings, Route53 DNS Others: Alibaba Cloud, Oracle, Salesforce, Snowflake, Cisco Sentinel’s connector ecosystem is designed to help security teams seamlessly unify signals across hybrid environments, without the need for heavy engineering effort. Explore the full list of connectors in our documentation here. App Assure Microsoft Sentinel data lake promise As part of our commitment to customer success, we are expanding the App Assure Microsoft Sentinel promise to Sentinel data lake. This means customers can confidently onboard their data, knowing that App Assure stands ready to help resolve connector issues such as replacing deprecated APIs with updated ones, and accelerating new integrations. Whether you're working with existing Independent Software Vendor (ISV) solutions or building new ones, App Assure will collaborate directly with ISVs to ensure seamless data ingestion into the lake. This promise reinforces our dedication to delivering reliable, scalable, and secure security operations, backed by engineering support and a thriving partner ecosystem. Cost management and billing We are introducing new cost management features in public preview to help customers with cost predictability, billing transparency, and operational efficiency. Customers can set usage-based alerts on specific meters to monitor and control costs. For example, you can receive alerts when query or notebook usage passes set limits, helping avoid unexpected expenses and manage budgets. In-product reports provide customers with insights into usage trends over time, enabling them to identify cost drivers and optimize data retention and processing strategies. To support the ingestion and standardization of diverse data sources, we are introducing a new Data Processing feature that applies a $0.10 per GB charge for all data as it is ingested into the data lake. This feature enables a broad array of transformations like redaction, splitting, filtering and normalizing data. This feature was not billed during public preview but will be chargeable at GA starting October 1,2025. Data lake ingestion charges of $0.05 per GB will apply to Entra asset data; starting October 1, 2025. This was previously not billed during public preview. Retaining security data to perform deep analytics and investigations is crucial for defending against threats. To help enable customers to retain all their security data for extended periods cost effectively, data lake storage, including asset data storage, is now billed with a simple and uniform data compression rate of 6:1 across all data sources. Please refer to Plan costs and understand Microsoft Sentinel pricing and billing article for more information. For detailed prerequisites and instructions on configuring and managing asset connectors, refer to the official documentation: Asset data in Microsoft Sentinel data lake. KQL and Notebook enhancements We are introducing several enhancements to our data lake analytics capabilities with an upgraded KQL and notebook experience. Security teams can now run multi-workspace KQL queries for broader threat correlation and schedule KQL jobs more frequently. Frequent KQL jobs enable SOC teams to automate historical threat intelligence matching, summarize alert trends, and aggregate signals across workspaces. For example, schedule recurring jobs to scan for matches against newly ingested IOCs, helping uncover threats that were previously undetected and strengthening threat hunting and investigation workflows. The enhanced Jobs page offers operational clarity for SOC teams with a comprehensive view into job health and activity. At the top, a summary dashboard provides instant visibility into key metrics, total jobs, completions, and failures, helping teams quickly assess job health. A filterable list view displays essential details such as job names, status, frequency, and last run information, enabling quick prioritization and triage. For more detailed diagnostics, users can view individual jobs to access job runs telemetry such as job run duration, row count, and additional historical execution trends, providing additional visibility. Notebooks are receiving a significant upgrade, offering streamlined user experience for querying the data lake. Users now benefit from IntelliSense support for syntax and table names, making query authoring faster and more intuitive. They can also configure custom compute session timeouts and warning windows to better manage resources. Scheduling notebooks as jobs is now simpler, and users can leverage GitHub Copilot for intelligent assistance throughout the process. Together, these KQL and notebook improvements deliver deeper, more customizable analytics, helping customers unlock richer insights, accelerate threat response, and scale securely across diverse environments. Powering agentic defense Data centralization powers AI agents and automation to access comprehensive, historical, and real-time data for advanced analytics, anomaly detection, and autonomous threat response. Support for tools like KQL queries, Spark notebooks, and machine learning models in the data lake, allows agentic systems to continuously learn, adapt, and act on emerging threats. Integration with Security Copilot and MCP Server further enhances agentic defense, enabling smarter, faster, and context-rich security operations—all built on the foundation of Sentinel’s unified data lake. Microsoft Sentinel 50 GB commitment tier promotional pricing To make Microsoft Sentinel more accessible to small and mid-sized customers, we are introducing a new 50 GB commitment tier in public preview, with promotional pricing offered from October 1, 2025, to March 31, 2026. Customers who choose the 50 GB commitment tier during this period will maintain their promotional rate until March 31, 2027. This offer is available in all regions where Microsoft Sentinel is sold, with regional variations in promotional pricing. It is accessible through EA, CSP, and Direct channels. The new 50 GB commitment tier details will be available starting October 1, 2025, on the Microsoft Sentinel pricing page. Thank you to our customers and partners We’re incredibly grateful for the continued partnership and collaboration from our customers and partners throughout this journey. Your feedback and trust have been instrumental in shaping Microsoft Sentinel data lake into what it is today. Thank you for being a part of this critical milestone—we’re excited to keep building together. Get started today By centralizing data, optimizing costs, expanding coverage, and enabling deep analytics, Microsoft Sentinel empowers security teams to operate smarter, faster, and more effectively. Get started with Microsoft Sentinel data lake today in the Microsoft Defender experience. To learn more, see: Microsoft Sentinel—AI-Powered Cloud SIEM & Platform Pricing: Pricing page, Plan costs and understand Microsoft Sentinel pricing and billing Documentation: Connect Sentinel to Defender, Jupyter notebooks in Microsoft Sentinel data lake, KQL and the Microsoft Sentinel data lake, Permissions for Microsoft Sentinel data lake, Manage data tiers and retention in Microsoft Defender experience Blogs: Sentinel data lake FAQ blog, Empowering defenders in the era of AI, Microsoft Sentinel graph announcement, App Assure Microsoft Sentinel data lake promiseIntroducing developer solutions for Microsoft Sentinel platform
Security is being reengineered for the AI era, moving beyond static, rule-bound controls and toward after-the-fact response toward platform-led, machine-speed defense. The challenge is clear: fragmented tools, sprawling signals, and legacy architectures that can’t match the velocity and scale of modern attacks. What’s needed is an AI-ready, data-first foundation - one that turns telemetry into a security graph, standardizes access for agents, and coordinates autonomous actions while keeping humans in command of strategy and high-impact investigations. Security teams already center operations on their SIEM for end-to-end visibility, and we’re advancing that foundation by evolving Microsoft Sentinel into both the SIEM and the platform for agentic defense—connecting analytics and context across ecosystems. And today, we’re introducing new platform capabilities that build on Sentinel data lake: Sentinel graph for deeper insight and context; Sentinel MCP server and tools to make data agent ready; new developer capabilities; and Security Store for effortless discovery and deployment—so protection accelerates to machine speed while analysts do their best work. Today, customers use a breadth of solutions to keep themselves secure. Each solution typically ingests, processes, and stores the security data it needs which means applications maintain identical copies of the same underlying data. This is painful for both customers and partners, who don’t want to build and maintain duplicate infrastructure and create data silos that make it difficult to counter sophisticated attacks. With today’s announcement, we’re directly addressing those challenges by giving partners the ability to create solutions that can reason over the single copy of the security data that each customer has in their Sentinel data lake instance. Partners can create AI solutions that use Sentinel and Security Copilot and distribute them in Microsoft Security Store to reach audiences, grow their revenue, and keep their customers safe. Sentinel already has a rich partner ecosystem with hundreds of SIEM solutions that include connectors, playbooks, and other content types. These new platform capabilities extend those solutions, creating opportunities for partners to address new scenarios and bring those solutions to market quickly since they don’t need to build complex data pipelines or store and process new data sets in their own infrastructure. For example, partners can use Sentinel connectors to bring their own data into the Sentinel data lake. They can create Jupyter notebook jobs in the updated Sentinel Visual Studio Code extension to analyze that data or take advantage of the new Model Context Protocol (MCP) server which makes the data understandable and accessible to AI agents in Security Copilot. With Security Copilot’s new vibe-coding capabilities, partners can create their agent in the same Sentinel Visual Studio Code extension or the environment of their choice. The solution can then be packaged and published to the new Microsoft Security Store, which gives partners an opportunity to expand their audience and grow their revenue while protecting more customers across the ecosystem. These capabilities are being embraced across our ecosystem by mature and emerging partners alike. Services partners such as Accenture and ISVs such as Zscaler and ServiceNow are already creating solutions that leverage the capabilities of the Sentinel platform. Partners have already brought several solutions to market using the integrated capabilities of the Sentinel platform: Illumio. Illumio for Microsoft Sentinel combines Illumio Insights with Microsoft Sentinel data lake and Security Copilot to revolutionize detection and response to cyber threats. It fuses data from Illumio and all the other sources feeding into Sentinel to deliver a unified view of threats, giving SOC analysts, incident responders, and threat hunters visibility and AI-driven breach containment capabilities for lateral traffic threats and attack paths across hybrid and multi-cloud environments. To learn more, visit Illumio for Microsoft Sentinel. OneTrust. OneTrust’s AI-ready governance platform enables 14,000 customers globally – including over half of the Fortune 500 – to accelerate innovation while ensuring responsible data use. Privacy and risk teams know that undiscovered personal data in their digital estate puts their business and customers at risk. OneTrust’s Privacy Risk Agent uses Security Copilot, Purview scan logs, Entra ID data, and Jupyter notebook jobs in the Sentinel data lake to automatically discover personal data, assess risk, and take mitigating actions. To learn more, visit here. Tanium. The Tanium Security Triage Agent accelerates alert triage using real-time endpoint intelligence from Tanium. Tanium intends to expand its agent to ingest contextual identity data from Microsoft Entra using Sentinel data lake. Discover how Tanium’s integrations empower IT and security teams to make faster, more informed decisions. Simbian. Simbian’s Threat Hunt Agent makes hunters more effective by automating the process of validating threat hunt hypotheses with AI. Threat hunters provide a hypothesis in natural language, and the Agent queries and analyzes the full breadth of data available in Sentinel data lake to validate the hypothesis and do deep investigation. Simbian's AI SOC Agent investigates and responds to security alerts from Sentinel, Defender, and other alert sources and also uses Sentinel data lake to enhance the depth of investigations. Learn more here. Lumen. Lumen’s Defender℠ Threat Feed for Microsoft Sentinel helps customers correlate known-bad artifacts with activity in their environment. Lumen’s Black Lotus Labs® harnesses unmatched network visibility and machine intelligence to produce high-confidence indicators that can be operationalized at scale for detection and investigation. Currently Lumen’s Defender℠ Threat Feed for Microsoft Sentinel is available as an invite only preview. To request an invite, reach out to the Lumen Defender Threat Feed Sales team. The updated Sentinel Visual Studio Code extension for Microsoft Sentinel The Sentinel Extension for Visual Studio code brings new AI and packaging capabilities on top of existing Jupyter notebook jobs to help developers efficiently create new solutions. Building with AI Impactful AI security solutions need access and understanding of relevant security data to address a scenario. The new Microsoft Sentinel Model Context Protocol (MCP) server makes data in Sentinel data lake AI-discoverable and understandable to agents so they can reason over it to generate powerful new insights. It integrates with the Sentinel VS Code extension so developers can use those tools to explore the data in the lake and have agents use those tools as they do their work. To learn more, read the Microsoft Sentinel MCP server announcement. Microsoft is also releasing MCP tools to make creating AI agents more straightforward. Developers can use Security Copilot’s MCP tools to create agents within either the Sentinel VS Code extension or the environment of their choice. They can also take advantage of the low code agent authoring experience right in the Security Copilot portal. To learn more about the Security Copilot pro code and low code agent authoring experiences visit the Security Copilot blog post on Building your own Security Copilot agents. Jupyter Notebook Jobs Jupyter notebooks jobs are an important part of the Sentinel data lake and were launched at our public preview a couple of months ago. See the documentation here for more details on Jupyter notebooks jobs and how they can be used in a solution. Note that when jobs write to the data lake, agents can use the Sentinel MCP tools to read and act on those results in the same way they’re able to read any data in the data lake. Packaging and Publishing Developers can now package solutions containing notebook jobs and Copilot agents so they can be distributed through the new Microsoft Security Store. With just a few clicks in the Sentinel VS Code extension, a developer can create a package which they can then upload to Security Store. Distribution and revenue opportunities with Security Store Sentinel platform solutions can be packaged and offered through the new Microsoft Security Store, which gives partners new ways to grow revenue and reach customers. Learn more about the ways Microsoft Security Store can help developers reach customers and grow revenue by visiting securitystore.microsoft.com. Getting started Developers can get started building powerful applications that bring together Sentinel data, Jupyter notebook jobs, and Security Copilot today: Become a partner to publish solutions to Microsoft Security Store Onboarding to Sentinel data lake Downloading the Sentinel Visual Studio Code extension Learn about Security Copilot news Learn about Microsoft Security Store
Announcing Microsoft Sentinel Model Context Protocol (MCP) server – Public Preview
Security is being reengineered for the AI era—moving beyond static, rulebound controls and after-the-fact response toward platform-led, machine-speed defense. The challenge is clear: fragmented tools, sprawling signals, and legacy architectures that can’t match the velocity and scale of modern attacks. What’s needed is an AI-ready, data-first foundation—one that turns telemetry into a security graph, standardizes access for agents, and coordinates autonomous actions while keeping humans in command of strategy and high-impact investigations. Security teams already center operations on their SIEM for end-to-end visibility, and we’re advancing that foundation by evolving Microsoft Sentinel into both the SIEM and the platform for agentic defense—connecting analytics and context across ecosystems. And now, we’re introducing new platform capabilities that build on Sentinel data lake: Sentinel graph for deeper insight and context; an MCP server and tools to make data agent ready; new developer capabilities; and Security Store for effortless discovery and deployment—so protection accelerates to machine speed while analysts do their best work. Introducing Sentinel MCP server We’re excited to announce the public preview of Microsoft Sentinel MCP (Model Context Protocol) server, a fully managed cloud service built on an open standard that lets AI agents seamlessly access the rich security context in your Sentinel data lake. Recent advances in large language models have enabled AI agents to perform reasoning—breaking down complex tasks, inferring patterns, and planning multistep actions, making them capable of autonomously performing business processes. To unlock this potential in cybersecurity, agents must operate with your organization’s real security context, not just public training data. Sentinel MCP server solves that by providing standardized, secure access to that context—across graph relationships, tabular telemetry, and vector embeddings—via reusable, natural language tools, enabling security teams to unlock the full potential of AI-driven automation and focus on what matters most. Why Model Context Protocol (MCP)? Model Context Protocol (MCP) is a rapidly growing open standard that allows AI models to securely communicate with external applications, services, and data sources through a well-structured interface. Think of MCP as a bridge that lets an AI agents understand and invoke an application’s capabilities. These capabilities are exposed as discrete “tools” with natural language inputs and outputs. The AI agent can autonomously choose the right tool (or combination of tools) for the task it needs to accomplish. In simpler terms, MCP standardizes how an AI talks to systems. Instead of developers writing custom connectors for each application, the MCP server presents a menu of available actions to the AI in a language it understands. This means an AI agent can discover what it can do (search data, run queries, trigger actions, etc.) and then execute those actions safely and intelligently. By adopting an open standard like MCP, Microsoft is ensuring that our AI integrations are interoperable and future-proof. Any AI application that speaks MCP can connect. Security Copilot offers built-in integration, while other MCP-compatible platforms can leverage your Sentinel data and services can quickly connect by simply adding a new MCP server and typing Sentinel’s MCP server URL. How to Get Started Sentinel MCP server is a fully managed service now available to all Sentinel data lake customers. If you are already onboarded to Sentinel data lake, you are ready to begin using MCP. Not using Sentinel data lake yet? Learn more here. Currently, you can connect to the Sentinel MCP server using Visual Studio Code (VS Code) with the GitHub Copilot add-on. Here’s a step-by-step guide: Open VS Code and authenticate with an account that has at least Security Reader role access (required to query the data lake via the Sentinel MCP server) Open the Command Palette in VS Code (Ctrl + Shift + P) Type or select “MCP: Add Server…” Choose “HTTP” (HTTP or Server-Sent Event) Enter the Sentinel MCP server URL: “https://sentinel.microsoft.com/mcp/data-exploration" When prompted, allow authentication with the Sentinel MCP server by clicking “Allow” Once connected, GitHub Copilot will be linked to Sentinel MCP server. Open the agent pane, set it to Agent mode (Ctrl + Shift + I), and you are ready to go. GitHub Copilot will autonomously identify and utilize Sentinel MCP tools as necessary. You can now experience how AI agents powered by the Sentinel MCP server access and utilize your security context using natural language, without knowing KQL, which tables to query and wrangle complex schemas. Try prompts like: “Find the top 3 users that are at risk and explain why they are at risk.” “Find sign-in failures in the last 24 hours and give me a brief summary of key findings.” “Identify devices that showed an outstanding amount of outgoing network connections.” To learn more about the existing capabilities of Sentinel MCP tools, refer to our documentation. Security Copilot will also feature native integration with Sentinel MCP server; this seamless connectivity will enhance autonomous security agents and the open prompting experience. Check out the Security Copilot blog for additional details. What’s coming next? The public preview of Sentinel MCP server marks the beginning of a new era in cybersecurity—one where AI agents operate with full context, precision, and autonomy. In the coming months, the MCP toolset will expand to support natural language access across tabular, graph, and embedding-based data, enabling agents to reason over both structured and unstructured signals. This evolution will dramatically boost agentic performance, allowing them to handle more complex tasks, surface deeper insights, and accelerate protection to machine speed. As we continue to build out this ecosystem, your feedback will be essential in shaping its future. Be sure to check out the Microsoft Secure for a deeper dive and live demo of Sentinel MCP server in action.Secure Model Context Protocol (MCP) Implementation with Azure and Local Servers
Introduction The Model Context Protocol (MCP) enables AI systems to interact with external data sources and tools through a standardized interface. While powerful, MCP can introduce security risks in enterprise environments. This tutorial shows you how to implement MCP securely using local servers, Azure OpenAI with APIM, and proper authentication. Understanding MCP's Security Risks There are a couple of key security concerns to consider before implementing MCP: Data Exfiltration: External MCP servers could expose sensitive data. Unauthorized Access: Third-party services become potential security risks. Loss of Control: Unknown how external services handle your data. Compliance Issues: Difficulty meeting regulatory requirements with external dependencies. The solution? Keep everything local and controlled. Secure Architecture Before we dive into implementation, let's take a look at the overall architecture of our secure MCP solution: This architecture consists of three key components working together: Local MCP Server - Your custom tools run entirely within your local environment, reducing external exposure risks. Azure OpenAI + APIM Gateway - All AI requests are routed through Azure API Management with Microsoft Entra ID authentication, providing enterprise-grade security controls and compliance. Authenticated Proxy - A lightweight proxy service handles token management and request forwarding, ensuring seamless integration. One of the key benefits of this architecture is that no API key is required. Traditional implementations often require storing OpenAI API keys in configuration files, environment variables, or secrets management systems, creating potential security vulnerabilities. This approach uses Azure Managed Identity for backend authentication and Azure CLI credentials for client authentication, meaning no sensitive API keys are ever stored, logged, or exposed in your codebase. For more security, APIM and Azure OpenAI resources can be configured with IP restrictions or network rules to only accept traffic from certain sources. These configurations are available for most Azure resources and provide an additional layer of network-level security. This security-forward approach gives you the full power of MCP's tool integration capabilities while keeping your implementation completely under your control. How to Implement MCP Securely 1. Local MCP Server Implementation Building the MCP Server Let's start by creating a simple MCP server in .NET Core. 1. Create a web application dotnet new web -n McpServer 2.Add MCP packages dotnet add package ModelContextProtocol --prerelease dotnet add package ModelContextProtocol.AspNetCore --prerelease 3. Configure Program.cs var builder = WebApplication.CreateBuilder(args); builder.Services.AddMcpServer() .WithHttpTransport() .WithToolsFromAssembly(); var app = builder.Build(); app.MapMcp(); app.Run(); WithToolsFromAssembly() automatically discovers and registers tools from the current assembly. Look into the C# SDK for other ways to register tools for your use case. 4. Define Tools Now, we can define some tools that our MCP server can expose. here is a simple example for tools that echo input back to the client: using ModelContextProtocol.Server; using System.ComponentModel; namespace Tools; [McpServerToolType] public static class EchoTool { [McpServerTool] [Description("Echoes the input text back to the client in all capital letters.")] public static string EchoCaps(string input) { return new string(input.ToUpperInvariant()); } [McpServerTool] [Description("Echoes the input text back to the client in reverse.")] public static string ReverseEcho(string input) { return new string(input.Reverse().ToArray()); } } Key components of MCP tools are the McpServerToolType class decorator indicating that this class contains MCP tools, and the McpServerTool method decorator with a description that explains what the tool does. Alternative: STDIO Transport If you want to use STDIO transport instead of SSE (implemented here), check out this guide: Build a Model Context Protocol (MCP) Server in C# 2. Create a MCP Client with Cline Now that we have our MCP server set up with tools, we need a client that can discover and invoke these tools. For this implementation, we'll use Cline as our MCP client, configured to work through our secure Azure infrastructure. 1. Install Cline VS Code Extension Install the Cline extension in VS Code. 2. Deploy secure Azure OpenAI Endpoint with APIM Instead of connecting Cline directly to external AI services (which could expose the secure implementation to external bad actors), we will route through Azure API Management (APIM) for enterprise security. With this implementation, all requests go through Microsoft Entra ID and we use managed identity for all authentications. Quick Setup: Deploy the Azure OpenAI with APIM solution. Ensure your Azure OpenAI resources are configured to allow your APIM's managed identity to make calls. The APIM policy below uses managed identity authentication to connect to Azure OpenAI backends. Refer to the Azure OpenAI documentation on managed identity authentication for detailed setup instructions. 3. Configure APIM Policy After deploying APIM, configure the following policy to enable Azure AD token validation, managed identity authentication, and load balancing across multiple OpenAI backends: <!-- Azure API Management Policy for OpenAI Endpoint --> <!-- Implements Azure AD Token validation, managed identity authentication --> <!-- Supports round-robin load balancing across multiple OpenAI backends --> <!-- Requests with 'gpt-5' in the URL are routed to a single backend --> <!-- The client application ID '04b07795-8ddb-461a-bbee-02f9e1bf7b46' is the official Azure CLI app registration --> <!-- This policy allows requests authenticated by Azure CLI (az login) when the required claims are present --> <policies> <inbound> <!-- IP Allow List Fragment (external fragment for client IP restrictions) --> <include-fragment fragment-id="YourCompany-IPAllowList" /> <!-- Azure AD Token Validation for Azure CLI app ID --> <validate-azure-ad-token tenant-id="YOUR-TENANT-ID-HERE" header-name="Authorization" failed-validation-httpcode="401" failed-validation-error-message="Unauthorized. Access token is missing or invalid."> <client-application-ids> <application-id>04b07795-8ddb-461a-bbee-02f9e1bf7b46</application-id> </client-application-ids> <audiences> <audience>api://YOUR-API-AUDIENCE-ID-HERE</audience> </audiences> <required-claims> <claim name="roles" match="any"> <value>YourApp.User</value> </claim> </required-claims> </validate-azure-ad-token> <!-- Acquire Managed Identity access token for backend authentication --> <authentication-managed-identity resource="https://cognitiveservices.azure.com" output-token-variable-name="managed-id-access-token" ignore-error="false" /> <!-- Set Authorization header for backend using the managed identity token --> <set-header name="Authorization" exists-action="override"> <value>@("Bearer " + (string)context.Variables["managed-id-access-token"])</value> </set-header> <!-- Check if URL contains 'gpt-5' and set backend accordingly --> <choose> <when condition="@(context.Request.Url.Path.ToLower().Contains("gpt-5"))"> <set-variable name="selected-backend-url" value="https://your-region1-oai.openai.azure.com/openai" /> </when> <otherwise> <cache-lookup-value key="backend-counter" variable-name="backend-counter" /> <choose> <when condition="@(context.Variables.ContainsKey("backend-counter") == false)"> <set-variable name="backend-counter" value="@(0)" /> </when> </choose> <set-variable name="current-backend-index" value="@((int)context.Variables["backend-counter"] % 7)" /> <choose> <when condition="@((int)context.Variables["current-backend-index"] == 0)"> <set-variable name="selected-backend-url" value="https://your-region1-oai.openai.azure.com/openai" /> </when> <when condition="@((int)context.Variables["current-backend-index"] == 1)"> <set-variable name="selected-backend-url" value="https://your-region2-oai.openai.azure.com/openai" /> </when> <when condition="@((int)context.Variables["current-backend-index"] == 2)"> <set-variable name="selected-backend-url" value="https://your-region3-oai.openai.azure.com/openai" /> </when> <when condition="@((int)context.Variables["current-backend-index"] == 3)"> <set-variable name="selected-backend-url" value="https://your-region4-oai.openai.azure.com/openai" /> </when> <when condition="@((int)context.Variables["current-backend-index"] == 4)"> <set-variable name="selected-backend-url" value="https://your-region5-oai.openai.azure.com/openai" /> </when> <when condition="@((int)context.Variables["current-backend-index"] == 5)"> <set-variable name="selected-backend-url" value="https://your-region6-oai.openai.azure.com/openai" /> </when> <when condition="@((int)context.Variables["current-backend-index"] == 6)"> <set-variable name="selected-backend-url" value="https://your-region7-oai.openai.azure.com/openai" /> </when> </choose> <set-variable name="next-counter" value="@(((int)context.Variables["backend-counter"] + 1) % 1000)" /> <cache-store-value key="backend-counter" value="@((int)context.Variables["next-counter"])" duration="300" /> </otherwise> </choose> <!-- Always set backend service using selected-backend-url variable --> <set-backend-service base-url="@((string)context.Variables["selected-backend-url"])" /> <!-- Inherit any base policies defined outside this section --> <base /> </inbound> <backend> <base /> </backend> <outbound> <base /> </outbound> <on-error> <base /> </on-error> </policies> This policy creates a secure gateway that validates Azure AD tokens from your local Azure CLI session, then uses APIM's managed identity to authenticate with Azure OpenAI backends, eliminating the need for API keys. It automatically load-balances requests across multiple Azure OpenAI regions using round-robin distribution for optimal performance. 4. Create Azure APIM proxy for Cline This FastAPI-based proxy forwards OpenAI-compatible API requests from Cline through APIM using Azure AD authentication via Azure CLI credentials, eliminating the need to store or manage OpenAI API keys. Prerequisites: Python 3.8 or higher Azure CLI (ensure az login has been run at least once) Ensure the user running the proxy script has appropriate Azure AD roles and permissions. This script uses Azure CLI credentials to obtain bearer tokens. Your user account must have the correct roles assigned and access to the target API audience configured in the APIM policy above. Quick setup for the proxy: Create this requirements.txt: fastapi uvicorn requests azure-identity Create this Python script for the proxy source code azure_proxy.py: import os import requests from fastapi import FastAPI, Request from fastapi.responses import StreamingResponse import uvicorn from azure.identity import AzureCliCredential # CONFIGURATION APIM_BASE_URL = <APIM BASE URL HERE> AZURE_SCOPE = <AZURE SCOPE HERE> PORT = int(os.environ.get("PORT", 8080)) app = FastAPI() credential = AzureCliCredential() # Use a single requests.Session for connection pooling from requests.adapters import HTTPAdapter session = requests.Session() session.mount("https://", HTTPAdapter(pool_connections=100, pool_maxsize=100)) import time _cached_token = None _cached_expiry = 0 def get_bearer_token(scope: str) -> str: """Get an access token using AzureCliCredential, caching until expiry is within 30 seconds.""" global _cached_token, _cached_expiry now = int(time.time()) if _cached_token and (_cached_expiry - now > 30): return _cached_token try: token_obj = credential.get_token(scope) _cached_token = token_obj.token _cached_expiry = token_obj.expires_on return _cached_token except Exception as e: raise RuntimeError(f"Could not get Azure access token: {e}") @app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"]) async def proxy(request: Request, path: str): # Assemble the destination URL (preserve trailing slash logic) dest_url = f"{APIM_BASE_URL.rstrip('/')}/{path}".rstrip("/") if request.url.query: dest_url += "?" + request.url.query # Get the Bearer token bearer_token = get_bearer_token(AZURE_SCOPE) # Prepare headers (copy all, overwrite Authorization) headers = dict(request.headers) headers["Authorization"] = f"Bearer {bearer_token}" headers.pop("host", None) # Read body body = await request.body() # Send the request to APIM using the pooled session resp = session.request( method=request.method, url=dest_url, headers=headers, data=body if body else None, stream=True, ) # Stream the response back to the client return StreamingResponse( resp.raw, status_code=resp.status_code, headers={k: v for k, v in resp.headers.items() if k.lower() != "transfer-encoding"}, ) if __name__ == "__main__": # Bind the app to 127.0.0.1 to avoid any Firewall updates uvicorn.run(app, host="127.0.0.1", port=PORT) Run the setup: pip install -r requirements.txt az login # Authenticate with Azure python azure_proxy.py Configure Cline to use the proxy: Using the OpenAI Compatible API Provider: Base URL: http://localhost:8080 API Key: <any random string> Model ID: <your Azure OpenAI deployment name> API Version: <your Azure OpenAI deployment version> The API key field is required by Cline but unused in our implementation - any random string works since authentication happens via Azure AD. 5. Configure Cline to listen to your MCP Server Now that we have both our MCP server running and Cline configured with secure OpenAI access, the final step is connecting them together. To enable Cline to discover and use your custom tools, navigate to your installed MCP servers on Cline, select Configure MCP Servers, and add in the configuration for your server: { "mcpServers": { "mcp-tools": { "autoApprove": [ "EchoCaps", "ReverseEcho", ], "disabled": false, "timeout": 60, "type": "sse", "url": "http://<your localhost url>/sse" } } } Now, you can use Cline's chat interface to interact with your secure MCP tools. Try asking Cline to use your custom tools - for example, "Can you echo 'Hello World' in capital letters?" and watch as it calls your local MCP server through the infrastructure you've built. Conclusion There you have it: A secure implementation of MCP that can be tailored to your specific use case. This approach gives you the power of MCP while maintaining enterprise security. You get: AI capabilities through secure Azure infrastructure. Custom tools that never leave your environment. Standard MCP interface for easy integration. Complete control over your data and tools. The key is keeping MCP servers local while routing AI requests through your secure Azure infrastructure. This way, you gain MCP's benefits without compromising security. Disclaimer While this tutorial provides a secure foundation for MCP implementation, organizations are responsible for configuring their Azure resources according to their specific security requirements and compliance standards. Ensure proper review of network rules, access policies, and authentication configurations before deploying to production environments. Resources MCP SDKs and Tools: MCP C# SDK MCP Python SDK Cline SDK Cline User Guide Azure OpenAI with APIM Azure API Management Network Security: Azure API Management - restrict caller IPs Azure API Management with an Azure virtual network Set up inbound private endpoint for Azure API Management Azure OpenAI and AI Services Network Security: Configure Virtual Networks for Azure AI services Securing Azure OpenAI inside a virtual network with private endpoints Add an Azure OpenAI network security perimeter az cognitiveservices account network-ruleCybersecurity: What Every Business Leader Needs to Know Now
As a Senior Cybersecurity Solution Architect, I’ve had the privilege of supporting organisations across the United Kingdom, Europe, and the United States—spanning sectors from finance to healthcare—in strengthening their security posture. One thing has become abundantly clear: cybersecurity is no longer the sole domain of IT departments. It is a strategic imperative that demands attention at board-level. This guide distils five key lessons drawn from real-world engagements to help executive leaders navigate today’s evolving threat landscape. These insights are not merely technical—they are cultural, operational, and strategic. If you’re a C-level executive, this article is a call to action: reassess how your organisation approaches cybersecurity before the next breach forces the conversation. In this article, I share five lessons (and quotes) from the field that help demystify how to enhance an organisation’s security posture. 1. Shift the Mindset “This has always been our approach, and we’ve never experienced a breach—so why should we change it?” A significant barrier to effective cybersecurity lies not in the sophistication of attackers, but in the predictability of human behaviour. If you’ve never experienced a breach, it’s tempting to maintain the status quo. However, as threats evolve, so too must your defences. Many cyber threats exploit well-known vulnerabilities that remain unpatched or rely on individuals performing routine tasks in familiar ways. Human nature tends to favour comfort and habit—traits that adversaries are adept at exploiting. Unlike many organisations, attackers readily adopt new technologies to advance their objectives, including AI-powered ransomware to execute increasingly sophisticated attacks. It is therefore imperative to recognise—without delay—that the advent of AI has dramatically reduced both the effort and time required to compromise systems. As the UK’s National Cyber Security Centre (NCSC) has stated: “AI lowers the barrier for novice cyber criminals, hackers-for-hire and hacktivists to carry out effective access and information gathering operations. This enhanced access will likely contribute to the global ransomware threat over the next two years.” Similarly, McKinsey & Company observed: “As AI quickly advances cyber threats, organisations seem to be taking a more cautious approach, balancing the benefits and risks of the new technology while trying to keep pace with attackers’ increasing sophistication.” To counter this evolving threat landscape, organisations must proactively leverage AI in their cyber defence strategies. Examples include: Identity and Access Management (IAM): AI enhances IAM by analysing real-time signals across systems to detect risky sign-ins and enforce adaptive access controls. Example: Microsoft Entra Agents for Conditional Access use AI to automate policy recommendations, streamlining access decisions with minimal manual input. Figure 1: Microsoft Entra Agents Threat Detection: AI accelerates detection, response, and recovery, helping organisations stay ahead of sophisticated threats. Example: Microsoft Defender for Cloud’s AI threat protection identifies prompt injection, data poisoning, and wallet attacks in real time. Incident Response: AI facilitates real-time decision-making, removing emotional bias and accelerating containment and recovery during security incidents. Example: Automatic Attack Disruption in Defender XDR, which can automatically contain a breach in progress. AI Security Posture Management AI workloads require continuous discovery, classification, and protection across multi-cloud environments. Example: Microsoft Defender for Cloud’s AI Security Posture Management secures custom AI apps across Azure, AWS, and GCP by detecting misconfigurations, vulnerabilities, and compliance gaps. Data Security Posture Management (DSPM) for AI AI interactions must be governed to ensure privacy, compliance, and insider risk mitigation. Example: Microsoft Purview DSPM for AI enables prompt auditing, applies Data Loss Prevention (DLP) policies to third-party AI apps like ChatGPT, and supports eDiscovery and lifecycle management. AI Threat Protection Organisations must address emerging AI threat vectors, including prompt injection, data leakage, and model exploitation. Example: Defender for AI (private preview) provides model-level security, including governance, anomaly detection, and lifecycle protection. Embracing innovation, automation, and intelligent defence is the secret sauce for cyber resilience in 2026. 2. Avoid One-Off Purchases – Invest with a Strategy “One MDE and one Sentinel to go, please.” Organisations often approach me intending to purchase a specific cybersecurity product—such as Microsoft Defender for Endpoint (MDE)—without a clearly articulated strategic rationale. My immediate question is: what is the broader objective behind this purchase? Is it driven by perceived value or popularity, or does it form part of a well-considered strategy to enhance endpoint security? Cybersecurity investments should be guided by a long-term, holistic strategy that spans multiple years and is periodically reassessed to reflect evolving threats. Strengthening endpoint protection must be integrated into a wider effort to improve the organisation’s overall security posture. This includes ensuring seamless integration between security solutions and avoiding operational silos. For example, deploying robust endpoint protection is of limited value if identities are not safeguarded with multi-factor authentication (MFA), or if storage accounts remain publicly accessible. A cohesive and forward-looking approach ensures that all components of the security architecture work in concert to mitigate risk effectively. Security Adoption Journey (Based on Zero Trust Framework) Assess – Evaluate the threat landscape, attack surface, vulnerabilities, compliance obligations, and critical assets. Align – Link security objectives to broader business goals to ensure strategic coherence. Architect – Design integrated and scalable security solutions, addressing gaps and eliminating operational silos. Activate – Implement tools with robust governance and automation to ensure consistent policy enforcement. Advance – Continuously monitor, test, and refine the security posture to stay ahead of evolving threats. Security tools are not fast food—they work best as part of a long-term plan, not a one-off order. This piecemeal approach runs counter to the modern Zero Trust security model, which assumes no single tool will prevent every breach and instead implements layered defences and integration. 3. Legacy Systems Are Holding You Back “Unfortunately, we are unable to implement phishing-resistant MFA, as our legacy app does not support integration with the required protocols.” A common challenge faced by many organisations I have worked with is the constraint on innovation within their cybersecurity architecture, primarily due to continued reliance on legacy applications—often driven by budgetary or operational necessity. These outdated systems frequently lack compatibility with modern security technologies and may introduce significant vulnerabilities. A notable example is the deployment of phishing-resistant multi-factor authentication (MFA)—such as FIDO2 security keys or certificate-based authentication—which requires advanced identity protocols and conditional access policies. These capabilities are available exclusively through Microsoft Entra ID. To address this issue effectively, it is essential to design security frameworks based on the organisation’s future aspirations rather than its current limitations. By adopting a forward-thinking approach, organisations can remain receptive to emerging technologies that align with their strategic cybersecurity objectives. Moreover, this perspective encourages investment in acquiring the necessary talent, thereby reducing reliance on extensive change management and staff retraining. I advise designing for where you want to be in the next 1–3 years—ideally cloud-first and identity-driven—essentially adopting a Zero Trust architecture, rather than being constrained by the limitations of legacy systems. 4. Collaboration Is a Security Imperative “This item will need to be added to the dev team's backlog. Given their current workload, they will do their best to implement GitHub Security in Q3, subject to capacity.” Cybersecurity threats may originate from various parts of an organisation, and one of the principal challenges many face is the fragmented nature of their defence strategies. To effectively mitigate such risks, cybersecurity must be embedded across all departments and functions, rather than being confined to a single team or role. In many organisations, the Chief Information Security Officer (CISO) operates in isolation from other C-level executives, which can limit their influence and complicate the implementation of security measures across the enterprise. Furthermore, some teams may lack the requisite expertise to execute essential security practices. For instance, an R&D lead responsible for managing developers may not possess the necessary skills in DevSecOps. To address these challenges, it is vital to ensure that the CISO is empowered to act without political or organisational barriers and is supported in implementing security measures across all business units. When the CISO has backing from the COO and HR, initiatives such as MFA rollout happen faster and more thoroughly. Cross-Functional Security Responsibilities Role Security Responsibilities R&D - Adopt DevSecOps practices - Identify vulnerabilities early - Manage code dependencies - Detect exposed secrets - Embed security in CI/CD pipelines CIO - Ensure visibility over organizational data - Implement Data Loss Prevention (DLP) - Safeguard sensitive data lifecycle - Ensure regulatory compliance CTO - Secure cloud environments (CSPM) - Manage SaaS security posture (SSPM) - Ensure hardware and endpoint protection COO - Protect digital assets - Secure domain management - Mitigate impersonation threats - Safeguard digital marketing channels and customer PII Support & Vendors - Deliver targeted training - Prevent social engineering attacks - Improve awareness of threat vectors HR - Train employees on AI-related threats - Manage insider risks - Secure employee data - Oversee cybersecurity across the employee lifecycle Empowering the CISO to act across departments helps organisations shift towards a security-first culture—embedding cybersecurity into every function, not just IT. 5. Compliance Is Not Security “We’re compliant, so we must be secure.” Many organisations mistakenly equate passing audits—such as ISO 27001 or SOC 2—with being secure. While compliance frameworks help establish a baseline for security, they are not a guarantee of protection. Determined attackers are not deterred by audit checklists; they exploit gaps, misconfigurations, and human error regardless of whether an organisation is certified. Moreover, due to the rapidly evolving nature of the cyber threat landscape, compliance frameworks often struggle to keep pace. By the time a standard is updated, attackers may already be exploiting new techniques that fall outside its scope. This lag creates a false sense of security for organisations that rely solely on regulatory checkboxes. Security is a continuous risk management process—not a one-time certification. It must be embedded into every layer of the enterprise and treated with the same urgency as other core business priorities. Compliance may be the starting line, not the finish line. Effective security goes beyond meeting regulatory requirements—it demands ongoing vigilance, adaptability, and a proactive mindset. Conclusion: Cybersecurity Is a Continuous Discipline Cybersecurity is not a destination—it is a continuous journey. By embracing strategic thinking, cross-functional collaboration, and emerging technologies, organisations can build resilience against today’s threats and tomorrow’s unknowns. The lessons shared throughout this article are not merely technical—they are cultural, operational, and strategic. If there is one key takeaway, it is this: avoid piecemeal fixes and instead adopt an integrated, future-ready security strategy. Due to the rapidly evolving nature of the cyber threat landscape, compliance frameworks alone cannot keep pace. Security must be treated as a dynamic, ongoing process—one that is embedded into every layer of the enterprise and reviewed regularly. Organisations should conduct periodic security posture reviews, leveraging tools such as Microsoft Secure Score or monthly risk reports, and stay informed about emerging threats through threat intelligence feeds and resources like the Microsoft Digital Defence Report, CISA (Cybersecurity and Infrastructure Security Agency), NCSC (UK National Cyber Security Centre), and other open-source intelligence platforms. As Ann Johnson aptly stated in her blog: “The most prepared organisations are those that keep asking the right questions and refining their approach together.” Cyber resilience demands ongoing investment—in people (through training and simulation drills), in processes (via playbooks and frameworks), and in technology (through updates and adoption of AI-driven defences). To reduce cybersecurity risk over time, resilient organisations must continually refine their approach and treat cybersecurity as an ongoing discipline. The time to act is now. Resources: https://www.ncsc.gov.uk/report/impact-of-ai-on-cyber-threat Defend against cyber threats with AI solutions from Microsoft - Microsoft Industry Blogs Generative AI Cybersecurity Solutions | Microsoft Security Require phishing-resistant multifactor authentication for Microsoft Entra administrator roles - Microsoft Entra ID | Microsoft Learn AI is the greatest threat—and defense—in cybersecurity today. Here’s why. Microsoft Entra Agents - Microsoft Entra | Microsoft Learn Smarter identity security starts with AI https://www.microsoft.com/en-us/security/blog/2025/06/12/cyber-resilience-begins-before-the-crisis/ https://www.microsoft.com/en-us/security/security-insider/threat-landscape/microsoft-digital-defense-report-2023-critical-cybersecurity-challenges https://www.microsoft.com/en-us/security/blog/2025/06/12/cyber-resilience-begins-before-the-crisis/Graph RAG for Security: Insights from a Microsoft Intern
As a software engineering intern at Microsoft Security, I had the exciting opportunity to explore how Graph Retrieval-Augmented Generation (Graph RAG) can enhance data security investigations. This blog post shares my learning journey and insights from working with this evolving technology.Empowering Secure AI Innovation: Data Security and Compliance for AI Agents
As organizations embrace the transformative power of generative AI, agentic AI is quickly becoming a core part of enterprise innovation. Whether organizations are just beginning their AI journey or scaling advanced solutions, one thing is clear: agents are poised to transform every function and workflow across organizations. IDC predicts that over 1 billion new business process agents will be created in the next four years 1 . This surge in AI adoption is empowering employees across roles – from low-code makers to pro-code developers – to build and use AI in new ways. Business leaders are eager to support this momentum, but they also recognize the need to innovate responsibly with AI. Microsoft Purview’s evolution When Microsoft 365 Copilot launched in November 2022, it sparked a wave of excitement and an immediate question: how do we secure and govern the data powering these AI experiences? Microsoft Purview quickly evolved to meet this need, extending its data security and compliance capabilities to the Microsoft 365 Copilot ecosystem. It delivered discoverability, protection, and governance value that helped customers discover data risks such as data oversharing, protect sensitive data to prevent data loss and insider risks, and govern AI usage to meet regulations and policies. Now, as customers move beyond pre-built agents like Copilot to develop their own AI agents and applications, Microsoft Purview has evolved to extend the same data protections built for Microsoft 365 Copilot to AI agents. Today, those protections span the entire development spectrum—from no-code and low-code tools like Copilot Studio to pro-code environments such as Azure AI Foundry. Microsoft Purview helps address challenges across the development spectrum Makers – typically business users or citizen developers who build solutions using low-code or no-code tools – shouldn’t need to become security experts to build AI responsibly. Yet, without proper safeguards, these agents can inadvertently expose sensitive data or violate compliance policies. That is why with Microsoft Purview, security and IT teams can feel confident about the agents being built in their organizations. When makers build agents through the Agent Builder or directly in Copilot Studio, security admins can set up Microsoft Purview’s data security and compliance controls that work behind the scenes to support makers in building secure and compliant agents. These controls automatically enforce policies, monitor data access, and ensure compliance without requiring the maker to become a security expert without requiring makers to take additional actions. In fact, a recent Microsoft study found that 71% of developer decision-makers acknowledge that these constraints result in security trade-offs and development delays 2 . Pro-code developers are under increasing pressure to deliver fast, flexible, and seamlessly integrated solutions, yet data security often becomes a deployment blocker or an afterthought. Building enterprise-grade data security and compliance capabilities from scratch is not only time-consuming but also requires deep domain expertise. This is where Microsoft Purview steps in. As an industry leader in data security and compliance, Purview does the heavy lifting, so developers don’t have to. Now in preview, Purview SDK can be used by developers to embed robust, enterprise-ready data protections directly into their AI applications, instead of building complex security frameworks on their own. The Purview SDK is a comprehensive set of REST APIs, documentation, and code samples, allowing developers to easily incorporate Microsoft Purview’s capabilities into their workflows—regardless of their integrated development environment (IDE). This empowers them to move fast without compromising on security or compliance and at the same time, Microsoft Purview helps security teams remain in control. : By embedding Purview APIs into the IDE, developers help enable their AI apps to be secured and governed at runtime Startups, ISVs, and partners can leverage the Purview SDK to seamlessly integrate Purview’s industry-leading features into their AI agents and applications. This enables their offerings to become Purview-aware, empowering customers to more easily secure and govern data within their AI environments. For example, Christian Veillette, Chief Technology Officer at Arthur Health, a Quisitive customer, states “The synergistic integration of MazikCare, the Quisitive Intelligence Platform, and the data compliance power of Purview SDK, including its DSPM for AI, forms a foundational pillar for trustworthy and safe AI-driven healthcare transformations. This powerful combination ensures continuous oversight and instant enforcement of compliance policies, giving IT leadership full assurance in the output of every AI model and upholding the highest safety standards. By centralizing policy enforcement, security concerns are significantly eased, empowering leadership to confidently steer their organizations through the AI transformation journey.” Microsoft partner, Infotechtion, has also leveraged the new Purview SDK to embed Purview value into their GenAI initiatives. Vivek Bhatt, Infotechtion’s Chief Technology Officer says, “Embedding Purview SDK into Infotechtion's AI governance solution improved trust and security by aligning Gen-AI interactions with Microsoft Purview's enterprise policies.” Microsoft Purview also natively integrates with Azure AI Foundry, enabling seamless, built-in security and compliance for AI workloads without requiring additional development effort. With this integration, signals from Azure AI Foundry are automatically surfaced in Microsoft Purview’s Data Security Posture Management (DSPM) for AI, Insider Risk Management, and compliance solutions. This means security teams can monitor AI usage, detect data risks, and enforce compliance policies across AI agents and applications—whether they’re built in-house or with Azure AI Foundry models. This reinforces Microsoft’s commitment to delivering secure-by-default AI innovation—empowering organizations to scale responsibly with confidence. : Data security admins can now find data security and compliance insights across Microsoft Copilots, agents built with Agent Builder and Copilot Studio, and custom AI apps and agents in Microsoft Purview DSPM for AI. Explore more partner case studies from Ernst & Young and Infosys to see how they’re leveraging Purview SDK. Learn more about Purview SDK and Microsoft Purview for Azure AI Foundry. Unified visibility and control Whether supporting pro-code developers or low-code makers, Microsoft Purview enables organizations to secure and govern AI across organizations. With Purview, security teams can discover data security risks, protect sensitive data against data leakage and insider risks, and govern AI interactions. Discover data security risks With Data Security Posture Management (DSPM) for AI, data security teams can discover detailed data risk insights in AI interactions across Microsoft Copilots, agents built in Agent Builder and Copilot Studio, and custom AI apps and agents. Data security admins can now find data security and compliance insights across Microsoft Copilots, agents built with Agent Builder and Copilot Studio, and custom AI apps and agents all in Microsoft Purview DSPM for AI. Protect sensitive data against data leaks and insider risks In DSPM for AI, data security admins can also get recommended insights to improve their organization’s security posture like minimizing risks of data oversharing. For example, an admin might get a recommendation to set up a data loss prevention (DLP) policy that prevents agents in Microsoft 365 Copilot from using certain labeled documents as grounding data to generate summaries or responses. By setting up this policy, organizations can prevent confidential legal documents—with specific language that could lead to improper guidance—from being summarized. It also ensures that “Internal only” documents aren’t used to create content that might be shared outside the organization. Extend data loss prevention (DLP) policies to agents in Microsoft 365 to protect sensitive data. Agents often pull data from sources like SharePoint and Dataverse, and Microsoft Purview helps protect that data every step of the way. It honors sensitivity labels, enforces access permissions, and applies label inheritance so that AI-generated content carries the same protections as its source. With auto-labeling in Dataverse, sensitive data is classified as soon as it’s ingested—reducing manual effort and maintaining consistent protection. When responses draw from multiple sources with different labels, the most restrictive label is applied to uphold compliance and minimize risk. : Sensitivity labels will be automatically applied to data in Dataverse. : AI-generated responses will inherit and honor the source data’s sensitivity labels. In addition to data and permission controls that help address data oversharing or leakage, security teams also need ways to detect users' risky activities in AI apps and agents that could potentially lead to data security incidents. With risky AI usage indicators, policy template, and analytics report in Microsoft Purview Insider Risk Management, security teams with appropriate permissions can detect risky activities. For example, there could be a departing employee receiving an unusual number of AI responses across Copilots and agents containing sensitive data, deviating from their past activity patterns. Security teams can then effectively detect and respond to these potential incidents to minimize the negative impact. For example, they can configure Adaptive Protection to automatically block a high-risk user from accessing sensitive data. An Insider Risk Management alert from a Risky AI usage policy shows a user with anomalous activities. Govern AI Interactions to detect non-compliant usage Microsoft Purview provides a comprehensive set of tools to govern AI usage and detect non-compliant user activities. AI interactions across Microsoft Copilots, AI apps and agents, are recorded in Audit logs. eDiscovery enables legal and compliance teams with appropriate permissions to collect and review AI-generated content for internal investigations or litigation. Data Lifecycle Management enables teams to set policies to retain or dispose of AI interactions, while Communication Compliance helps detect risky or inappropriate use of AI, such as harmful content or other violations against code-of-conduct policies. Together, these capabilities give organizations the visibility and control they need to innovate responsibly with AI. AI interactions across Microsoft Copilots, AI apps and agents are recorded in Audit logs. AI interactions across Microsoft Copilots, AI apps and agents can be collected and reviewed in eDiscovery. Microsoft Purview Communication Compliance can detect non-compliant content in AI prompts across Microsoft Copilots, AI apps and agents. Securing the Future of AI Innovation — Explore Additional Resources As organizations accelerate their adoption of agentic AI, the need for built-in security and compliance has never been more critical. Microsoft Purview empowers both makers and developers to innovate with confidence—ensuring that every AI interaction is secure, compliant, and aligned with enterprise standards. By embedding protection across the entire development lifecycle, Purview helps organizations unlock the full potential of AI while maintaining the trust, transparency, and control that responsible innovation demands. To dive deeper into how Microsoft Purview supports secure AI development, explore our additional resources, documentation, and integration guides: Learn more about Security for AI solutions on our webpage Learn more about Microsoft Purview SDK Learn more about Purview pricing Get started with Azure AI Foundry Get started with Microsoft Purview 1 IDC, 1 Billion New Logical Applications: More Background, Gary Chen, Jim Mercer, April 2024 https://blogs.idc.com/2025/04/04/the-agentic-evolution-of-enterprise-applications/ 2 Microsoft, AI App Security Quantitative Study, April 2025Fishing for Syslog with Azure Kubernetes and Logstash
Deploy Secure Syslog Collection on Azure Kubernetes Service with Terraform Organizations managing distributed infrastructure face a common challenge: collecting syslog data securely and reliably from various sources. Whether you're aggregating logs from network devices, Linux servers, or applications, you need a solution that scales with your environment while maintaining security standards. This post walks through deploying Logstash on Azure Kubernetes Service (AKS) to collect RFC 5425 syslog messages over TLS. The solution uses Terraform for infrastructure automation and forwards collected logs to Azure Event Hubs for downstream processing. You'll learn how to build a production-ready deployment that integrates with Azure Sentinel, Azure Data Explorer, or other analytics platforms. Solution Architecture The deployment consists of several Azure components working together: Azure Kubernetes Service (AKS): Hosts the Logstash deployment with automatic scaling capabilities Internal Load Balancer: Provides a static IP endpoint for syslog sources within your network Azure Key Vault: Stores TLS certificates for secure syslog transmission Azure Event Hubs: Receives processed syslog data using the Kafka protocol Log Analytics Workspace: Monitors the AKS cluster health and performance Syslog sources send RFC 5425-compliant messages over TLS to the Load Balancer on port 6514. Logstash processes these messages and forwards them to Event Hubs, where they can be consumed by various Azure services or third-party tools. Prerequisites Before starting the deployment, ensure you have these tools installed and configured: Terraform: Version 1.5 or later Azure CLI: Authenticated to your Azure subscription kubectl: For managing Kubernetes resources after deployment Several Azure resources must be created manually before running Terraform, as the configuration references them. This approach provides flexibility in organizing resources across different teams or environments. Step 1: Create Resource Groups Create three resource groups to organize the solution components: az group create --name rg-syslog-prod --location eastus az group create --name rg-network-prod --location eastus az group create --name rg-data-prod --location eastus Each resource group serves a specific purpose: rg-syslog-prod : Contains the AKS cluster, Key Vault, and Log Analytics Workspace rg-network-prod : Holds networking resources (Virtual Network and Subnets) rg-data-prod : Houses the Event Hub Namespace for data ingestion Step 2: Configure Networking Create a Virtual Network with dedicated subnets for AKS and the Load Balancer: az network vnet create \ --resource-group rg-network-prod \ --name vnet-syslog-prod \ --address-prefixes 10.0.0.0/16 \ --location eastus az network vnet subnet create \ --resource-group rg-network-prod \ --vnet-name vnet-syslog-prod \ --name snet-aks-prod \ --address-prefixes 10.0.1.0/24 az network vnet subnet create \ --resource-group rg-network-prod \ --vnet-name vnet-syslog-prod \ --name snet-lb-prod \ --address-prefixes 10.0.2.0/24 The network design uses non-overlapping CIDR ranges to prevent routing conflicts. The Load Balancer subnet will later be assigned the static IP address 10.0.2.100 . Step 3: Set Up Event Hub Namespace Create an Event Hub Namespace with a dedicated Event Hub for syslog data: az eventhubs namespace create \ --resource-group rg-data-prod \ --name eh-syslog-prod \ --location eastus \ --sku Standard az eventhubs eventhub create \ --resource-group rg-data-prod \ --namespace-name eh-syslog-prod \ --name syslog The Standard SKU provides Kafka protocol support, which Logstash uses for reliable message delivery. The namespace automatically includes a RootManageSharedAccessKey for authentication. Step 4: Configure Key Vault and TLS Certificate Create a Key Vault to store the TLS certificate: az keyvault create \ --resource-group rg-syslog-prod \ --name kv-syslog-prod \ --location eastus For production environments, import a certificate from your Certificate Authority: az keyvault certificate import \ --vault-name kv-syslog-prod \ --name cert-syslog-prod \ --file certificate.pfx \ --password <pfx-password> For testing purposes, you can generate a self-signed certificate: az keyvault certificate create \ --vault-name kv-syslog-prod \ --name cert-syslog-prod \ --policy "$(az keyvault certificate get-default-policy)" Important: The certificate's Common Name (CN) or Subject Alternative Name (SAN) must match the DNS name your syslog sources will use to connect to the Load Balancer. Step 5: Create Log Analytics Workspace Set up a Log Analytics Workspace for monitoring the AKS cluster: az monitor log-analytics workspace create \ --resource-group rg-syslog-prod \ --workspace-name log-syslog-prod \ --location eastus Understanding the Terraform Configuration With the prerequisites in place, let's examine the Terraform configuration that automates the remaining deployment. The configuration follows a modular approach, making it easy to customize for different environments. Referencing Existing Resources The Terraform configuration begins by importing references to the manually created resources: data "azurerm_client_config" "current" {} data "azurerm_resource_group" "rg-main" { name = "rg-syslog-prod" } data "azurerm_resource_group" "rg-network" { name = "rg-network-prod" } data "azurerm_resource_group" "rg-data" { name = "rg-data-prod" } data "azurerm_virtual_network" "primary" { name = "vnet-syslog-prod" resource_group_name = data.azurerm_resource_group.rg-network.name } data "azurerm_subnet" "kube-cluster" { name = "snet-aks-prod" resource_group_name = data.azurerm_resource_group.rg-network.name virtual_network_name = data.azurerm_virtual_network.primary.name } data "azurerm_subnet" "kube-lb" { name = "snet-lb-prod" resource_group_name = data.azurerm_resource_group.rg-network.name virtual_network_name = data.azurerm_virtual_network.primary.name } These data sources establish connections to existing infrastructure, ensuring the AKS cluster and Load Balancer deploy into the correct network context. Deploying the AKS Cluster The AKS cluster configuration balances security, performance, and manageability: resource "azurerm_kubernetes_cluster" "primary" { name = "aks-syslog-prod" location = data.azurerm_resource_group.rg-main.location resource_group_name = data.azurerm_resource_group.rg-main.name dns_prefix = "aks-syslog-prod" default_node_pool { name = "default" node_count = 2 vm_size = "Standard_DS2_v2" vnet_subnet_id = data.azurerm_subnet.kube-cluster.id } identity { type = "SystemAssigned" } network_profile { network_plugin = "azure" load_balancer_sku = "standard" network_plugin_mode = "overlay" } oms_agent { log_analytics_workspace_id = data.azurerm_log_analytics_workspace.logstash.id } } Key configuration choices: System-assigned managed identity: Eliminates the need for service principal credentials Azure CNI in overlay mode: Provides efficient pod networking without consuming subnet IPs Standard Load Balancer SKU: Enables zone redundancy and higher performance OMS agent integration: Sends cluster metrics to Log Analytics for monitoring The cluster requires network permissions to create the internal Load Balancer: resource "azurerm_role_assignment" "aks-netcontrib" { scope = data.azurerm_virtual_network.primary.id principal_id = azurerm_kubernetes_cluster.primary.identity[0].principal_id role_definition_name = "Network Contributor" } Configuring Logstash Deployment The Logstash deployment uses Kubernetes resources for reliability and scalability. First, create a dedicated namespace: resource "kubernetes_namespace" "logstash" { metadata { name = "logstash" } } The internal Load Balancer service exposes Logstash on a static IP: resource "kubernetes_service" "loadbalancer-logstash" { metadata { name = "logstash-lb" namespace = kubernetes_namespace.logstash.metadata[0].name annotations = { "service.beta.kubernetes.io/azure-load-balancer-internal" = "true" "service.beta.kubernetes.io/azure-load-balancer-ipv4" = "10.0.2.100" "service.beta.kubernetes.io/azure-load-balancer-internal-subnet" = data.azurerm_subnet.kube-lb.name "service.beta.kubernetes.io/azure-load-balancer-resource-group" = data.azurerm_resource_group.rg-network.name } } spec { type = "LoadBalancer" selector = { app = kubernetes_deployment.logstash.metadata[0].name } port { name = "logstash-tls" protocol = "TCP" port = 6514 target_port = 6514 } } } The annotations configure Azure-specific Load Balancer behavior, including the static IP assignment and subnet placement. Securing Logstash with TLS Kubernetes Secrets store the TLS certificate and Logstash configuration: resource "kubernetes_secret" "logstash-ssl" { metadata { name = "logstash-ssl" namespace = kubernetes_namespace.logstash.metadata[0].name } data = { "server.crt" = data.azurerm_key_vault_certificate_data.logstash.pem "server.key" = data.azurerm_key_vault_certificate_data.logstash.key } type = "Opaque" } The certificate data comes directly from Key Vault, maintaining a secure chain of custody. Logstash Container Configuration The deployment specification defines how Logstash runs in the cluster: resource "kubernetes_deployment" "logstash" { metadata { name = "logstash" namespace = kubernetes_namespace.logstash.metadata[0].name } spec { selector { match_labels = { app = "logstash" } } template { metadata { labels = { app = "logstash" } } spec { container { name = "logstash" image = "docker.elastic.co/logstash/logstash:8.17.4" security_context { run_as_user = 1000 run_as_non_root = true allow_privilege_escalation = false } resources { requests = { cpu = "500m" memory = "1Gi" } limits = { cpu = "1000m" memory = "2Gi" } } volume_mount { name = "logstash-config-volume" mount_path = "/usr/share/logstash/pipeline/logstash.conf" sub_path = "logstash.conf" read_only = true } volume_mount { name = "logstash-ssl-volume" mount_path = "/etc/logstash/certs" read_only = true } } } } } } Security best practices include: Running as a non-root user (UID 1000) Disabling privilege escalation Mounting configuration and certificates as read-only Setting resource limits to prevent runaway containers Automatic Scaling Configuration The Horizontal Pod Autoscaler ensures Logstash scales with demand: resource "kubernetes_horizontal_pod_autoscaler" "logstash_hpa" { metadata { name = "logstash-hpa" namespace = kubernetes_namespace.logstash.metadata[0].name } spec { scale_target_ref { kind = "Deployment" name = kubernetes_deployment.logstash.metadata[0].name api_version = "apps/v1" } min_replicas = 1 max_replicas = 30 target_cpu_utilization_percentage = 80 } } This configuration maintains between 1 and 30 replicas, scaling up when CPU usage exceeds 80%. Logstash Pipeline Configuration The Logstash configuration file defines how to process syslog messages: input { tcp { port => 6514 type => "syslog" ssl_enable => true ssl_cert => "/etc/logstash/certs/server.crt" ssl_key => "/etc/logstash/certs/server.key" ssl_verify => false } } output { stdout { codec => rubydebug } kafka { bootstrap_servers => "${name}.servicebus.windows.net:9093" topic_id => "syslog" security_protocol => "SASL_SSL" sasl_mechanism => "PLAIN" sasl_jaas_config => 'org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="Endpoint=sb://${name}.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=${primary_key};EntityPath=syslog";' codec => "json" } } The configuration: Listens on port 6514 for TLS-encrypted syslog messages Outputs to stdout for debugging (visible in container logs) Forwards processed messages to Event Hubs using the Kafka protocol Deploying the Solution With all components configured, deploy the solution using Terraform: Initialize Terraform in your project directory: terraform init Review the planned changes: terraform plan Apply the configuration: terraform apply Connect to the AKS cluster: az aks get-credentials \ --resource-group rg-syslog-prod \ --name aks-syslog-prod Verify the deployment: kubectl -n logstash get pods kubectl -n logstash get svc kubectl -n logstash get hpa Configuring Syslog Sources After deployment, configure your syslog sources to send messages to the Load Balancer: Create a DNS record pointing to the Load Balancer IP (10.0.2.100). For example: syslog.yourdomain.com Configure syslog clients to send RFC 5425 messages over TLS to port 6514 Install the certificate chain on syslog clients if using a private CA or self-signed certificate Example rsyslog configuration for a Linux client: *.* @@syslog.yourdomain.com:6514;RSYSLOG_SyslogProtocol23Format Monitoring and Troubleshooting Monitor the deployment using several methods: View Logstash logs to verify message processing: kubectl -n logstash logs -l app=logstash --tail=50 Check autoscaling status: kubectl -n logstash describe hpa logstash-hpa Monitor in Azure Portal: Navigate to the Log Analytics Workspace to view AKS metrics Check Event Hub metrics to confirm message delivery Review Load Balancer health probes and connection statistics Security Best Practices This deployment incorporates several security measures: TLS encryption: All syslog traffic is encrypted using certificates from Key Vault Network isolation: The internal Load Balancer restricts access to the virtual network Managed identities: No credentials are stored in the configuration Container security: Logstash runs as a non-root user with minimal privileges For production deployments, consider these additional measures: Enable client certificate validation in Logstash for mutual TLS Add Network Security Groups to restrict source IPs Implement Azure Policy for compliance validation Enable Azure Defender for Kubernetes Integration with Azure Services Once syslog data flows into Event Hubs, you can integrate with various Azure services: Azure Sentinel: Configure Data Collection Rules to ingest syslog data for security analytics. See the Azure Sentinel documentation for detailed steps. Azure Data Explorer: Create a data connection to analyze syslog data with KQL queries. Azure Stream Analytics: Process syslog streams in real-time for alerting or transformation. Logic Apps: Trigger workflows based on specific syslog patterns or events. Cost Optimization To optimize costs while maintaining performance: Right-size the AKS node pool based on actual syslog volume Use Azure Spot instances for non-critical environments Configure Event Hub retention based on compliance requirements Enable auto-shutdown for development environments Conclusion This Terraform-based solution provides a robust foundation for collecting syslog data in Azure. The combination of AKS, Logstash, and Event Hubs creates a scalable pipeline that integrates seamlessly with Azure's security and analytics services. The modular design allows easy customization for different environments and requirements. Whether you're collecting logs from a handful of devices or thousands, this architecture scales to meet your needs while maintaining security and reliability. For next steps, consider implementing additional Logstash filters for data enrichment, setting up automated certificate rotation, or expanding the solution to collect other log formats. The flexibility of this approach ensures it can grow with your organization's logging requirements.
