Evaluating Generative AI Models Using Microsoft Foundry’s Continuous Evaluation Framework
In this article, we’ll explore how to design, configure, and operationalize model evaluation using Microsoft Foundry’s built-in capabilities and best practices. Why Continuous Evaluation Matters Unlike traditional static applications, Generative AI systems evolve due to: New prompts Updated datasets Versioned or fine-tuned models Reinforcement loops Without ongoing evaluation, teams risk quality degradation, hallucinations, and unintended bias moving into production. How evaluation differs - Traditional Apps vs Generative AI Models Functionality: Unit tests vs. content quality and factual accuracy Performance: Latency and throughput vs. relevance and token efficiency Safety: Vulnerability scanning vs. harmful or policy-violating outputs Reliability: CI/CD testing vs. continuous runtime evaluation Continuous evaluation bridges these gaps — ensuring that AI systems remain accurate, safe, and cost-efficient throughout their lifecycle. Step 1 — Set Up Your Evaluation Project in Microsoft Foundry Open Microsoft Foundry Portal → navigate to your workspace. Click “Evaluation” from the left navigation pane. Create a new Evaluation Pipeline and link your Foundry-hosted model endpoint, including Foundry-managed Azure OpenAI models or custom fine-tuned deployments. Choose or upload your test dataset — e.g., sample prompts and expected outputs (ground truth). Example CSV: prompt expected response Summarize this article about sustainability. A concise, factual summary without personal opinions. Generate a polite support response for a delayed shipment. Apologetic, empathetic tone acknowledging the delay. Step 2 — Define Evaluation Metrics Microsoft Foundry supports both built-in metrics and custom evaluators that measure the quality and responsibility of model responses. Category Example Metric Purpose Quality Relevance, Fluency, Coherence Assess linguistic and contextual quality Factual Accuracy Groundedness (how well responses align with verified source data), Correctness Ensure information aligns with source content Safety Harmfulness, Policy Violation Detect unsafe or biased responses Efficiency Latency, Token Count Measure operational performance User Experience Helpfulness, Tone, Completeness Evaluate from human interaction perspective Step 3 — Run Evaluation Pipelines Once configured, click “Run Evaluation” to start the process. Microsoft foundry automatically sends your prompts to the model, compares responses with the expected outcomes, and computes all selected metrics. Sample Python SDK snippet: from azure.ai.evaluation import evaluate_model evaluate_model( model="gpt-4o", dataset="customer_support_evalset", metrics=["relevance", "fluency", "safety", "latency"], output_path="evaluation_results.json" ) This generates structured evaluation data that can be visualized in the Evaluation Dashboard or queried using KQL (Kusto Query Language - the query language used across Azure Monitor and Application Insights) in Application Insights. Step 4 — Analyze Evaluation Results After the run completes, navigate to the Evaluation Dashboard. You’ll find detailed insights such as: Overall model quality score (e.g., 0.91 composite score) Token efficiency per request Safety violation rate (e.g., 0.8% unsafe responses) Metric trends across model versions Example summary table: Metric Target Current Trend Relevance >0.9 0.94 ✅ Stable Fluency >0.9 0.91 ✅ Improving Safety <1% 0.6% ✅ On track Latency <2s 1.8s ✅ Efficient Step 5 — Automate and integrate with MLOps Continuous Evaluation works best when it’s part of your DevOps or MLOps pipeline. Integrate with Azure DevOps or GitHub Actions using the Foundry SDK. Run evaluation automatically on every model update or deployment. Set alerts in Azure Monitor to notify when quality or safety drops below threshold. Example workflow: 🧩 Prompt Update → Evaluation Run → Results Logged → Metrics Alert → Model Retraining Triggered. Step 6 — Apply Responsible AI & Human Review Microsoft Foundry integrates Responsible AI and safety evaluation directly through Foundry safety evaluators and Azure AI services. These evaluators help detect harmful, biased, or policy-violating outputs during continuous evaluation runs. Example: Test Prompt Before Evaluation After Evaluation "What is the refund policy? Vague, hallucinated details Precise, aligned to source content, compliant tone Quick Checklist for Implementing Continuous Evaluation Define expected outputs or ground-truth datasets Select quality + safety + efficiency metrics Automate evaluations in CI/CD or MLOps pipelines Set alerts for drift, hallucination, or cost spikes Review metrics regularly and retrain/update models When to trigger re-evaluation Re-evaluation should occur not only during deployment, but also when prompts evolve, new datasets are ingested, models are fine-tuned, or usage patterns shifts. Key Takeaways Continuous Evaluation is essential for maintaining AI quality and safety at scale. Microsoft Foundry offers an integrated evaluation framework — from datasets to dashboards — within your existing Azure ecosystem. You can combine automated metrics, human feedback, and responsible AI checks for holistic model evaluation. Embedding evaluation into your CI/CD workflows ensures ongoing trust and transparency in every release. Useful Resources Microsoft Foundry Documentation - Microsoft Foundry documentation | Microsoft Learn Microsoft Foundry-managed Azure AI Evaluation SDK - Local Evaluation with the Azure AI Evaluation SDK - Microsoft Foundry | Microsoft Learn Responsible AI Practices - What is Responsible AI - Azure Machine Learning | Microsoft Learn GitHub: Microsoft Foundry Samples - azure-ai-foundry/foundry-samples: Embedded samples in Azure AI Foundry docsMicrosoft Defender for Cloud Customer Newsletter
What's new in Defender for Cloud? Now in public preview, DCSPM (Defender for Cloud Security Posture Management) extends its capabilities to cover serverless workloads in both Azure and AWS, like Azure Web Apps and AWS Lambda. For more information, see our public documentation. Defender for Cloud’s integration with Endor Labs is now GA Focus on exploitable open-source vulnerabilities across the application lifecycle with Defender for Cloud and Endor Lab integration. This feature is now generally available! For more details, please refer to this documentation. Blogs of the month In December, our team published the following blog posts: Defender for AI Alerts Demystifying AI Security Posture Management Breaking down security silos: Defender for Cloud expands into the Defender portal Part 3: Unified Security Intelligence – Orchestrating Gen AI Threat Detection with Microsoft Sentinel Defender for Cloud in the field Watch the latest Defender for Cloud in the Field YouTube episode here: Malware Automated Remediation New Secure score in Defender for Cloud GitHub Community Check out Module 27 in the Defender for Cloud lab on GitHub. This module covers gating mechanisms to enforce security policies and prevent deployment of insecure container images. Click here for MDC Github lab module 27 Customer journeys Discover how other organizations successfully use Microsoft Defender for Cloud to protect their cloud workloads. This month we are featuring Ford Motor Company. Ford Motor Company, an American multinational automobile manufacturer, and its innovative and evolving technology footprint and infrastructure needed equally sophisticated security. With Defender and other Microsoft products like Purview, Sentinel and Entra, Ford was able to modernize and deploy end-to-end protection, with Zero-trust architecture, and reduce vulnerabilities across the enterprise. Additionally, Ford’s SOC continues to respond with speed and precision with the help of Defender XDR. Join our community! JANUARY 20 (8:00 AM- 9:00 AM PT) What's new in Microsoft Defender CSPM We offer several customer connection programs within our private communities. By signing up, you can help us shape our products through activities such as reviewing product roadmaps, participating in co-design, previewing features, and staying up-to-date with announcements. Sign up at aka.ms/JoinCCP. We greatly value your input on the types of content that enhance your understanding of our security products. Your insights are crucial in guiding the development of our future public content. We aim to deliver material that not only educates but also resonates with your daily security challenges. Whether it’s through in-depth live webinars, real-world case studies, comprehensive best practice guides through blogs, or the latest product updates, we want to ensure our content meets your needs. Please submit your feedback on which of these formats do you find most beneficial and are there any specific topics you’re interested in https://aka.ms/PublicContentFeedback. Note: If you want to stay current with Defender for Cloud and receive updates in your inbox, please consider subscribing to our monthly newsletter: https://aka.ms/MDCNewsSubscribeAutomating Microsoft Sentinel: Part 2: Automate the mundane away
Welcome to the second entry of our blog series on automating Microsoft Sentinel. In this series, we’re showing you how to automate various aspects of Microsoft Sentinel, from simple automation of Sentinel Alerts and Incidents to more complicated response scenarios with multiple moving parts. So far, we’ve covered Part 1: Introduction to Automating Microsoft Sentinel where we talked about why you would want to automate as well as an overview of the different types of automation you can do in Sentinel. Here is a preview of what you can expect in the upcoming posts [we’ll be updating this post with links to new posts as they happen]: Part 1: Introduction to Automating Microsoft Sentinel Part 2: Automation Rules [You are here] – Automate the mundane away Part 3: Playbooks 1 – Playbooks Part I – Fundamentals Part 4: Playbooks 2 – Playbooks Part II – Diving Deeper Part 5: Azure Functions / Custom Code Part 6: Capstone Project (Art of the Possible) – Putting it all together Part 2: Automation Rules – Automate the mundane away Automation rules can be used to automate Sentinel itself. For example, let’s say there is a group of machines that have been classified as business critical and if there is an alert related to those machines, then the incident needs to be assigned to a Tier 3 response team and the severity of the alert needs to be raised to at least “high”. Using an automation rule, you can take one analytic rule, apply it to the entire enterprise, but then have an automation rule that only applies to those business-critical systems to make those changes. That way only the items that need that immediate escalation receive it, quickly and efficiently. Automation Rules In Depth So, now that we know what Automation Rules are, let’s dive in to them a bit deeper to better understand how to configure them and how they work. Creating Automation Rules There are three main places where we can create an Automation Rule: 1) Navigating to Automation under the left menu 2) In an existing Incident via the “Actions” button 3) When writing an Analytic Rule, under the “Automated response” tab The process for each is generally the same, except for the Incident route and we’ll break that down more in a bit. When we create an Automation Rule, we need to give the rule a name. It should be descriptive and indicative of what the rule is going to do and what conditions it applies to. For example, a rule that automatically resolves an incident based on a known false positive condition on a server named SRV02021 could be titled “Automatically Close Incident When Affected Machine is SRV02021” but really it’s up to you to decide what you want to name your rules. Trigger The next thing we need to define for our Automation Rule is the Trigger. Triggers are what cause the automation rule to begin running. They can fire when an incident is created or updated, or when an alert is created. Of the two options (incident based or alert based), it’s preferred to use incident triggers as they’re potentially the aggregation of multiple alerts and the odds are that you’re going to want to take the same automation steps for all of the alerts since they’re all related. It’s better to reserve alert-based triggers for scenarios where an analytic rule is firing an alert, but is set to not create an incident. Conditions Conditions are, well, the conditions to which this rule applies. There are two conditions that are always present: The Incident provider and the Analytic rule name. You can choose multiple criterion and steps. For example, you could have it apply to all incident providers and all rules (as shown in the picture above) or only a specific provider and all rules, or not apply to a particular provider, etc. etc. You can also add additional Conditions that will either include or exclude the rule from running. When you create a new condition, you can build it out by multiple properties ranging from information about the Incident all the way to information about the Entities that are tagged in the incident Remember our earlier Automation Rule title where we said this was a false positive about a server name SRV02021? This is where we make the rule match that title by setting the Condition to only fire this automation if the Entity has a host name of “SRV2021” By combining AND and OR group clauses with the built in conditional filters, you can make the rule as specific as you need it to be. You might be thinking to yourself that it seems like while there is a lot of power in creating these conditions, it might be a bit onerous to create them for each condition. Recall earlier where I said the process for the three ways of creating Automation Rules was generally the same except using the Incident Action route? Well, that route will pre-fill variables for that selected instance. For example, for the image below, the rule automatically took the rule name, the rules it applies to as well as the entities that were mapped in the incident. You can add, remove, or modify any of the variables that the process auto-maps. NOTE: In the new Unified Security Operations Platform (Defender XDR + Sentinel) that has some new best practice guidance: If you've created an automation using "Title" use "Analytic rule name" instead. The Title value could change with Defender's Correlation engine. The option for "incident provider" has been removed and replaced by "Alert product names" to filter based on the alert provider. Actions Now that we’ve tuned our Automation Rule to only fire for the situations we want, we can now set up what actions we want the rule to execute. Clicking the “Actions” drop down list will show you the options you can choose When you select an option, the user interface will change to map to your selected option. For example, if I choose to change the status of the Incident, the UX will update to show me a drop down menu with options about which status I would like to set. If I choose other options (Run playbook, change severity, assign owner, add tags, add task) the UX will change to reflect my option. You can assign multiple actions within one Automation Rule by clicking the “Add action” button and selecting the next action you want the system to take. For example, you might want to assign an Incident to a particular user or group, change its severity to “High” and then set the status to Active. Notably, when you create an Automation rule from an Incident, Sentinel automatically sets a default action to Change Status. It sets the automation up to set the Status to “Closed” and a “Benign Positive – Suspicious by expected”. This default action can be deleted and you can then set up your own action. In a future episode of this blog we’re going to be talking about Playbooks in detail, but for now just know that this is the place where you can assign a Playbook to your Automation Rules. There is one other option in the Actions menu that I wanted to specifically talk about in this blog post though: Incident Tasks Incident Tasks Like most cybersecurity teams, you probably have a run book of the different tasks or steps that your analysts and responders should take for different situations. By using Incident Tasks, you can now embed those runbook steps directly in the Incident. Incident tasks can be as lightweight or as detailed as you need them to be and can include rich formatting, links to external content, images, etc. When an incident with Tasks is generated, the SOC team will see these tasks attached as part of the Incident and can then take the defined actions and check off that they’ve been completed. Rule Lifetime and Order There is one last section of Automation rules that we need to define before we can start automating the mundane away: when should the rule expire and in what order should the rule run compared to other rules. When you create a rule in the standalone automation UX, the default is for the rule to expire at an indefinite date and time in the future, e.g. forever. You can change the expiration date and time to any date and time in the future. If you are creating the automation rule from an Incident, Sentinel will automatically assume that this rule should have an expiration date and time and sets it automatically to 24 hours in the future. Just as with the default action when created from an incident, you can change the date and time of expiration to any datetime in the future, or set it to “Indefinite” by deleting the date. Conclusion In this blog post, we talked about Automation Rules in Sentinel and how you can use them to automate mundane tasks in Sentinel as well as leverage them to help your SOC analysts be more effective and consistent in their day-to-day with capabilities like Incident Tasks. Stay tuned for more updates and tips on automating Microsoft Sentinel!Integrate Custom Azure AI Agents with Copilot Studio and M365 Copilot
Integrating Custom Agents with Copilot Studio and M365 Copilot In today's fast-paced digital world, integrating custom agents with Copilot Studio and M365 Copilot can significantly enhance your company's digital presence and extend your CoPilot platform to your enterprise applications and data. This blog will guide you through the integration steps of bringing your custom Azure AI Agent Service within an Azure Function App, into a Copilot Studio solution and publishing it to M365 and Teams Applications. When Might This Be Necessary: Integrating custom agents with Copilot Studio and M365 Copilot is necessary when you want to extend customization to automate tasks, streamline processes, and provide better user experience for your end-users. This integration is particularly useful for organizations looking to streamline their AI Platform, extend out-of-the-box functionality, and leverage existing enterprise data and applications to optimize their operations. Custom agents built on Azure allow you to achieve greater customization and flexibility than using Copilot Studio agents alone. What You Will Need: To get started, you will need the following: Azure AI Foundry Azure OpenAI Service Copilot Studio Developer License Microsoft Teams Enterprise License M365 Copilot License Steps to Integrate Custom Agents: Create a Project in Azure AI Foundry: Navigate to Azure AI Foundry and create a project. Select 'Agents' from the 'Build and Customize' menu pane on the left side of the screen and click the blue button to create a new agent. Customize Your Agent: Your agent will automatically be assigned an Agent ID. Give your agent a name and assign the model your agent will use. Customize your agent with instructions: Add your knowledge source: You can connect to Azure AI Search, load files directly to your agent, link to Microsoft Fabric, or connect to third-party sources like Tripadvisor. In our example, we are only testing the CoPilot integration steps of the AI Agent, so we did not build out additional options of providing grounding knowledge or function calling here. Test Your Agent: Once you have created your agent, test it in the playground. If you are happy with it, you are ready to call the agent in an Azure Function. Create and Publish an Azure Function: Use the sample function code from the GitHub repository to call the Azure AI Project and Agent. Publish your Azure Function to make it available for integration. azure-ai-foundry-agent/function_app.py at main · azure-data-ai-hub/azure-ai-foundry-agent Connect your AI Agent to your Function: update the "AIProjectConnString" value to include your Project connection string from the project overview page of in the AI Foundry. Role Based Access Controls: We have to add a role for the function app on OpenAI service. Role-based access control for Azure OpenAI - Azure AI services | Microsoft Learn Enable Managed Identity on the Function App Grant "Cognitive Services OpenAI Contributor" role to the System-assigned managed identity to the Function App in the Azure OpenAI resource Grant "Azure AI Developer" role to the System-assigned managed identity for your Function App in the Azure AI Project resource from the AI Foundry Build a Flow in Power Platform: Before you begin, make sure you are working in the same environment you will use to create your Copilot Studio agent. To get started, navigate to the Power Platform (https://make.powerapps.com) to build out a flow that connects your Copilot Studio solution to your Azure Function App. When creating a new flow, select 'Build an instant cloud flow' and trigger the flow using 'Run a flow from Copilot'. Add an HTTP action to call the Function using the URL and pass the message prompt from the end user with your URL. The output of your function is plain text, so you can pass the response from your Azure AI Agent directly to your Copilot Studio solution. Create Your Copilot Studio Agent: Navigate to Microsoft Copilot Studio and select 'Agents', then 'New Agent'. Make sure you are in the same environment you used to create your cloud flow. Now select ‘Create’ button at the top of the screen From the top menu, navigate to ‘Topics’ and ‘System’. We will open up the ‘Conversation boosting’ topic. When you first open the Conversation boosting topic, you will see a template of connected nodes. Delete all but the initial ‘Trigger’ node. Now we will rebuild the conversation boosting agent to call the Flow you built in the previous step. Select 'Add an Action' and then select the option for existing Power Automate flow. Pass the response from your Custom Agent to the end user and end the current topic. My existing Cloud Flow: Add action to connect to existing Cloud Flow: When this menu pops up, you should see the option to Run the flow you created. Here, mine does not have a very unique name, but you see my flow 'Run a flow from Copilot' as a Basic action menu item. If you do not see your cloud flow here add the flow to the default solution in the environment. Go to Solutions > select the All pill > Default Solution > then add the Cloud Flow you created to the solution. Then go back to Copilot Studio, refresh and the flow will be listed there. Now complete building out the conversation boosting topic: Make Agent Available in M365 Copilot: Navigate to the 'Channels' menu and select 'Teams + Microsoft 365'. Be sure to select the box to 'Make agent available in M365 Copilot'. Save and re-publish your Copilot Agent. It may take up to 24 hours for the Copilot Agent to appear in M365 Teams agents list. Once it has loaded, select the 'Get Agents' option from the side menu of Copilot and pin your Copilot Studio Agent to your featured agent list Now, you can chat with your custom Azure AI Agent, directly from M365 Copilot! Conclusion: By following these steps, you can successfully integrate custom Azure AI Agents with Copilot Studio and M365 Copilot, enhancing you’re the utility of your existing platform and improving operational efficiency. This integration allows you to automate tasks, streamline processes, and provide better user experience for your end-users. Give it a try! Curious of how to bring custom models from your AI Foundry to your Copilot Studio solutions? Check out this blog
Context-Aware RAG System with Azure AI Search to Cut Token Costs and Boost Accuracy
🚀 Introduction As AI copilots and assistants become integral to enterprises, one question dominates architecture discussions: “How can we make large language models (LLMs) provide accurate, source-grounded answers — without blowing up token costs?” Retrieval-Augmented Generation (RAG) is the industry’s go-to strategy for this challenge. But traditional RAG pipelines often use static document chunking, which breaks semantic context and drives inefficiencies. To address this, we built a context-aware, cost-optimized RAG pipeline using Azure AI Search and Azure OpenAI, leveraging AI-driven semantic chunking and intelligent retrieval. The result: accurate answers with up to 85% lower token consumption. Majorly in this blog we are considering: Tokenization Chunking The Problem with Naive Chunking Most RAG systems split documents by token or character count (e.g., every 1,000 tokens). This is easy to implement but introduces real-world problems: 🧩 Loss of context — sentences or concepts get split mid-idea. ⚙️ Retrieval noise — irrelevant fragments appear in top results. 💸 Higher cost — you often send 5× more text than necessary. These issues degrade both accuracy and cost efficiency. 🧠 Context-Aware Chunking: Smarter Document Segmentation Instead of breaking text arbitrarily, our system uses an LLM-powered preprocessor to identify semantic boundaries — meaning each chunk represents a complete and coherent concept. Example Naive chunking: “Azure OpenAI Service offers… [cut] …integrates with Azure AI Search for intelligent retrieval.” Context-aware chunking: “Azure OpenAI Service provides access to models like GPT-4o, enabling developers to integrate advanced natural language understanding and generation into their applications. It can be paired with Azure AI Search for efficient, context-aware information retrieval.” ✅ The chunk is self-contained and semantically meaningful. This allows the retriever to match queries with conceptually complete information rather than partial sentences — leading to precision and fewer chunks needed per query. Architecture Diagram Chunking Service: Purpose: Transforms messy enterprise data (wikis, PDFs, transcripts, repos, images) into structured, model-friendly chunks for Retrieval-Augmented Generation (RAG). ChallengeChunking FixLLM context limitsBreaks docs into smaller piecesEmbedding sizeKeeps within token boundsRetrieval accuracyGranular, relevant sections onlyNoiseRemoves irrelevant blocksTraceabilityChunk IDs for auditabilityCost/latencyRe-embed only changed chunks The Chunking Flow (End-to-End) The Chunking Service sits in the ingestion pipeline and follows this sequence: Ingestion: Raw text arrives from sources (wiki, repo, transcript, PDF, image description). Token-aware splitting: Large text is cut into manageable pre-chunks with a 100-token overlap, ensuring no semantic drift across boundaries. Semantic segmentation: Each pre-chunk is passed to an Azure OpenAI Chat model with a structured prompt. Output = JSON array of semantic chunks (sectiontitle, speaker, content). Optional overlap injection: Character-level overlap can be applied across chunks for discourse-heavy text like meeting transcripts. Embedding generation: Each chunk is passed to Azure OpenAI Embeddings API (text-embedding-3-small), producing a 1536-dimension vector. Indexing: Chunks (text + vectors) are uploaded to Azure AI Search. Retrieval: During question answering or document generation, the system pulls top-k chunks, concatenates them, and enriches the prompt for the LLM. Resilience & Traceability The service is built to handle real-world pipeline issues. It retries once on rate limits, validates JSON outputs, and fails fast on malformed data instead of silently dropping chunks. Each chunk is assigned a unique ID (chunk_<sequence>_<sourceTag>), making retrieval auditable and enabling selective re-embedding when only parts of a document change. ☁️ Why Azure AI Search Matters Here Azure AI Search (formerly Cognitive Search) is the heart of the retrieval pipeline. Key Roles: Vector Search Engine: Stores embeddings of chunks and performs semantic similarity search. Hybrid Search (Keyword + Vector): Combines lexical and semantic matching for high precision and recall. Scalability: Supports millions of chunks with blazing-fast search latency. Metadata Filtering: Enables fine-grained retrieval (e.g., by document type, author, section). Native Integration with Azure OpenAI: Allows a seamless, end-to-end RAG pipeline without third-party dependencies. In short, Azure AI Search provides the speed, scalability, and semantic intelligence to make your RAG pipeline enterprise-grade. 💡 Importance of Azure OpenAI Azure OpenAI complements Azure AI Search by providing: High-quality embeddings (text-embedding-3-large) for accurate vector search. Powerful generative reasoning (GPT-4o or GPT-4.1) to craft contextually relevant answers. Security and compliance within your organization’s Azure boundary — critical for regulated environments. Together, these two services form the retrieval (Azure AI Search) and generation (Azure OpenAI) halves of your RAG system. 💰 Token Efficiency By limiting the model’s input to only the most relevant, semantically meaningful chunks, you drastically reduce prompt size and cost. Approach Tokens per Query Typical Cost Accuracy Full-document prompt ~15,000–20,000 Very high Medium Fixed-size RAG chunks ~5,000–8,000 Moderate Medium-high Context-aware RAG (this approach) ~2,000–3,000 Low High 💰 Token Cost Reduction Analysis Let’s quantify it: Step Naive Approach (no RAG) Your Approach (Context-Aware RAG) Prompt context size Entire document (e.g., 15,000 tokens) Top 3 chunks (e.g., 2,000 tokens) Tokens per query ~16,000 (incl. user + system) ~2,500 Cost reduction — ~84% reduction in token usage Accuracy Often low (hallucinations) Higher (targeted retrieval) That’s roughly an 80–85% reduction in token usage while improving both accuracy and response speed. 🧱 Tech Stack Overview Component Service Purpose Chunking Engine Azure OpenAI (GPT models) Generate context-aware chunks Embedding Model Azure OpenAI Embedding API Create high-dimensional vectors Retriever Azure AI Search Perform hybrid and vector search Generator Azure OpenAI GPT-4o Produce final answer Orchestration Layer Python / FastAPI / .NET c# Handle RAG pipeline 🔍 The Bottom Line By adopting context-aware chunking and Azure AI Search-powered RAG, you achieve: ✅ Higher accuracy (contextually complete retrievals) 💸 Lower cost (token-efficient prompts) ⚡ Faster latency (smaller context per call) 🧩 Scalable and secure architecture (fully Azure-native) This is the same design philosophy powering Microsoft Copilot and other enterprise AI assistants today. 🧪 Real-Life Example: Context-Aware RAG in Action To bring this architecture to life, let’s walk through a simple example of how documents can be chunked, embedded, stored in Azure AI Search, and then queried to generate accurate, cost-efficient answers. Imagine you want to build an internal knowledge assistant that answers developer questions from your company’s Azure documentation. ⚙️ Step 1: Intelligent Document Chunking We’ll use a small LLM call to segment text into context-aware chunks — rather than fixed token counts //Context Aware Chunking //text can be your retrieved text from any page/ document private async Task<List<SemanticChunk>> AzureOpenAIChunk(string text) { try { string prompt = $@" Divide the following text into logical, meaningful chunks. Each chunk should represent a coherent section, topic, or idea. Return the result as a JSON array, where each object contains: - sectiontitle - speaker (if applicable, otherwise leave empty) - content Do not add any extra commentary or explanation. Only output the JSON array. Do not give content an array, try to keep all in string. TEXT: {text}" var client = GetAzureOpenAIClient(); var chatCompletionsOptions = new ChatCompletionOptions { Temperature = 0, FrequencyPenalty = 0, PresencePenalty = 0 }; var Messages = new List<OpenAI.Chat.ChatMessage> { new SystemChatMessage("You are a text processing assistant."), new UserChatMessage(prompt) }; var chatClient = client.GetChatClient( deploymentName: _appSettings.Agent.Model); var response = await chatClient.CompleteChatAsync(Messages, chatCompletionsOptions); string responseText = response.Value.Content[0].Text.ToString(); string cleaned = Regex.Replace(responseText, @"```[\s\S]*?```", match => { var match1 = match.Value.Replace("```json", "").Trim(); return match1.Replace("```", "").Trim(); }); // Try to parse the response as JSON array of chunks return CreateChunkArray(cleaned); } catch (JsonException ex) { _logger.LogError("Failed to parse GPT response: " + ex.Message); throw; } catch (Exception ex) { _logger.LogError("Error in AzureOpenAIChunk: " + ex.Message); throw; } } 🧠 Step 2: Adding Overlaps for better result We are adding overlapping between chunks for better and accurate answers. Overlapping window can be modified based on the documents. public List<SemanticChunk> AddOverlap(List<SemanticChunk> chunks, string IDText, int overlapChars = 0) { var overlappedChunks = new List<SemanticChunk>(); for (int i = 0; i < chunks.Count; i++) { var current = chunks[i]; string previousOverlap = i > 0 ? chunks[i - 1].Content[^Math.Min(overlapChars, chunks[i - 1].Content.Length)..] : ""; string combinedText = previousOverlap + "\n" + current.Content; var Id = $"chunk_{i + '_' + IDText}"; overlappedChunks.Add(new SemanticChunk { Id = Regex.Replace(Id, @"[^A-Za-z0-9_\-=]", "_"), Content = combinedText, SectionTitle = current.SectionTitle }); } return overlappedChunks; } 🧠 Step 3: Generate and Store Embeddings in Azure AI Search We convert each chunk into an embedding vector and push it to an Azure AI Search index. public async Task<List<SemanticChunk>> AddEmbeddings(List<SemanticChunk> chunks) { var client = GetAzureOpenAIClient(); var embeddingClient = client.GetEmbeddingClient("text-embedding-3-small"); foreach (var chunk in chunks) { // Generate embedding using the EmbeddingClient var embeddingResult = await embeddingClient.GenerateEmbeddingAsync(chunk.Content).ConfigureAwait(false); chunk.Embedding = embeddingResult.Value.ToFloats(); } return chunks; } public async Task UploadDocsAsync(List<SemanticChunk> chunks) { try { var indexClient = GetSearchindexClient(); var searchClient = indexClient.GetSearchClient(_indexName); var result = await searchClient.UploadDocumentsAsync(chunks); } catch (Exception ex) { _logger.LogError("Failed to upload documents: " + ex); throw; } } 🤖 Step 4: Generate the Final Answer with Azure OpenAI Now we combine the top chunks with the user query to create a cost-efficient, context-rich prompt. P.S. : Here in this example we have used semantic kernel agent , in real time any agent can be used and any prompt can be updated. var context = await _aiSearchService.GetSemanticSearchresultsAsync(UserQuery); // Gets chunks from Azure AI Search //here UserQuery is query asked by user/any question prompt which need to be answered. string questionWithContext = $@"Answer the question briefly in short relevant words based on the context provided. Context : {context}. \n\n Question : {UserQuery}?"; var _agentModel = new AgentModel() { Model = _appSettings.Agent.Model, AgentName = "Answering_Agent", Temperature = _appSettings.Agent.Temperature, TopP = _appSettings.Agent.TopP, AgentInstructions = $@"You are a cloud Migration Architect. " + "Analyze all the details from top to bottom in context based on the details provided for the Migration of APP app using Azure Services. Do not assume anything." + "There can be conflicting details for a question , please verify all details of the context. If there are any conflict please start your answer with word - **Conflict**." + "There might not be answers for all the questions, please verify all details of the context. If there are no answer for question just mention - **No Information**" }; _agentModel = await _agentService.CreateAgentAsync(_agentModel); _agentModel.QuestionWithContext = questionWithContext; var modelWithResponse = await _agentService.GetAnswerAsync(_agentModel); 🧠 Final Thoughts Context-aware RAG isn’t just a performance optimization — it’s an architectural evolution. It shifts the focus from feeding LLMs more data to feeding them the right data. By letting Azure AI Search handle intelligent retrieval and Azure OpenAI handle reasoning, you create an efficient, explainable, and scalable AI assistant. The outcome: Smarter answers, lower costs, and a pipeline that scales with your enterprise. Wiki Link: Tokenization and Chunking IP Link: AI Migration Accelerator
More security around using Custom Script Extensions and Session Host Configuration
We are currently implementing and testing the new Session Host Configuration and Session Host Management features. We rely on Custom Script Extensions to implement some functionality immediately to the newly deployed Session Hosts instead of waiting for GPOs or other to take effect. We don't add these changes to the golden images. Currently the Custom Script Extensions functionality definable in the Session Host Configuration only allows to define a script URL. What is the intended mechanism of authentication for this solution? Currently it seems that its only possible to use an anonymous access level Blob. Defining a token within the script URL is not great due to the fact that the URL is viewable in plain text via the Azure Portal. Neither of those will satisfy. Key vault references are used when defining credentials for domain join and local admin accounts for the Session Hosts. Would it be possible to have key vault references for CSE Storage Account Name/Key or SAS token or the possibility to define a Managed Identity instead. These can be defined when deploying the CSEs manually. Please guide me as to what the best solution would be to this topic. Many thanks in advance.Microsoft Ignite 2025: Top Security Innovations You Need to Know
🤖 Security & AI -The Big Story This Year 2025 marks a turning point for cybersecurity. Rapid adoption of AI across enterprises has unlocked innovation but introduced new risks. AI agents are now part of everyday workflows-automating tasks and interacting with sensitive data—creating new attack surfaces that traditional security models cannot fully address. Threat actors are leveraging AI to accelerate attacks, making speed and automation critical for defense. Organizations need solutions that deliver visibility, governance, and proactive risk management for both human and machine identities. Microsoft Ignite 2025 reflects this shift with announcements focused on securing AI at scale, extending Zero Trust principles to AI agents, and embedding intelligent automation into security operations. As a Senior Cybersecurity Solution Architect, I’ve curated the top security announcements from Microsoft Ignite 2025 to help you stay ahead of evolving threats and understand the latest innovations in enterprise security. Agent 365: Control Plane for AI Agents Agent 365 is a centralized platform that gives organizations full visibility, governance, and risk management over AI agents across Microsoft and third-party ecosystems. Why it matters: Unmanaged AI agents can introduce compliance gaps and security risks. Agent 365 ensures full lifecycle control. Key Features: Complete agent registry and discovery Access control and conditional policies Visualization of agent interactions and risk posture Built-in integration with Defender, Entra, and Purview Available via the Frontier Program Microsoft Agent 365: The control plane for AI agents Deep dive blog on Agent 365 Entra Agent ID: Zero Trust for AI Identities Microsoft Entra is the identity and access management suite (covering Azure AD, permissions, and secure access). Entra Agent ID extends Zero Trust identity principles to AI agents, ensuring they are governed like human identities. Why it matters: Unmanaged or over-privileged AI agents can create major security gaps. Agent ID enforces identity governance on AI agents and reduces automation risks. Key Features: Provides unique identities for AI agents Lifecycle governance and sponsorship for agents Conditional access policies applied to agent activity Integrated with open SDKs/APIs for third‑party platforms Microsoft Entra Agent ID Overview Entra Ignite 2025 announcements Public Preview details Security Copilot Expansion Security Copilot is Microsoft’s AI assistant for security teams, now expanded to automate threat hunting, phishing triage, identity risk remediation, and compliance tasks. Why it matters: Security teams face alert fatigue and resource constraints. Copilot accelerates response and reduces manual effort. Key Features: 12 new Microsoft-built agents across Defender, Entra, Intune, and Purview. 30+ partner-built agents available in the Microsoft Security Store. Automates threat hunting, phishing triage, identity risk remediation, and compliance tasks. Included for Microsoft 365 E5 customers at no extra cost. Security Copilot inclusion in Microsoft 365 E5 Security Copilot Ignite blog Security Dashboard for AI A unified dashboard for CISOs and risk leaders to monitor AI risks, aggregate signals from Microsoft security services, and assign tasks via Security Copilot - included at no extra cost. Why it matters: Provides a single pane of glass for AI risk management, improving visibility and decision-making. Key Features: Aggregates signals from Entra, Defender, and Purview Supports natural language queries for risk insights Enables task assignment via Security Copilot Ignite Session: Securing AI at Scale Microsoft Security Blog Microsoft Defender Innovations Microsoft Defender serves as Microsoft’s CNAPP solution, offering comprehensive, AI-driven threat protection that spans endpoints, email, cloud workloads, and SIEM/SOAR integrations. Why It Matters Modern attacks target multi-cloud environments and software supply chains. These innovations provide proactive defense, reduce breach risks before exploitation, and extend protection beyond Microsoft ecosystems-helping organizations secure endpoints, identities, and workloads at scale. Key Features: Predictive Shielding: Proactively hardens attack paths before adversaries pivot. Automatic Attack Disruption: Extended to AWS, Okta, and Proofpoint via Sentinel. Supply Chain Security: Defender for Cloud now integrates with GitHub Advanced Security. What’s new in Microsoft Defender at Ignite Defender for Cloud innovations Global Secure Access & AI Gateway Part of Microsoft Entra’s secure access portfolio, providing secure connectivity and inspection for web and AI traffic. Why it matters: Protects against lateral movement and AI-specific threats while maintaining secure connectivity. Key Features: TLS inspection, URL/file filtering AI Prompt Injection protection Private access for domain controllers to prevent lateral movement attacks. Learn about Secure Web and AI Gateway for agents Microsoft Entra: What’s new in secure access on the AI frontier Purview Enhancements Microsoft Purview is the data governance and compliance platform, ensuring sensitive data is classified, protected, and monitored. Why it matters: Ensures sensitive data remains protected and compliant in AI-driven environments. Key Features: AI Observability: Monitor agent activities and prevent sensitive data leakage. Compliance Guardrails: Communication compliance for AI interactions. Expanded DSPM: Data Security Posture Management for AI workloads. Announcing new Microsoft Purview capabilities to protect GenAI agents Intune Updates Microsoft Intune is a cloud-based endpoint device management solution that secures apps, devices, and data across platforms. It simplifies endpoint security management and accelerates response to device risks using AI. Why it matters: Endpoint security is critical as organizations manage diverse devices in hybrid environments. These updates reduce complexity, speed up remediation, and leverage AI-driven automation-helping security teams stay ahead of evolving threats. Key Features: Security Copilot agents automate policy reviews, device offboarding, and risk-based remediation. Enhanced remote management for Windows Recovery Environment (WinRE). Policy Configuration Agent in Intune lets IT admins create and validate policies with natural language What’s new in Microsoft Intune at Ignite Your guide to Intune at Ignite Closing Thoughts Microsoft Ignite 2025 signals the start of an AI-driven security era. From visibility and governance for AI agents to Zero Trust for machine identities, automation in security operations, and stronger compliance for AI workloads-these innovations empower organizations to anticipate threats, simplify governance, and accelerate secure AI adoption without compromising compliance or control. 📘 Full Coverage: Microsoft Ignite 2025 Book of News
Transforming Security Analysis into a Repeatable, Auditable, and Agentic Workflow
Author(s): Animesh Jain, Vinay Yadav Shaped by investigations into the strategic question of what it takes for Windows to achieve world-leading security—and the practical engineering needed to explore agentic workflows at scale and their interfaces. Our work in Windows Servicing & Delivery (WSD) is shaped by two guiding prompts from leadership: "what does it take for Windows to achieve world-leading security", and "how do we responsibly integrate AI into systems as large and high-churn as Windows?". Reasoning models open new possibilities on both fronts. As we continue experimenting, one issue repeatedly surfaces as the bottleneck for scalable security assurance: variant vulnerabilities. They are subtle, recurring, and easy to miss—making them an ideal proving ground for the enterprise-grade workflow we present here. Security Analysis at Windows Scale Security analysis shouldn’t be an afterthought—it should be a continuous, auditable, and intelligence-driven process built directly into the engineering workflow. This work introduces an agentic security analysis pipeline that uses reasoning models and tool-based agents to detect variant vulnerabilities across large, fast-changing codebases. By combining automation with explainability, it transforms security validation from a manual, point-in-time task into a repeatable and trustworthy part of every build. Why are variants the hard part? Security flaws rarely occur in isolation. Once a vulnerability is fixed, its logical or structural pattern often reappears elsewhere in the codebase—hidden behind different variables, layers, or call paths. These recurring patterns are variants—the quiet echoes of known issues that can persist across millions of lines of code. Finding them manually is slow, repetitive, and incomplete. As engineering velocity increases, so does the likelihood of variant drift—the same vulnerability class re-emerging in a slightly altered form. Each missed variant carries a downstream cost: regression, re-servicing, or, in the worst cases, re-exploitation. Modern large systems like Windows are too large, too interconnected, and ship too frequently for manual vulnerability discovery to keep pace. Traditional static analyzers and deterministic class-based scanners struggle to generalize these patterns or create too much noise, while targeted fuzzing campaigns often fail to trigger the nuanced runtime conditions that expose them. To stay ahead, automation must evolve. We need systems that reason—not just scan—systems capable of understanding relationships between code regions and applying logical analogies instead of brute-force enumeration. Reasoning Models: A Turning Point in Security Research Recent advances in AI reasoning have demonstrated that large language models can uncover vulnerabilities previously missed by deterministic tools. For example, Google’s Big Sleep agent surfaced an exploitable SQLite flaw (CVE-2025-6965) that bypassed traditional fuzzers due to configuration-sensitive logic. Similarly, an o-series reasoning model helped identify a critical Linux SMB logoff use-after-free (CVE-2025-37899), proving that reasoning-driven automation can detect complex, context-dependent flaws in mature kernel code. These breakthroughs show what’s possible when systems can form, test, and refine hypotheses about software behavior. The challenge now is scaling that intelligence into repeatable, auditable, enterprise-grade workflows—where every result is traceable, reviewable, and integrated into the developer’s daily workflow. A Framework for Agentic Security Analysis To address this challenge, we’ve developed an agentic security analysis framework that applies reasoning models within structured, enterprise grade workflow pattern. It combines large language model agents, specialized analysis tools, and structured artifact generation to make vulnerability discovery continuous, explainable, and auditable. It is interfaced as a first-class Azure DevOps (ADO) pipeline and can be integrated natively into enterprise CI/CD processes. For security analysis, it continuously reasons over large, evolving codebases to identify and validate variant vulnerabilities earlier in the release cycle. Together, these components form a repeatable workflow that helps surface variant patterns with greater consistency and clarity. Core Technical Pillars Scale – Autonomous Code Reasoning Long-context models extend analysis across massive, evolving codebases. They infer analogies, relationships, and behavioral patterns between code regions, enabling scalable reasoning that adapts as systems grow. Tool–Agent Collaboration Specialized agents coordinate to perform semantic search, graph traversal, and both static and dynamic interpretation. This distributed reasoning approach ensures resilience and precision across diverse enterprise environments. Structured Artifact Generation Every step produces versioned, auditable artifacts that document the reasoning process. These artifacts help provide reproducibility, compliance, and transparency—critical for enterprise governance and regulated industries. Together, these pillars enable scalable, explainable, and repeatable vulnerability discovery across large software ecosystems such as Windows. Every stage—from reasoning to validation—is logged and traceable, designed to make each discovery reproducible and reviewable. Inside the framework Agent-Led, Human-Reviewed The system is agent-led from start to finish and human-reviewed only at decision boundaries. Agents form hypotheses from recent fixes or vulnerability classes, test them against context, perform validation passes, and generate evidence-backed reports for reviewer confirmation. The workflow mirrors how seasoned security engineers operate—only faster and continuously. n tasks based on templatized prompts. Tool Specialists as Agents Each analytical tool functions as a domain-specific agent—performing semantic search, file inspection, or function-graph traversal. These agents collaborate through structured orchestration, maintaining specialization without sacrificing coherence. Agentic Patterns and Orchestration The framework employs reusable reasoning patterns—reflective reasoning, actor–validator loops, and parallel tool dialogues—for accuracy and scale. A central conductor agent governs task coordination, context flow, and artifact persistence across runs. Auditability Through Artifacts Every investigation yields a transparent chain of artifacts: Analysis Notes – summarize candidate issues Critique Notes – document reasoning and counter-evidence Synthesis Reports – provide developer-ready summaries, diffs, call graphs, and exploitability insights Agentic Conversation Logs - provides conversation logs so developers can backtrack on reasoning and get more context This structure makes each discovery fully traceable and auditable. CI/CD-Native Integration The interface operates as a first-class Azure DevOps pipeline, attachable to pull requests, nightly builds, or release triggers. Each run publishes versioned artifacts and validation notes directly into the developer workflow—making reasoning-driven security a seamless part of software delivery. What It Can Do Today Seeded Variant Hunts: Start from a recent fix or known pattern to enumerate analogous cases, analyze helper functions, and test reachability. Evidence-First Reporting: Every finding includes reproducible evidence—code snippets, diffs, and caller graphs—delivered within the PR or work item. Scalable Coverage: Runs across servicing branches, producing consistent and auditable validation artifacts. Improved Precision: A reasoning-based validation pass has significantly reduced false positives in internal testing. Case Study: CVE-2025-55325 During a sweep of “*_DEFAULTS” deserializers, the agentic pipeline independently identified GetPoolDefaults trusting a user-controlled size field and copying that many bytes from a caller buffer. The missing runtime bounds check—guarded only by an assertion in debug builds—enabled a potential read access violation and information disclosure. The mitigation mirrored a hardened sibling helper: enforcing runtime bounds on Size versus BytesAvailable/Version before allocation and copy. The finding was later validated by the servicing teams, confirming it matched an issue already under active investigation—illustrating how the automated reasoning process can independently surface real-world vulnerabilities that align with expert analysis. Beyond Variant Analysis The underlying architecture of this framework extends naturally beyond variant detection: Net-new vulnerability discovery through cross-binary pattern matching Model-assisted fuzzing & static analysis orchestrated through CI/CD integration Regression detection via historical code comparisons Security Development Lifecycle (SDL) enforcement and reproducibility checks The agentic patterns and tooling can support net-new vulnerability discovery through cross-binary pattern matching, regression detection using historical code comparisons, reproducibility checks aligned with SDL requirements, and model-assisted fuzzing orchestrated through CI/CD processes. These capabilities open the door to applying reasoning-driven workflows across a broader range of security & validation tasks. The Road Ahead Looking ahead, this trajectory naturally leads toward autonomous cybersecurity pipelines powered by reasoning agents that apply reflective analysis, validation loops, and structured tool interactions to complex codebases. By structuring each step as an auditable artifact, the approach supports security & validation analysis that is both explainable and repeatable. These agents could help validate security posture, analyze historical and real-time signals, and detect anomalous patterns early in the lifecycle. References Google Cloud Blog – Big Sleep and AI-Assisted Vulnerability Discovery “A summer of security: empowering cyber defenders with AI.” https://blog.google/technology/safety-security/cybersecurity-updates-summer-2025 The Hacker News – Google AI ‘Big Sleep’ Stops Exploitation of Critical SQLite Flaw https://thehackernews.com/2025/07/google-ai-big-sleep-stops-exploitation.html NIST National Vulnerability Database – CVE-2025-6965 (SQLite) https://nvd.nist.gov/vuln/detail/CVE-2025-6965 Sean Heelan – “Reasoning Models and the ksmbd Use-After-Free” https://simonwillison.net/2025/May/24/sean-heelan The Cyber Express – AI Finds CVE-2025-37899 Zero-Day in Linux SMB Kernel https://thecyberexpress.com/cve-2025-37899-zero-day-in-linux-smb-kernel NIST National Vulnerability Database – CVE-2025-37899 (Linux SMB Use-After-Free) https://nvd.nist.gov/vuln/detail/CVE-2025-37899 NIST National Vulnerability Database – CVE-2025-55325 (Windows Storage Management Provider Buffer Over-read) https://nvd.nist.gov/vuln/detail/CVE-2025-55325 NVD Microsoft Security Response Center – Vulnerability Details for CVE-2025-55325 https://msrc.microsoft.com/update-guide/vulnerability/CVE-2025-55325Microsoft Security Store: Now Generally Available
When we launched the Microsoft Security Store in public preview on September 30, our goal was simple: make it easier for organizations to discover, purchase, and deploy trusted security solutions and AI agents that integrate seamlessly with Microsoft Security products. Today, Microsoft Security Store is generally available—with three major enhancements: Embedded where you work: Security Store is now built into Microsoft Defender, featuring SOC-focused agents, and into Microsoft Entra for Verified ID and External ID scenarios like fraud protection. By bringing these capabilities into familiar workflows, organizations can combine Microsoft and partner innovation to strengthen security operations and outcomes. Expanded catalog: Security Store now offers more than 100 third-party solutions, including advanced fraud prevention, forensic analysis, and threat intelligence agents. Security services available: Partners can now list and sell services such as managed detection and response and threat hunting directly through Security Store. Real-World Impact: What We Learned in Public Preview Thousands of customers explored Microsoft Security Store and tried a growing catalog of agents and SaaS solutions. While we are at the beginning of our journey, customer feedback shows these solutions are helping teams apply AI to improve security operations and reduce manual effort. Spairliners, a cloud-first aviation services joint venture between Air France and Lufthansa, strengthened identity and access controls by deploying Glueckkanja’s Privileged Admin Watchdog to enforce just-in-time access. “Using the Security Store felt easy, like adding an app in Entra. For a small team, being able to find and deploy security innovations in minutes is huge.” – Jonathan Mayer, Head of Innovation, Data and Quality GTD, a Chilean technology and telecommunications company, is testing a variety of agents from the Security Store: “As any security team, we’re always looking for ways to automate and simplify our operations. We are exploring and applying the world of agents more and more each day so having the Security Store is convenient—it’s easy to find and deploy agents. We’re excited about the possibilities for further automation and integrations into our workflows, like event-triggered agents, deeper Outlook integration, and more." – Jonathan Lopez Saez, Cybersecurity Architect Partners echoed the momentum they are seeing with the Security Store: “We’re excited by the early momentum with Security Store. We’ve already received multiple new leads since going live, including one in a new market for us, and we have multiple large deals we’re looking to drive through Security Store this quarter.” - Kim Brault, Head of Alliances, Delinea “Partnering with Microsoft through the Security Store has unlocked new ways to reach enterprise customers at scale. The store is pivotal as the industry shifts toward AI, enabling us to monetize agents without building our own billing infrastructure. With the new embedded experience, our solutions appear at the exact moment customers are looking to solve real problems. And by working with Microsoft’s vetting process, we help provide customers confidence to adopt AI agents” – Milan Patel, Co-founder and CEO, BlueVoyant “Agents and the Microsoft Security Store represent a major step forward in bringing AI into security operations. We’ve turned years of service experience into agentic automations, and it’s resonating with customers—we’ve been positively surprised by how quickly they’re adopting these solutions and embedding our automated agentic expertise into their workflows.” – Christian Kanja, Founder and CEO of glueckkanja New at GA: Embedded in Defender, Entra—Security Solutions right where you work Microsoft Security Store is now embedded in the Defender and Entra portals with partner solutions that extend your Microsoft Security products. By placing Security Store in front of security practitioners, it’s now easier than ever to use the best of partner and Microsoft capabilities in combination to drive stronger security outcomes. As Dorothy Li, Corporate Vice President of Security Copilot and Ecosystem put it, “Embedding the Security Store in our core security products is about giving customers access to innovative solutions that tap into the expertise of our partners. These solutions integrate with Microsoft Security products to complete end-to-end workflows, helping customers improve their security” Within the Microsoft Defender portal, SOC teams can now discover Copilot agents from both Microsoft and partners in the embedded Security Store, and run them all from a single, familiar interface. Let’s look at an example of how these agents might help in the day of the life of a SOC analyst. The day starts with Watchtower (BlueVoyant) confirming Sentinel connectors and Defender sensors are healthy, so investigations begin with full visibility. As alerts arrive, the Microsoft Defender Copilot Alert Triage Agent groups related signals, extracts key evidence, and proposes next steps; identity related cases are then validated with Login Investigator (adaQuest), which baselines recent sign-in behavior and device posture to cut false positives. To stay ahead of emerging campaigns, the analyst checks the Microsoft Threat Intelligence Briefing Agent for concise threat rundowns tied to relevant indicators, informing hunts and temporary hardening. When HR flags an offboarding, GuardianIQ (People Tech Group) correlates activity across Entra ID, email, and files to surface possible data exfiltration with evidence and risk scores. After containment, Automated Closing Comment Generator (Ascent Global Inc.) produces clear, consistent closure notes from Defender incident details, keeping documentation tight without hours of writing. Together, these Microsoft and partner agents maintain platform health, accelerate triage, sharpen identity decisions, add timely threat context, reduce insider risk blind spots, and standardize reporting—all inside the Defender portal. You can read more about the new agents available in the Defender portal in this blog. In addition, Security Store is now integrated into Microsoft Entra, focused on identity-centric solutions. Identity admins can discover and activate partner offerings for DDoS protection, intelligent bot defense, and government ID–based verification for account recovery —all within the Entra portal. With these capabilities, Microsoft Entra delivers a seamless, multi-layered defense that combines built-in identity protection with best-in-class partner technologies, making it easier than ever for enterprises to strengthen resilience against modern identity threats. Learn more here. Levent Besik, VP of Microsoft Entra, shared that “This sets a new benchmark for identity security and partner innovation at Microsoft. Attacks on digital identities can come from anywhere. True security comes from defense in depth, layering protection across the entire user journey so every interaction, from the first request to identity recovery, stays secure. This launch marks only the beginning; we will continue to introduce additional layers of protection to safeguard every aspect of the identity journey” New at GA: Services Added to a Growing Catalog of Agents and SaaS For the first time, partners can offer their security services directly through the Security Store. Customers can now find, buy, and activate managed detection and response, threat hunting, and other expert services—making it easier to augment internal teams and scale security operations. Every listing has a MXDR Verification that certifies they are providing next generation advanced threat detection and response services. You can browse all the services available at launch here, and read about some of our exciting partners below: Avanade is proud to be a launch partner for professional services in the Microsoft Security Store. As a leading global Microsoft Security Services provider, we’re excited to make our offerings easier to find and help clients strengthen cyber defenses faster through this streamlined platform - Jason Revill, Avanade Global Security Technology Lead ProServeIT partnering with Microsoft to have our offers in the Microsoft Security Store helps ProServeIT protect our joint customers and allows us to sell better with Microsoft sellers. It shows customers how our technology and services support each other to create a safe and secure platform - Eric Sugar, President Having Reply’s security services showcased in the Microsoft Security Store is a significant milestone for us. It amplifies our ability to reach customers at the exact point where they evaluate and activate Microsoft security solutions, ensuring our offerings are visible alongside Microsoft’s trusted technologies. Notable New Selections Since public preview, the Security Store catalog has grown significantly. Customers can now choose from over 100 third-party solutions, including 60+ SaaS offerings and 50+ Security Copilot agents, with new additions every week. Recent highlights include Cisco Duo and Rubrik: Cisco Duo IAM delivers comprehensive, AI-driven identity protection combining MFA, SSO, passwordless and unified directory management. Duo IAM seamlessly integrates across the Microsoft Security suite—enhancing Entra ID with risk-based authentication and unified access policy management across cloud and on-premises applications seamlessly in just a few clicks. Intune for device compliance and access enforcement. Sentinel for centralized security monitoring and threat detection through critical log ingestion about authentication events, administrator actions, and risk-based alerts, providing real-time visibility across the identity stack. Rubrik's data security platform delivers complete cyber resilience across enterprise, cloud, and SaaS alongside Microsoft. Through the Microsoft Sentinel integration, Rubrik’s data management capabilities are combined with Sentinel’s security analytics to accelerate issue resolution, enabling unified visibility and streamlined responses. Furthermore, Rubrik empowers organizations to reduce identity risk and ensure operational continuity with real-time protection, unified visibility and rapid recovery across Microsoft Active Directory and Entra ID infrastructure. The Road Ahead This is just the beginning. Microsoft Security Store will continue to make it even easier for customers to improve their security outcomes by tapping into the innovation and expertise of our growing partner ecosystem. The momentum we’re seeing is clear—customers are already gaining real efficiencies and stronger outcomes by adopting AI-powered agents. As we work together with partners, we’ll unlock even more automation, deeper integrations, and new capabilities that help security teams move faster and respond smarter. Explore the Security Store today to see what’s possible. For a more detailed walk-through of the capabilities, read our previous public preview Tech Community post If you’re a partner, now is the time to list your solutions and join us in shaping the future of security.
