rss.livelink.threads-in-node

From reactive firefighting to proactive operations: custom skills for Azure SRE Agent

rmmartins — Sat, 01 Aug 2026 22:45:21 GMT

TL;DR: Azure SRE Agent is great at reactive incident response, but doesn't cover proactive operations out of the box. I built an open-source pack of 8 custom skills that add governance audits, FinOps reporting, capacity planning, postmortems, and more. Combined with Scheduled Investigations, they move your operations from reactive to proactive.

How a customer conversation revealed the gap between what Azure SRE Agent does out of the box, and what production teams actually need.

The conversation that started it all

Prerequisites: You need an active Azure SRE Agent deployment. If you don't have one yet, the setup takes about 15 minutes.

A few weeks ago, I sat down with an engineering team that had never heard of Azure SRE Agent. They were running a growing SaaS platform on Azure, handling incidents manually, and burning engineering hours on repetitive diagnostics. The classic "we don't have an SRE team, but we need SRE outcomes" situation.

I walked them through what Azure SRE Agent brings to the table: AI-powered incident investigation, automated root cause analysis, 40+ MCP connectors, sandboxed code execution, memory across investigations. Their reaction was immediate: "This solves our reactive problem. But what about the proactive stuff?"

That question stuck with me. Because they were right.

Azure SRE Agent: what it is (and what it isn't)

For those unfamiliar: Azure SRE Agent is Microsoft's AI-powered site reliability agent, generally available since March 2026. Internally at Microsoft, over 1,300 SRE Agents are running in production, handling 35,000+ incidents per month and saving 20,000+ engineering hours monthly.

Here's what it excels at:

Reactive incident response: an alert fires, the agent investigates, correlates telemetry, identifies root cause, and either resolves or escalates with full context.
Diagnostics automation: KQL queries, resource health checks, dependency mapping, all executed in a sandboxed environment.
Memory and learning: each investigation builds organizational knowledge. The agent remembers past incidents and applies that context to new ones.
Integrations: PagerDuty, ServiceNow, Teams, GitHub (including Bring Your Own GitHub App since June 2026), Azure Monitor, and custom APIs via plugins.

What it doesn't do out of the box:

Run proactive governance audits
Track cost optimization opportunities on a schedule
Assess architecture quality against Well-Architected Framework
Generate capacity forecasts before you hit quota limits
Produce blameless postmortems with structured 5-Whys analysis
Monitor your Defender Secure Score and suggest improvements
Evaluate whether your AI/OpenAI workloads are production-ready

That gap between reactive excellence and proactive operations is exactly what I built the Skills Pack to fill.

Custom skills: the extensibility model

Since June 2026, Azure SRE Agent supports Custom Skills and Plugins. A skill is a structured prompt with instructions, context, and output format that the agent executes using its full toolkit (Azure Resource Graph, KQL, ARM APIs, Cost Management APIs, etc.).

Think of it this way:

The agent provides the reasoning engine and tool access
A skill provides the domain knowledge and methodology

You write a SKILL.md file describing what to check, how to score findings, and what format to report in. The agent handles execution, data gathering, correlation, and output generation. No SDK, no compiled code, no deployment pipeline. Just structured instructions in Markdown.

The proactive operations skills pack

I created 8 custom skills that transform Azure SRE Agent from a reactive incident responder into a proactive operations partner:

The skills at a glance

#	Skill	What it does	When to run
01	Well-Architected Review	5-pillar WAF assessment with maturity scoring	Before production launches, quarterly
02	Compliance and Governance	Policy, RBAC, tagging, locks, naming audit	Weekly
03	Capacity Planning	Quota utilization, growth projection, scaling prep	Bi-weekly
04	FinOps Intelligence	Cost optimization + team chargeback in one report	Monthly
05	Incident Postmortem	Blameless postmortem generator with 5-Whys	After every SEV1/SEV2
06	Defender Secure Score	Score monitoring + prioritized improvement plan	Weekly
07	Digital Native Governance	Startup governance maturity (15 checks, scored 0-100)	Monthly
08	AI Foundry and OpenAI Posture	Security, reliability and cost posture for AI workloads	Bi-weekly

Skill 08 pairs well with the architecture patterns in my companion post: Azure AI Foundry: from zero to production. If you're running Foundry workloads, that guide covers APIM as AI Gateway, spillover architecture, and the production checklist that this skill audits against.

How they differ from Azure Advisor

This is the most common question I get:

	Azure Advisor	SRE Agent Skills
Output	Flat list of per-resource recommendations	Scored reports with maturity levels and priority
Correlation	None, each recommendation is isolated	Connects findings across domains ("fix 2.2 first, it blocks 2.5")
Remediation	Link to documentation	Ready-to-paste `az` CLI commands
Scheduling	Passive (you check when you remember)	Runs on your schedule, surfaces issues proactively
Coverage	Cost, Security, Reliability, OpEx, Performance	+ FinOps chargeback, Postmortem, AI posture, governance maturity
Context	Generic (same advice for everyone)	Asks about your scenario and adapts

Advisor is a linter. These skills are closer to a staff SRE who reads everything, correlates findings, and delivers a prioritized action plan.

Example: Digital Native Governance in action

Here's what happens when you tell your SRE Agent: "Run a governance maturity check on my production subscription"

The agent:

Queries Azure Resource Graph for resource inventory
Checks Azure Policy assignments and compliance state
Audits RBAC role assignments against least-privilege principles
Validates tagging strategy completeness
Checks resource locks on critical resources
Evaluates naming conventions
Assesses network segmentation
Reviews backup and DR configuration
Checks monitoring and alerting coverage
Evaluates secret management practices

...and 5 more checks, each scored. You get a single number (e.g., 67/100) plus a prioritized list of what to fix first, with exact commands.

For a startup heading toward an enterprise customer's security review, this is the difference between "we think we're ready" and "here's our scored assessment with evidence."

Combining skills with scheduled investigations

The real power shows up when you combine custom skills with Scheduled Investigations (GA July 2026). Instead of running skills manually, you configure recurring schedules:

Every Monday 8am: Run Compliance & Governance → Slack results to #platform-ops
1st of each month: Run FinOps Intelligence → Email cost report to engineering leads
Every other Friday: Run Capacity Planning → Flag anything above 70% quota utilization
After every deployment: Run AI Foundry Posture → Validate no security regressions

This is where SRE Agent moves from Level 3 (reactive intelligence) to Level 4 (proactive operations) on the SRE maturity model. You stop waiting for things to break and start finding issues before they become incidents.

Getting started

Go to sre.azure.com → Skill Builder
Click + Create skill, paste any SKILL.md from the repo
Start with Digital Native Governance — fastest to see results
Set a schedule or run on-demand

The repo includes sample outputs for every skill so you can preview what you'll get before installing.

What's next

The skills are open source (MIT) and designed to be customized. Each SKILL.md is self-contained: adjust thresholds, add checks specific to your domain, change scoring weights. The CONTRIBUTING.md has guidelines if you want to add new skills.

Ideas the community has already suggested:

SLA/SLO monitoring dashboard
Network topology validator
Migration readiness assessment
Disaster recovery drill runner
Container Apps / AKS health check

If your team is running Azure SRE Agent and feeling the same gap between reactive and proactive, the repo is here. The community response in the first week (dozens of stars, multiple forks and contributions) tells me this resonates.

Next steps

Install the skills pack — clone the repo, copy the skill folders into your SRE Agent configuration, and enable Skill 01 (Governance Audit) as a Scheduled Investigation running weekly.
Run one investigation manually — trigger a governance audit against a non-production subscription. Review the output, adjust thresholds in SKILL.md to match your environment's tolerance.
Add to your sprint rhythm — schedule FinOps (Skill 02) bi-weekly and Capacity Planning (Skill 05) monthly. After two cycles, you'll have baseline data to prove proactive value to leadership.

Resources:

Questions or want to share how you're using SRE Agent? Leave a comment or open an issue on the repo.

Azure AI Foundry: from zero to production

rmmartins — Sat, 01 Aug 2026 22:46:31 GMT

What I cover when a customer asks "we want to build AI applications on Azure, where do we start?"

TL;DR: Azure AI Foundry is the unified platform for building AI apps on Azure. Start with Standard PAYGO, use Priority Processing selectively when latency matters but traffic is bursty, and move predictable baseline traffic to PTU when utilization exceeds 60-70%. Use ptucalc.com to model your costs before committing.

This guide is for engineering teams moving from prototype to production on Azure AI Foundry. If you're still evaluating whether Foundry is the right platform, start at ai.azure.com.

The starting point

A few weeks ago, I had a conversation with a customer's engineering team that was ready to build their first AI-powered application on Azure. They had experimented with ChatGPT, prototyped with the OpenAI API directly, and now needed to understand: how do we go from playground to production at enterprise scale?

That conversation became a workshop, then a deck, and now this guide. The questions they asked are the same ones I hear from every team making this jump.

What is Azure AI Foundry?

Azure AI Foundry is the unified platform for building, deploying, and operating AI applications on Azure. Think of it as the control plane for everything AI in your Azure environment:

Model Catalog: 1,900+ models (OpenAI, Meta Llama, Mistral, Cohere, Phi, others)
Prompt Engineering: playground, prompt flow, evaluation tools
Deployment Options: Standard and Priority pay-per-token processing, Provisioned Throughput (PTU), Global/Data Zone routing
Safety and Governance: content filters, red teaming tools, model monitoring
Agent Framework: multi-step AI agents with tool-calling, code interpreter, file search

Foundry is not just another Azure service. It is the layer that ties models, data, compute, and governance together into one development surface.

The decisions you will face

Every team building on Foundry hits the same questions in roughly the same order.

1. Model selection

The model landscape in mid-2026:

Model	Best for	Trade-off
GPT-5.x	Orchestration, complex reasoning, multi-step agents	Highest capability, highest cost
GPT-5-mini	Fast tasks, classification, summarization	90% of GPT-5 quality at 20% of cost
GPT-4.1	Legacy workloads (deprecating)	Stable but being superseded
Phi-4	Edge deployment, fine-tuning, embedding	Small, fast, cheap, customizable
Llama 3.x	Open-weight flexibility, on-prem requirements	Full control, self-managed

My recommendation for agentic workloads: GPT-5.x for the orchestrator (best tool-calling accuracy), GPT-5-mini for sub-tasks (classification, extraction, formatting), and Phi-4 or fine-tuned models for domain-specific components.

2. Model lifecycle

Every model in Foundry follows a lifecycle: Preview, GA, Legacy, Deprecated, Retired.

What matters in practice:

GA lasts roughly 18 months. Sounds like a lot, but it goes fast when you have a system in production.
Legacy means a replacement is available. Start planning migration.
Deprecated gives you about 90 days to migrate. After that, the API returns 410 Gone and your system stops.

The critical detail: if you use Provisioned Throughput (PTU), model migration is NOT automatic. You must do it manually: plan a maintenance window, test the new model with existing prompts, validate quality, and swap. Standard/Global Standard deployments auto-upgrade, but you don't control when.

My recommendation: create a model governance process. Monitor Azure Updates, maintain automated quality tests per model, and start migration planning at least 60 days before retirement.

3. Deployment type: Standard, Priority, or PTU

Most teams overthink this. The rule is simple:

Start with PAYGO (Pay-As-You-Go) when:

You're in development/testing
Traffic is unpredictable or bursty
You're still figuring out which models you'll use long-term

Add Priority Processing when:

The workload is user-facing and sensitive to latency
Traffic is bursty or concentrated in business hours
You want model-specific latency targets without reserving PTU capacity

Move to PTU (Provisioned Throughput Units) when:

Sustained utilization exceeds 60-70% of equivalent PTU capacity
You need guaranteed latency (no noisy-neighbor throttling)
You're running production workloads with predictable patterns

4. Where Priority Processing fits

Priority Processing is the middle ground between Standard PAYGO and PTU. It keeps pay-per-token billing and requires no reservation, but eligible requests are handled by a priority service tier with a defined, model-specific latency target. You can enable it for an entire Global Standard or US Data Zone Standard deployment, or select it per request with service_tier: "priority".

The benefit is simple: more consistent low latency for interactive agents, copilots, and other time-sensitive paths without paying for dedicated capacity during idle periods. A useful pattern is to reserve Priority Processing for requests users are actively waiting on, while background summarization, extraction, and evaluation stay on Standard. For predictable steady-state traffic, PTU is still the stronger economic and capacity choice.

Priority is not reserved capacity. Azure can process a request on the Standard tier during peak demand, for long-context requests on certain models, or when traffic increases by more than 50% tokens per minute in under 15 minutes. When that happens, the response reports service_tier: "default" and the request is billed at the Standard rate. Monitor ServiceTierRequest and ServiceTierResponse in Azure Monitor so you can see requested versus actual processing instead of assuming every request stayed on Priority.

Rule of thumb: Standard for flexible general traffic, Priority for latency-sensitive bursts, and PTU for predictable baseline throughput. Many production systems will use all three.

5. How PTU actually works

PTU is a token-bucket model. Each PTU reserves a fixed throughput in tokens per minute. The rate varies by model:

GPT-5-mini: ~3,500 TPM per PTU
GPT-5: varies by variant
GPT-4.1: 3,000 TPM per PTU (deprecating)

So 100 PTUs of GPT-5-mini give you roughly 350,000 tokens/minute guaranteed. Go past that and the API returns 429. No queue, no wait. Hard cutoff.

6. The cost math (this is where it gets interesting)

Tier	Price (reference Jul/2026)	Commitment
On-Demand	~$2/hour/PTU = $14,400/month	None
Monthly Reserved	~$0.72/hour/PTU = $5,184/month	1 month
Yearly Reserved	~$0.60/hour/PTU = $4,320/month	1 year

The break-even: if sustained utilization is above 60-70% of your PTU capacity, monthly reservation already beats PAYGO.

⚠️ These are reference prices as of July 2026. EA/MCA negotiated rates may differ. Always validate against your specific agreement.

I built ptucalc.com to help with exactly this calculation. It is open source. Plug in your usage patterns and it tells you the optimal tier and PTU count.

7. Spillover architecture

The pattern I recommend for production:

Configure your deployment with PTU as primary and PAYGO as spillover. You get:

Guaranteed latency for your baseline traffic (PTU)
No dropped requests during spikes (PAYGO absorbs overflow)
Cost optimization (PTU for steady-state, PAYGO only for peaks)

You configure this at the deployment level in Foundry. No application code changes.

Part 2: production hardening

Everything above gets you running. The sections below get you running safely, at scale, with governance.

APIM as your AI Gateway

For any production AI workload, I recommend putting Azure API Management (APIM) between your applications and the models. APIM acts as a centralized AI Gateway with six capabilities that Foundry alone does not provide:

Load balancing: round-robin or weighted distribution across multiple PTU/PAYGO backends. Enables DR and capacity distribution across regions.
Rate limiting by token: unlike traditional rate limiting by request count, APIM counts actual tokens consumed. A request that uses 10,000 tokens weighs differently than one using 100. Much fairer for consumption control.
Circuit breaker: when a PTU backend returns 429, APIM automatically fails over to the next backend (another PTU or PAYGO). No client-side retry needed.
Semantic caching: caches responses by semantic similarity of the prompt. If someone asked something similar in the last N minutes, it returns from cache. Reduces cost and latency for recurring questions.
Token tracking: consumption metrics per app, per team, per user. Emits to Azure Monitor. Essential for chargeback when multiple teams share the same models.
Content safety: gateway-level policies that block malicious inputs before they reach the model. Defense in depth on top of Foundry's content filters.

The pattern: your applications and agents call APIM, not the model directly. APIM routes, controls, monitors, and protects.

Reference architecture for agentic workloads

For teams building multi-agent systems, this is the reference architecture I recommend:

Orchestration layer: a primary agent (typically using the best tool-calling model available, today GPT-5.x) that coordinates sub-agents, maintains conversation state, and decides the next action.
Specialized agents layer: each agent optimized for a specific task using the right model. A data extraction agent on Phi-4, a compliance agent on GPT-4.1, a UX agent on GPT-5. Different models for different tasks, optimizing both cost and quality.
Gateway layer (APIM): sits between agents and models. Each agent has different rate limits, routes to different models, and the circuit breaker protects against throttling. This is where you centralize governance.
Models layer (Foundry): multiple deployments with PTU for base load and PAYGO for burst. Multi-region for DR. Spillover happens automatically via APIM routing.

The key point: agents never call models directly. They always go through the gateway. If a misbehaving agent starts consuming too many tokens, you cut it at the gateway without touching the agent's code.

If you're running Azure SRE Agent alongside your AI workloads, skill 08 (AI Foundry & OpenAI Posture) can audit your Foundry deployment against these architecture patterns on a schedule. See my companion post: Custom skills for Azure SRE Agent.

Anti-patterns to avoid

These are mistakes I see repeatedly in production. Most of them seem obvious once pointed out, but they happen all the time:

Don't do this	Do this instead	Impact if ignored
API keys in code	Managed Identity + Key Vault	Credential leak, billing attack
One endpoint for everything	APIM Gateway + per-app routing	Noisy neighbor, no visibility
Provision for peak	Spillover (PTU base + PAYGO burst)	60%+ idle capacity, waste
Ignore model lifecycle	Test pipeline + migration plan	410 Gone in production, outage
Default max_tokens (4096)	Calculate max_tokens per use case	Inflated PTU utilization, capacity waste
Retry without backoff	Exponential backoff + jitter	Retry storm, cascading 429s

The max_tokens one is subtle: Azure calculates PTU utilization based on input tokens PLUS reserved max_tokens, even if the actual response uses fewer. If you set max_tokens to 4096 but your typical response is 200 tokens, you are wasting capacity. ptucalc.com has a specific tool for this.

Production checklist

Things I check before any customer goes live:

Security and network

Private endpoints configured (no public internet exposure)
Managed Identity for authentication (no API keys in code)
Content Safety filters tuned (not just defaults)
VNet integration if required by compliance
Data residency: model deployment region matches data requirements
APIM as AI Gateway (no direct model access from applications)

Reliability

Multi-region deployment (primary + failover)
Retry logic with exponential backoff and jitter
Circuit breaker pattern for downstream dependencies
Health probes and availability monitoring
Spillover configured (PTU primary, PAYGO overflow)
Priority Processing scoped to latency-sensitive paths, not every request
APIM load balancing across PTU backends

Observability

Token consumption metrics in Azure Monitor
Latency P50/P95/P99 dashboards
Cost allocation tags on all Foundry resources
Alerting on 429 rates (throttling indicator)
Model performance evaluation pipeline (drift detection)
APIM token tracking enabled (per app/team/user)
Priority requests split by ServiceTierRequest and ServiceTierResponse

Cost governance

Budget alerts configured
Chargeback tags for multi-team environments
PTU utilization monitoring (target: 70-85%)
max_tokens tuned per use case (not default 4096)
Regular review cadence (monthly) for tier optimization

Model governance

Model lifecycle monitoring (Azure Updates subscription)
Automated quality tests per model version
Migration plan documented (60+ days before retirement)
PTU migration runbook (manual swap required)
Semantic caching configured in APIM for recurring queries

The typical progression

Most teams I work with follow this path:

Don't skip steps. Each phase teaches you something about your workload that informs the next decision.

Where to start

Open the Foundry Playground and test models against your actual use cases
Model your costs with ptucalc.com before committing to PTU
Deploy with spillover from day one. It costs nothing extra when PTU handles the load, but saves you when spikes hit
Set up monitoring early. You cannot optimize what you cannot measure

Next steps

Deploy a model in Foundry — pick GPT-4o on PAYGO, wire it to a single endpoint, and run a test prompt through the REST API. Time: 20 minutes.
Run ptucalc against your traffic — export your token consumption from Azure Monitor and model the break-even point. If you're above 60% sustained utilization, PTU likely pays for itself.
Add APIM in front — even in dev. Configure a single policy with token-rate-limit and emit-token-metric. This gives you observability and a retry layer from day one.

Resources:

Questions about deployment strategy or cost modeling? Leave a comment.

Your startup doesn't have an SRE team. Now what?

rmmartins — Wed, 29 Jul 2026 15:19:46 GMT

Most startups I work with have between 3 and 15 engineers. None of them have a dedicated SRE. The infrastructure runs on Azure, the team ships fast, and reliability is everyone's job until something breaks at 2 AM and it becomes one person's problem.

This is not a criticism. It is the rational allocation of scarce engineering time. But it creates a gap: nobody is watching the slow drift toward misconfiguration, overspend, and compliance debt. By the time these surface, they surface as incidents.

I spent the last few months building something to close that gap, and I want to share both the tool and the thinking behind it.

The problem with reactive operations

When a 10-person startup hits its first real outage, the postmortem usually reveals the same pattern: a quota was at 94% and nobody checked, a Network Security Group rule was too broad and nobody audited, a storage account had public access enabled six months ago during a debugging session and nobody reverted it.

The fixes are trivial. The problem is that nobody had the bandwidth to look. Traditional monitoring covers the "is it up?" question well. It does not cover "are we accumulating risk?"

Azure SRE Agent was built to fill exactly this space. It runs proactive, scheduled checks against your infrastructure and surfaces findings before they become pages. The built-in skills cover common scenarios, but every environment has its own shape. A fintech startup cares about different things than a gaming company.

Custom skills: teaching the agent your priorities

SRE Agent supports custom skills, which are structured prompts that tell the agent what to check, how to check it, and how to present findings. Think of them as runbooks that execute themselves on a schedule.

I built a pack of 8 skills that cover the gaps I kept seeing across startup engagements:

A Well-Architected Framework review that scores all 5 pillars and flags the weakest one
A compliance audit covering RBAC drift, missing resource locks, inconsistent tagging
Capacity planning that checks quota utilization and projects when you will hit limits
FinOps analysis combining cost optimization with chargeback allocation
Blameless postmortem generation using the 5 Whys method
Defender for Cloud Secure Score monitoring with prioritized remediation
A governance maturity assessment designed specifically for Digital Native companies
An AI Foundry and Azure OpenAI posture check for teams running LLM workloads

Each skill runs read-only. No write operations, no deployments, no changes to your environment. The agent reads your infrastructure state and produces a scored report with specific remediation commands you can choose to run or ignore.

Design decisions that matter

A few choices shaped how these skills work, and I think they are worth explaining because they apply to anyone building custom skills.

Scoring over pass/fail

Every skill produces a numeric score. A binary "compliant / not compliant" result is less useful than knowing you are at 41/100 on governance maturity and your weakest area is identity management. Scores let you track improvement over time and prioritize effort.

Inline documentation links

Each finding includes a link to the relevant Microsoft Learn page. When the agent tells you that your Azure OpenAI deployment lacks a private endpoint, the remediation section includes the exact az CLI command and a link to the documentation explaining why it matters. The goal is that an engineer can go from finding to fix without opening a browser to search.

Tiered scheduling

Not all checks need the same frequency. The capacity planning skill should run daily because quotas can spike fast. The Well-Architected review is a weekly check because architectural drift happens slowly. I organized the 8 skills into three priority tiers so teams can configure schedules that match the volatility of each risk category.

No write operations, ever

This was a hard line from day one. These skills read infrastructure state through Azure Resource Graph, Azure CLI queries, and REST API calls. They never create, modify, or delete resources. The remediation commands appear in the output as suggestions for a human to review and execute. This constraint makes the skills safe to run on production subscriptions without additional approval workflows.

What the output looks like

Here is a real example from the Digital Native Governance skill running against a test subscription.

The agent checks 6 governance categories: Alerting, Subscription Topology, Cost Controls, Identity, Web Protection, and Operational Foundations. Each category gets a score against its maximum, and the total determines a maturity level (Foundation, Developing, Established, or Optimized).

In my test run, the subscription scored 41/100, placing it at "Developing" maturity. Web Protection scored 0/15 because there was no WAF, no custom domain, no DDoS protection. Alerting scored 3/20 because Service Health and Resource Health alerts were missing entirely. That is specific enough to act on immediately.

The FinOps skill found ~$43/month in recoverable waste on a subscription that costs about $177/month. Empty container registries, deallocated VMs still paying for disks and public IPs, overprovisioned Defender plans. Not a huge number, but for a pre-seed startup running lean, that is real money sitting idle.

And here is what the Well-Architected review looks like when it scores all five pillars:

Lessons from building this

Three things I did not expect when I started.

First, prompt structure matters more than prompt length. Early versions of these skills were verbose, with long explanations of each check. The agent performed better with terse, structured instructions that clearly separated what to check, how to check it, and how to format the result. Clarity beats volume.

Second, the LLM behind the agent occasionally rendered emoji shortcodes (:red_circle:) as literal text instead of Unicode characters. This sounds minor but it made the output hard to scan visually. The fix was a single line in the skill definition: "Use Unicode emoji characters directly, never use emoji shortcodes." Explicit formatting instructions prevent the model from making stylistic choices that degrade readability.

Third, scoring philosophy requires intentional calibration. If you weight security checks too heavily, every subscription looks terrible and the report loses signal. If you weight them too lightly, teams ignore real risk. I landed on roughly 35% security, 30% reliability, 25% cost, and 10% architecture for the AI workload skill after testing against several real environments and seeing which distributions produced actionable variance between "good" and "needs work" subscriptions.

How to use this

The skills are open-source at github.com/ricmmartins/azure-sre-agent-skills. Each skill is a single Markdown file you paste into the SRE Agent portal at sre.azure.com. Setup takes about five minutes per skill.

You do not need all eight. If your startup is pre-revenue and running a single subscription, start with Capacity Planning and FinOps. If you are preparing for an enterprise customer's security questionnaire, start with Compliance and Defender Secure Score. Pick the two or three that match your current pain.

The skills are MIT-licensed and designed to be forked. If your compliance requirements include specific tagging conventions or naming standards, edit the skill to check for those. The structure is documented so you can build your own from scratch if none of mine fit.

How this differs from Azure Advisor

The first question people ask: why not just use Azure Advisor?

Advisor gives you a flat list of recommendations per resource. It tells you "this VM could be smaller" or "enable HTTPS on this App Service." Useful, but limited in three ways.

First, Advisor has no concept of correlation. It does not know that your failing health probe, your missing Resource Health alert, and your absent backup vault are three symptoms of the same problem: nobody set up operational foundations for that workload. These skills connect findings across domains and produce a single maturity score that tells you where you actually stand.

Second, Advisor does not generate narrative reports. It exports a CSV of individual recommendations, but there is no scored document with correlated findings, maturity levels, and prioritized remediation commands that you can forward to a CTO or attach to a security questionnaire. These skills produce that artifact directly in the chat, ready to copy or download.

Third, Advisor is passive. You go to the portal and look at it. SRE Agent runs your skills on a schedule and surfaces problems before they surface themselves as incidents. The difference between "there is a recommendation waiting in the portal" and "the agent flagged a quota at 85% utilization this morning" is the difference between a suggestion and a safety net.

Think of Advisor as a linter. These skills are closer to a staff SRE who reads everything, correlates the findings, and writes you a report with priorities and az CLI commands ready to paste.

What this is not

This is not a replacement for proper observability, incident management, or SRE staffing. When your startup reaches the scale where you need a dedicated SRE function, you should build one.

What these skills give you is coverage during the phase when you do not have that function yet. They catch the slow-moving risks that slip through the cracks when everyone is heads-down shipping features. Think of them as a part-time auditor who checks in daily and tells you what drifted since yesterday.

For startups on Azure, that gap between "we should probably check our security posture" and "we have someone whose job it is to check our security posture" can last years. These skills are one way to fill it.

Scaling - Startup Onboarding - Microsoft Family Company

Andamiro — Wed, 22 Jul 2026 08:06:55 GMT

Hi,

I'm wondering if I could get paired with an advisor.

Since we joined Microsoft as a corporation, in 2015, we have been developing a gaming console, we paused development with the defunction of BizSpark and later Microsoft Ventures was tiered up. However, 10 years later, we joined under a new gaming division that also creates software.

Both corporations, Greentwip, whose R&D is part of Microsoft now, and Cybertwip, whose focus is Energy and Games, which makes us a bit esoteric, is at the stage of sales, but we are unable to join the venture network, either by Azure (the platform leads to errors and other not found pages).

Let's start the thread here, and openly showcase.

To start with, we have Andamiro and Titus Interactive under our active assets, and we're partnering with SpaceX, and other undisclosed corporations that are of Microsoft interest and brokerage.

Along the development, we have a few projects for Xbox, and the Virtua console being manufactured by Shenzhen QBuy, I'll post the demos here in the thread.

We have notified the Microsoft board, and so forth. Yeah, but I feel we need to showcase our progress.

So far:

The MediaTek partner has given us Kernels, and drivers, although some are incomplete we managed to get Intel and ARM processors to work, only Intel has internet, no WiFi drivers yet, but the OS if fully compliant, at this stage is implementing Windows applications execution.

The support we have for the MVII OS is cross platform, it runs on Intel and ARM hardware, except for Macintosh.

A few games are natively being ported, such as Evil Zone, Top Gun, to mention a few, we managed to port Street Fighter EX Plus Alpha, and other titles that are considered Lost Media, or whose hardware vendors have been discontinued.

This is important to keypoint, as sales on perpetual media, are to be preserved, instead of just cutting support from.

We at Cybertwip consider that other corporations, such as Sony Corporation, which, when requested; whose source code was not delivered, including our games or partner corporations, must be preserved.

AI development, PAccel is one the AI tools we have, able to run 3x faster than onnxruntime and llama.cpp, but anyways, we demo here:

Titus Interactive signature game, Evil Zone, recovered Lost Media

Example of recovered Lost Media games, Street Fighter EX Plus AlphaNeoGeo Hub store, via SNK partnership and brokerageMVII launchingMVII running on MediaTek ARM hardware

Your H100s on Azure won't do distributed training. Here's probably why.

rmmartins — Thu, 16 Jul 2026 21:39:28 GMT

I recently had to dig into why a customer's H100 VMs couldn't do distributed training on Azure. They had ND-series capacity allocated, the hardware was there, but the VMs weren't connected over InfiniBand. The cause turned out to be a deployment configuration issue, not a hardware problem. Finding a single clear explanation of what went wrong and how to fix it was harder than it should have been.

The cause comes down to one of two things. Either they're on the wrong H100 SKU, or their VMs aren't in the same placement group. Both are fixable, but you have to understand what went wrong first.

The naming trap

Azure has two H100 VM families. The names look similar enough that people pick the wrong one without realizing it.

	NC40ads H100 v5	ND96isr H100 v5
GPUs	1x H100 NVL (94 GB)	8x H100 SXM (80 GB each)
InfiniBand	None	8x 400 Gb/s (3.2 Tbps per VM)
GPUDirect RDMA	No	Yes
Intra-VM GPU link	N/A (single GPU)	NVLink 4.0 + NVSwitch
Network bandwidth	40 Gbps Ethernet	80 Gbps Ethernet + InfiniBand
vCPUs	40	96
RAM	320 GiB	1,900 GiB

The NCads H100 v5 family has two sizes: NC40ads (1 GPU, 40 Gbps) and NC80adis (2 GPUs, 80 Gbps). Neither has InfiniBand. If you deploy 10 of these and try to run a multi-node training job, the GPUs communicate over standard Ethernet. For the all-reduce operations that distributed training depends on, you're looking at 40 Gbps per VM versus 3.2 Tbps aggregate IB on a single ND node, roughly 80x less bandwidth. In practice the gap is worse because Ethernet lacks GPUDirect RDMA, so every transfer bounces through main memory and the CPU.

The ND is the training machine. Eight H100 SXM GPUs per VM connected by NVLink internally, and each GPU gets its own dedicated 400 Gb/s InfiniBand link for talking to GPUs on other VMs. With GPUDirect RDMA, NCCL sends data directly between GPU memory across nodes, bypassing the CPU entirely.

If your training job needs more GPUs than fit in one VM, you need ND. No workaround for this.

When InfiniBand matters (and when it doesn't)

Not every GPU workload needs IB.

Single-GPU inference or dev work is fine on NC. No inter-node communication involved.

LoRA and QLoRA fine-tuning up to roughly 70B usually fits inside a single ND VM (8 GPUs with model parallelism). You're not crossing VM boundaries, so IB isn't in the picture.

Full-parameter SFT or RLHF on 8B+ is where it gets interesting. An 8B model in bf16 is about 16 GB just for the parameters. Full-parameter Adam training typically requires 12 to 16 bytes per parameter when you account for fp32 master weights, gradients, and optimizer first/second moments. That puts an unsharded 8B run at 96 to 128 GB before activations even enter the picture. With FSDP or ZeRO-3 sharding across the 8 GPUs inside a single ND VM, it fits without crossing node boundaries. But if you're trying to do this on standalone VMs without a sharding strategy, you'll need multi-node, and that means fast interconnect.

Large-scale pre-training at 70B+ runs across dozens or hundreds of GPUs. InfiniBand is not optional at this scale. Gradient synchronization over Ethernet would make the job impractical.

Rule of thumb: if your training job uses more GPUs than fit in one VM, InfiniBand starts mattering. More than two VMs, it matters a lot.

Making the VMs talk to each other

Having ND VMs in the same region isn't enough. They need to be on the same InfiniBand fabric to communicate at full speed.

Azure handles this through placement groups. A placement group is a set of VMs deployed on the same network fabric segment. VMs in the same group get non-blocking, full-bisection IB connectivity. VMs in different placement groups, or standalone VMs not in any VMSS or availability set, do not get InfiniBand communication with each other at all.

What happens when IB isn't available depends on your NCCL configuration. If NCCL_IB_DISABLE=1 is set, or NCCL can't find IB devices, it falls back to TCP over Ethernet (slow but functional). If IB interfaces exist but aren't connected to peers (the standalone ND case), NCCL may hang indefinitely waiting to establish the IB connection. Set NCCL_DEBUG=INFO in your environment to see which transport NCCL is actually using and where it gets stuck.

The following diagram illustrates this specific scenario: same ND hardware, deployed differently, completely different outcomes.

To get your NDs into a single placement group, deploy them in a Virtual Machine Scale Set (VMSS) with singlePlacementGroup set to true:

{ "type": "Microsoft.Compute/virtualMachineScaleSets", "properties": { "singlePlacementGroup": true } }

This works with both Flexible and Uniform orchestration modes (the IB setup docs confirm support for both). The key constraint is the placement group, not the orchestration mode. With singlePlacementGroup=true, the scale set is capped at 100 VMs by default (expandable to 300 via support request). For most startup training jobs, even the default cap gives you 800 H100 GPUs, which is plenty for post-training and medium-scale pre-training. If you need more than 2,400 GPUs on a single IB fabric, that's a different conversation involving Azure HPC team engagement.

The important thing: setting singlePlacementGroup=false (which enables large scale sets beyond 100 VMs) breaks InfiniBand networking entirely. The placement groups documentation states this explicitly: "Large scale (SPG=false) does not support InfiniBand networking." VMs land in different placement groups, and IB does not work across groups. If you need more than 100 nodes for IB training, open a support request to raise the single placement group limit rather than switching to multi-placement-group mode.

Once your NDs are in the same placement group, InfiniBand is automatically configured between VMs. No manual fabric setup required. The ND H100 v5 documentation confirms: "These connections are automatically configured between VMs occupying the same virtual machine scale set, and support GPU Direct RDMA."

Contiguous capacity has to be available in the region at deployment time. If the region is fragmented, the deployment might fail even though individual VMs would succeed. That's a capacity planning conversation, not a technical bug.

You can also use an availability set for the same purpose. VMs in the same availability set get IB connectivity. VMSS is usually the better option because it gives you scaling and lifecycle management, but availability sets work if you're managing VMs individually.

Docs: Set up InfiniBand on HPC VMs | Placement groups

You can't retrofit this without downtime

The natural follow-up question: "We already have ND VMs running as standalone instances. Can we rearrange them into a contiguous IB cluster without downtime?"

No. Azure doesn't offer a customer-facing operation to reposition VMs onto contiguous physical hardware. These H100 VM families do not support live migration, so there's no mechanism (customer-initiated or platform-initiated) that moves a running VM to a different physical host for network topology purposes.

The path is: deallocate your existing ND VMs, create a VMSS with singlePlacementGroup=true, and redeploy into the new scale set.

This means downtime. It also means contiguous capacity needs to be available in the region at that moment. If you're early in your deployment (and most startups are), set this up correctly from the start. Retrofitting later is possible but painful, especially if your region is capacity constrained.

The worst-case scenario here is real: you deallocate your VMs, try to redeploy into a placement group, and the region doesn't have enough contiguous rack space. Now you have no training cluster running and you're waiting for capacity to free up. If you're in a tight region, coordinate with your Microsoft account team to pre-validate contiguous availability before scheduling the maintenance window.

Images and drivers

Azure publishes HPC marketplace images (Ubuntu-HPC, AlmaLinux-HPC) that ship with NVIDIA GPU drivers, CUDA, NCCL, and Mellanox OFED pre-installed. Use these. They save you a day of driver debugging.

If you need a custom image or container, you'll need NVIDIA GPU drivers matching your CUDA version, Mellanox OFED for InfiniBand, and NCCL built with IB and GPUDirect RDMA support.

After deployment, validate the setup:

ibstat should show IB devices in Active state on every VM. Here's what success looks like:

CA 'mlx5_0' CA type: MT4129 Port 1: State: Active Physical state: LinkUp Rate: 400
If you see State: Down or Physical state: Polling, the IB link isn't established. Check that your VMs are actually in the same placement group.
lspci | grep Mellanox confirms the hardware is visible to the OS. You should see 8 ConnectX-7 devices on an ND96isr.
Run NCCL all-reduce tests across nodes with NCCL_DEBUG=INFO set:

export NCCL_DEBUG=INFO export NCCL_IB_DISABLE=0 mpirun -np 16 --hostfile hosts nccl-tests/build/all_reduce_perf -b 8 -e 8G -f 2 -g 1

With GPUDirect RDMA working correctly, you should see bus bandwidth approaching 380+ Gb/s for large messages (8 GB+). If you're seeing numbers well below that, or NCCL logs show NET/Socket instead of NET/IB, check that OFED is loaded (ofed_info), the IB interface is up (ibstat), and your VMs are in the same placement group.

Docs: Configure HPC VMs | ND H100 v5 specs

Before you start training

A quick checklist before launching your first distributed job:

You're on ND96isr H100 v5, not NC40ads H100 v5
VMs are in a VMSS with singlePlacementGroup=true (or same availability set). Standalone VMs won't have IB connectivity with each other.
Using Ubuntu-HPC or AlmaLinux-HPC image (or custom with OFED + NCCL)
ibstat shows IB Active on all nodes (all 8 ports per VM)
NCCL all-reduce test confirms expected bandwidth for large messages
NCCL logs show NET/IB transport, not NET/Socket
Training framework configured with backend='nccl'

Azure Service Health has four alert types. You probably configured one.

rmmartins — Wed, 15 Jul 2026 22:26:55 GMT

Most startups I work with have Service Health alerts configured on at least one subscription. That part's fine. The problem is they usually have it set up for Service Issues only, on their main subscription, while they're running six or seven.

Service Issues is the "Azure is down" alert. You want it. But it's one of four notification types, and the other three are the ones that catch the problems you could have seen coming.

The four types

Azure Service Health sends four categories of alertable notifications. (The portal also shows a fifth category, Billing updates, but that one isn't available as an alert rule trigger.) Here's what each alertable type looks like in the portal:

Service Issues are outages and degraded performance happening right now. This is what most people set up. It tells you something is already broken.

Health Advisories are about changes coming to your services. API deprecations, model retirements, required configuration changes, feature removals. For AI startups running Azure OpenAI, this is where model version retirement notices show up. If you're not subscribed, you find out when your inference calls start returning 410 Gone.

Planned Maintenance covers scheduled maintenance windows that could impact your services. Azure gives advance notice so you can prepare. If you're running GPU training jobs that take 48 hours, knowing about a maintenance window before you start the job matters.

Security Advisories cover vulnerabilities or security incidents affecting services you use. Critical patches, recommended actions, risk assessments for your deployment.

Most startups configure the first one and skip the rest.

"But won't I get spammed?"

The most common pushback when I suggest enabling all four types: "There are 250+ services in the filter list. If I select all of them, won't I drown in alerts?"

No. Service Health only fires notifications when Microsoft publishes an event affecting your subscription, for services you have deployed, in regions where you have resources. If you're running Container Apps, Key Vault, Cognitive Services, and Storage in East US, you get alerts for those. Everything else stays quiet.

I pulled the service list for a customer with 7 subscriptions recently. Across all of them, they had 23 resource types deployed. That's a very manageable alert volume, even with all four event types enabled.

The scope trap in the portal

This one trips people up. When you create an activity log alert rule, the portal shows a "Resource group" field that looks like it's required. That field is where the alert rule resource itself gets stored, not what it monitors. Service Health alerts are scoped at the subscription level.

You don't need a separate alert rule for each resource group. One rule per subscription covers everything in it. If you already have a Service Issues alert working, you set the scope correctly the first time. Edit that existing rule to enable all four event types, or create a new one and disable the old Service Issues-only rule (otherwise Service Issues fires twice).

Docs: Create activity log alert rule

You probably have more subscriptions than you think

Dev, staging, prod, shared services, maybe MSDN subs for engineers. Each subscription needs its own Service Health alert rule. Alerts on prod don't cover dev.

If you have 7 subscriptions and alerts on 1, that's 86% of your environment running without health notifications.

You can fix this at scale with Azure Policy. Microsoft provides a built-in policy called "Configure subscriptions to enable Service Health Monitoring Alert Rules." One policy assignment at the management group level, all current and future subscriptions get alerts automatically. One thing to note: for subscriptions that already exist at the time of assignment, you need to create a remediation task. Without it, the policy only evaluates newly non-compliant resources going forward.

Docs: Deploy Service Health alerts at scale using Azure Policy

Why this matters more for AI startups

If you're running Azure OpenAI in production, Health Advisories are where model retirement notices land. Azure periodically retires model versions and updates API versions. Without the advisory alert, your team finds out when something breaks in production.

Same story for GPU VMs. If you have long-running training jobs on ND or NC series, a maintenance window you didn't know about can kill a job that's been running for 3 days. Planned Maintenance alerts give you advance notice to checkpoint or reschedule. Azure typically provides about a 35-day self-service window for maintenance that requires a reboot. Within that window, you choose when to apply it. For more granular in-VM awareness, use Scheduled Events, which gives your workload about 15 minutes notice before a reboot actually happens. Long-running training jobs should checkpoint periodically regardless, but knowing the window exists lets you plan around it.

For security-conscious customers (and if you're handling inference at scale, you should be), Security Advisories flag vulnerabilities in services you're actually using. Not theoretical risks across all of Azure, just the ones relevant to your deployment.

Setup checklist

Open Azure Monitor, go to Alerts, create a new alert rule:

Select Service Health as the signal type:

Then configure:

Set scope to your subscription (not a resource group)
Signal type: Activity log, then Service Health
Enable all four event types: Service Issues, Health Advisories, Planned Maintenance, Security Advisories
Select all services (you'll only get alerts for ones deployed in your subscription/region)
Create an action group: email, Teams webhook, or both
Repeat for every subscription (or just disable the old rule if updating)

Or skip the manual work: assign the built-in Azure Policy at your management group level and let it propagate. Remember to create a remediation task for existing subscriptions.

Business verification auto failed after 5 minutes, can not find the founders hub or support in Azure

Starloop — Mon, 13 Jul 2026 13:53:51 GMT

Hi team,

Would really appreciate your help here 🙏

We applied for Microsoft for Startups Level 2 and my business verification was auto rejected.

I entered my Australian Business Number (ABN). The form asked for a 21 character alphanumeric entry, which doesn't match Australia's standard 11 digit format. I adjusted my entry to fit by adding 0's at the start.

Now it is telling me that I will move to pay as you go after the inital level of credits expire. One of the reasons I've paid to get a business registered was so we could use level 2 credits to keep building. I have been looking for hours but can not find any way to lodge a ticket or speak to someone. I messages Azure support on X and they said they can not help.

Does anyone have any guidance here? It would be very much appreciated.

Best regards,

Jake

Azure OpenAI quota approved but GPT-5.5 still shows 0 quota in Azure AI Foundry (Founders Hub)

josh_intero — Fri, 10 Jul 2026 14:20:18 GMT

Hi everyone,

I'm hoping someone from the Microsoft for Startups or Azure AI team can help point me in the right direction. I've reached a point where I believe this is a backend provisioning issue rather than a quota request issue.

Environment

Microsoft for Startups Founders Hub subscription
Active Founders Hub Azure credits
Azure AI Foundry
Azure OpenAI resource in East US 2
Deployment type: Global Standard

What happened

I submitted a quota request for GPT-5.5.
The first request was denied because I selected the wrong region.
I resubmitted for East US 2.
The request was approved.
I received an email stating my quota was increased to:
- 200 quota
- 200,000 TPM
- 600 RPM
However, Azure AI Foundry still shows 0 available quota for GPT-5.5.

Current behavior

GPT-5.5 appears in the Quota page.
Available quota is still 0.
I cannot create a deployment because no quota is available.
This has persisted after waiting and refreshing.

Additional information

My Founders Hub credits are active and linked to this subscription.
The subscription ID in the approval email matches the subscription I'm using.
Azure support suggested I was on a "free tier," but that doesn't appear consistent with an active Founders Hub sponsorship and an approved quota request.
This appears to be a case where the quota approval completed, but the quota was never actually applied to the subscription.

Has anyone seen this before, or is there someone from the Azure AI/OpenAI team who can verify the backend quota allocation?

I'd really appreciate any guidance or an internal escalation path.

Thank you!

Microsoft for Startups automated verification repeatedly fails for a legally registered company

AlexanderFaktorovitch-LYCI — Thu, 02 Jul 2026 15:53:49 GMT

I am the founder of LYCI GmbH, a Swiss company legally registered in the Commercial Register of the Canton of Zürich, Switzerland. My Microsoft for Startups business verification has failed on several attempts. All company details are accurate, publicly verifiable, and were checked and confirmed in my support ticket.

Support consulted their Engineering team and confirmed the startup verification process is fully automated by design. There is no team able to manually review, override, or approve verification requests, and no mechanism to identify or disclose which specific check failed. Engineering confirmed no manual verification or exception is possible, and the ticket was then closed.

This leaves a legally registered company, with all information publicly verifiable, no path forward through the standard channel.

Please find closed the ticket: Tracking ID [2606250050001977].

Request: Could a moderator please escalate this to the Microsoft for Startups program/verification team for manual review?

Cannot access startup verification flow

jhoffmanSpilledInc — Wed, 24 Jun 2026 21:36:58 GMT

I signed up for Azure startup credits (self-service path, no investor code) approximately 83 days ago and received my initial $1,000 in credits on an Azure Sponsorship subscription. My 90-day business verification window to unlock the additional $4,000 closes within the next week.

I'm writing in hope that either someone from Microsoft can get me in touch with the right team, or a fellow founder has located a viable workflow that I didn't.

What I've tried:

Founders Hub portal (portal.startups.microsoft.com): Redirects to azure portal.

Azure portal home: Shows a "Build your startup" page with an "Unlock $1,000 in Azure credits — Verify with LinkedIn" prompt. This is the initial $1,000 onboarding step, which I already completed months ago. The portal does not recognize that I am already enrolled. Clicking "Verify with LinkedIn" returns a {"error":{"code":"BadRequest","message":"BadRequest (Conflict)"}} response from startupsweb.production.portalrp.azure.com/api/auth/linkedin/callback.

Azure support ticket: Filed a ticket under Benefits - Offers / Entitlement check (ticket #2606240010000362). The T1 agent is not reading the content of my messages and keeps directing me back to the Founders Hub portal and the same broken links. He is attempting to close the ticket without resolution.

email address removed for privacy reasons: Received an auto-reply that this is an unmonitored inbox directing me to submit a ticket through the Founders Hub portal — which redirects to the aforementioned dead end workflow.

I believe this is related to the program experience migration that went live May 29, 2026. My account was created before this date, so per Microsoft's own documentation I should be on the pre-migration portal experience, but neither the old nor new experience is functional for my account. It seems like those of us with in-progress states are just getting lost in the shuffle of the migration.

From Error Log to Closed Ticket, Without Leaving Your Terminal

lovanartem — Mon, 15 Jun 2026 19:55:53 GMT

You describe the problem. The assistant pulls the context from what you already have, drafts the ticket, files it, tracks the replies, and closes it out, and it asks before it does anything irreversible. No portal tabs. No re-typing the resource ID Azure already knows.
See a quick demo or jump straight to the code: https://github.com/Azure-Samples/azure_support_ticket_mcp

The common challenge

Every team running on Azure eventually opens a support ticket. Here is the catch: the investigation happens in your terminal or editor, but the ticket frequently happens in a browser. Bridging those two worlds is pure overhead, and you pay it at the worst possible moment, mid-incident.

Opening one ticket means stepping through:

Confirm the tenant, pick the subscription.
Find the right support service among hundreds.
Drill down a per-service problem-classification tree.
Set severity, enter contact details, write up the issue.
Re-type the resource ID and error you were just looking at.

And filing is only the start. A support ticket is a conversation: replies to read, follow-up questions to answer, logs to attach, a status to flip when it resolves. Each one is another trip to the portal, another context switch out of the place you actually work.

The value proposition is simple: collapse that entire lifecycle into the place you already work, in plain language, in seconds. Stay in flow. Let the assistant do the mechanical parts and keep the decisions for yourself.

How the lifecycle works

Every ticket follows the same path, and the same safety gate sits in the middle of it. Opening a ticket runs left to right; once it exists, the same conversational interface carries it through the rest of its life.

The second lane is the part most ticket tools skip. Once a ticket is open, support is a two-way conversation, and the server handles that side too: read the full thread of customer and Microsoft replies, get a local summary of where things stand, reply to the support engineer, and attach logs, traces, or screenshots, all without returning to the portal.

That preview-then-confirm gate is the whole trust model. The first call returns exactly what is about to happen; the second call carries it out, and only if nothing changed in between. Reading, summarizing, and triaging are instant, with nothing to approve; creating, replying, attaching, and closing always pause for your yes.

The Solution

The Azure Support Ticket MCP is a Model Context Protocol (MCP) server that exposes the Azure support ticket lifecycle as a set of conversational tools. Three things make it more than a thin API wrapper:

It infers context. Give it a resource ID or a portal URL and it reads the subscription, resource group, and service from it, then ranks the right problem classification from your description instead of making you walk the tree.
It is local-first. The Azure support catalog, the services and their problem-classification trees, is cached on disk, so the common path is instant and works even on a flaky connection.
It is safe by design. Every action that changes something is preview-then-confirm: nothing reaches Azure until you approve the exact payload.

Getting started

Install the binary (a single command, the repo README has the current one-liner).
Register it with your MCP-capable assistant.
Start describing problems in plain language.

The fastest way in is to pipe a failure straight from your terminal into a ticket:

# a failed infra provision copilot -i "ticket this: $(azd up 2>&1)" # a misbehaving pod copilot -i "ticket this: $(kubectl describe pod my-pod)" # a red CI run copilot -i "ticket this: $(gh run view <run-id> --log-failed)"

From that raw output, the server extracts the resource IDs, error codes, correlation IDs, and HTTP status, takes a first guess at severity, and scrubs obvious secrets on a best-effort basis. You then review and approve the full draft before any ticket is created, so anything the scrub might miss is still in front of you to catch first. Prefer plain English? That works too:

copilot -i "open a ticket — my AKS cluster prod-aks can’t scale out"

In short

The problem: filing and managing Azure support tickets pulls you out of your flow into a multi-step portal process, again and again, over the life of each ticket.
MCP: an open standard that lets AI assistants take real, permissioned actions through tools, not just answer questions.
The solution: an open-source MCP server that runs the whole ticket lifecycle from your terminal, context-aware, local-first, and gated by preview-then-confirm.

Try it, or read the code

It is open source under the MIT license. A short demo, installation, the full capability list, and the design notes are all in the repository — issues, ideas, and pull requests welcome.

https://github.com/Azure-Samples/azure_support_ticket_mcp

Azure Startup Credit Issue

OzanTuranli — Tue, 09 Jun 2026 12:28:44 GMT

Hi, I've been trying to verify my LinkedIn account to unlock $1,000 in Azure startup credits for over a week. Every time the OAuth callback hits error and, I get: {"error":{"code":"BadRequest","message":"BadRequest (Conflict)"}}.

I've revoked all LinkedIn app permissions on both LinkedIn and Microsoft sides. I suspect that the issue is an orphaned LinkedIn identity record in your backend from a previous failed attempt. My support ticket has gone unanswered for over a week now and I also created a post in the learn community that went unanswered for days now.

Could you point me to the right person? Happy to provide any account details needed.

Urgent Guidance Needed - Azure Credits Exhausted During Active AI Infrastructure Scaling

PLIMSOLL — Mon, 18 May 2026 08:00:54 GMT

Hello Microsoft for Startups Community,

My name is Joseph Thomas, and I am currently building our startup through Microsoft for Startups Founders Hub. We have recently exhausted both our initial and extended Azure sponsorship allocations ($1,000 and the subsequent $5,000 sponsorship tier).

Our team is actively scaling AI infrastructure entirely on Azure, including Azure AI Foundry workloads, multi-model orchestration, inference pipelines, and cloud application infrastructure. As development and deployment activity accelerated, our Azure consumption grew significantly faster than originally projected.

We are now beginning to see active billing charges accrue, and I am working proactively to avoid disruption to our ongoing production and development environments. I have already submitted support requests through both the Founders Hub portal and Azure Billing Support.

I wanted to reach out to the community here to ask:

Has anyone successfully secured additional sponsorship credits or a sponsorship extension after exhausting their initial allocations?
What is the best escalation path for an AI startup heavily invested in Azure AI Foundry and Azure infrastructure?
Has anyone recently gone through the “Level Up” review or higher sponsorship-tier evaluation process, and if so, do you have any recommendations?

We are fully committed to building long term on Azure and would greatly appreciate any guidance, recommendations, or insight from other founders or Microsoft team members who have navigated this stage successfully.

Thank you for your time.

Joseph Thomas

Unable to access Founders Hub portal - M365 benefit redemption blocked

Macdmacd — Tue, 05 May 2026 21:03:41 GMT

I am an approved Founders Hub member with an active $1,000 Azure credit and I am unable to access the Founders Hub portal or redeem my Microsoft 365 Business Premium benefit.

I have already contacted Azure Billing Support regarding this issue, but I believe this falls under your team's remit rather than billing, so I am reaching out directly.

The issues I am experiencing:

1. portal.startups.microsoft.com/login only offers a "Log in with LinkedIn" option — my Founders Hub account is not linked to LinkedIn, so this does not work for me.

2. Clicking "Log in with your Microsoft account here" redirects me to portal.startups.microsoft.com/signup, which is the new user signup page — not a login page.

3. Navigating directly to portal.startups.microsoft.com also lands on the signup page with no way to manually enter my Microsoft account email.

All roads lead back to the same circular loop with no resolution.

My Founders Hub account is registered under a gmail not my custom domain. I am fully signed into the Azure Portal (portal.azure.com) with this account and can confirm my Azure credit is active.

What I need help with:

- Accessing my existing Founders Hub account

- Redeeming the Microsoft 365 Business Premium benefit

- Setting up a Microsoft 365 tenant so I can use Teams under my custom domain

Please advise on how to proceed.

Startup credits expired silently — GPT-5.5 charges — is there any path to resolution?

Noviamind — Thu, 30 Apr 2026 13:00:17 GMT

I've been in the Microsoft for Startups program since August 2025. My €5,000 credits expired on April 12 with no notification. I checked my entire mailbox and there is nothing from Microsoft about it.

On April 27, I deployed GPT-5.5 on Foundry — two days after Microsoft released it — still believing I was covered. I found out on April 29 by checking my own consumption data and shut everything down immediately.

I launched my AI startup in production two weeks ago. The invoice for April hasn't arrived yet but I estimate it at €4,000–5,000. This could end my company.

I've been through two levels of billing support and both say they cannot adjust charges. I've now escalated to the support manager.

My question: has anyone been through this and found a path to resolution? And does anyone have a contact at Microsoft for Startups with actual decision-making authority?

I'm not looking to debate the policy. I'm looking for a human at Microsoft who understands what "startup" means.

Can't login to Startup Portal

lspstartup — Wed, 29 Apr 2026 00:14:13 GMT

When I try to login using my email and password, I keep getting prompted to apply for the start up program. I'm using the same account I use for the Azure portal and it shows my proper credits.

How can I find out what's wrong?

The flat-subscription problem

rmmartins — Wed, 22 Apr 2026 19:06:12 GMT

A real design review: management groups, policies, break-glass accounts, and the five things I'd tweak before going to production.

Here's what I see at most startups when they first show up on Azure: one subscription, one Global Admin, everything in the same resource group, and everyone's an Owner.

That works when you have three engineers and one environment. It stops working around the time you have a production workload, a dev environment, shared infrastructure, and an engineer who accidentally deleted the wrong resource group on a Friday afternoon.

The next step is usually "let's create more subscriptions." That's the right instinct. But without management groups and policies tying them together, you end up with four subscriptions, four sets of inconsistent RBAC assignments, no shared tagging strategy, and no audit trail showing who deployed what.

If you're at this stage and want a starting point, the Startup-Scale Landing Zone gives you an opinionated Bicep template with management groups, policies, and RBAC already wired together. This post goes deeper: what happens when a team takes those concepts and customizes them for their own environment.

The design

A startup VP of Engineering sent me their proposed management group hierarchy and asked me to review it before going to production. They'd done their homework: read the Cloud Adoption Framework docs, researched config options, and put together a three-level hierarchy with specific policies and RBAC at each level.

Here's the breakdown:

Tenant Root Group is the automatic top-level MG that Azure creates in every tenant. Be very selective about what you assign here. Anything at this level affects every subscription you'll ever create, including ones that don't exist yet. Some organizations do assign enterprise-wide "must have" policies at root, but for a startup still figuring out its governance posture, keeping root clean and pushing baselines to a company MG one level down gives you more flexibility.

Company MG sits directly below and carries the baseline that applies to everything: required tags on all resources (env, owner, cost-center, app), allowed regions locked to three US regions, Defender for Cloud enabled everywhere, and all diagnostic logs routed to a central Log Analytics workspace. Engineering gets Reader at this level, so everyone can see everything but can't change anything by default.

Three child MGs below that:

Nonprod MG is the relaxed zone. Tags are audited but not denied, so engineers can experiment without being blocked by policy. Public IPs are allowed. Engineering gets Contributor. This is where you iterate fast without filing PIM requests.

Prod MG is the strict zone. Tags are denied if missing. Public IPs are blocked. Encryption at rest is required. VM SKUs are restricted. Engineering gets Reader by default, and Contributor access is available through PIM (just-in-time, time-limited activation). You have to explicitly request write access, and it expires.

Platform MG protects the shared infrastructure that everything depends on. The Terraform state storage account, central Log Analytics workspace, and shared Key Vault all live here. Platform team gets Contributor; everyone else gets Reader. Critical resources are protected from deletion.

Under each MG, the subscriptions:

MG	Subscription	Purpose
Nonprod	dev	Development and testing
Nonprod	devtest (MSDN)	Engineer's personal scratch (MSDN-bound)
Prod	prod	Production workloads
Platform	cloud-infra	Terraform state, Log Analytics, Key Vault, workload identity

The parts that nail it

The hierarchy is flat and functional. CAF says keep it three to four levels deep and don't create management groups just for the sake of structure. This design does exactly that: a company MG for baselines, then Nonprod/Prod/Platform for the policy gradient. It's not "the one CAF pattern" (CAF deliberately avoids prescribing a single topology), but it's a clean startup pattern that scales to dozens of subscriptions without restructuring.

Audit in dev, deny in prod. Dev environments that deny everything become unusable. Engineers stop experimenting. Prod environments that only audit become insecure. The split is the right trade-off: visibility without friction in dev, enforcement without exceptions in prod.

The platform subscription for shared services. Centralizing Terraform state, the Log Analytics workspace, and shared Key Vault into a separate subscription (with its own RBAC) means application teams can't accidentally delete the infrastructure that manages their infrastructure. This is the "trust boundary" pattern, and most startups skip it until they learn the hard way.

What i'd change before going live

PIM licensing isn't one-seat-fits-all. They mentioned having "1 P2 seat" for PIM. PIM requires an Entra ID P2 (or Governance) license per user who's eligible for activation, plus anyone who approves or reviews PIM access. If four engineers need just-in-time Contributor access to production and one manager approves, that's five P2 licenses (~$9/user/month). Still cheap insurance compared to "everyone has standing Contributor," but budget for it correctly.

Think about SKU restrictions as a trade-off. Their prod MG had "restrict to approved SKUs." An allow-list gives you strict standardization (only pre-approved SKUs work), but every time Azure launches a new VM series, someone has to update it. A deny-list ("block these specific expensive or unnecessary SKUs") is easier to maintain since new SKUs are available by default. The right choice depends on your team: if you need tight control over what runs in prod, keep the allow-list. If you move fast and want less policy maintenance, a deny-list with periodic reviews is simpler.

Resource locks beat policy for protecting critical infra. Their Platform MG had "deny deletion of state storage / log workspace" as a policy. Azure Resource Locks (CanNotDelete) are simpler and more visible for this. A lock shows up right on the resource in the portal, so engineers see it immediately. A deny-delete policy is invisible until it blocks you, and the error message doesn't always make it obvious why. Locks are also easier to temporarily remove when you legitimately need to rotate or replace a resource.

Add cost alerts on every subscription from day one. Their design didn't mention budget alerts. Azure Cost Management lets you set budget thresholds per subscription with email and webhook notifications. Set them before any workloads deploy, not after the first surprise bill. Start with 80% and 100% of expected monthly spend. It takes 5 minutes and can save thousands.

Cap the MSDN subscription. Their devtest sub was MSDN-bound, described as "personal scratch." MSDN subscriptions come with a monthly credit ($50-$150 depending on the license tier), but the spending limit can be removed, which means charges hit a valid payment method with no cap. Keep the spending limit ON for scratch subs. If it's been removed, set a budget alert at the credit amount. Also note that some Marketplace and external services may bill separately regardless of the spending limit.

The break-glass question

This team was federating their primary domain with Google Workspace as the SAML identity provider (their whole company runs on Google). They asked: "Can I use my .onmicrosoft.com account as a break-glass account while my federated company.com is my daily driver?"

Yes. This is exactly the pattern Microsoft recommends.

Microsoft's security benchmark (PA-5) specifically calls for cloud-only break-glass accounts that bypass external IdP dependencies. If your Google SAML federation goes down (Google outage, misconfigured SAML cert, domain issues), all federated accounts fail to sign in. Cloud-only .onmicrosoft.com accounts authenticate directly against Entra ID with no external dependency.

How to harden them:

Create two break-glass accounts. Microsoft recommends at least two. Store credentials in separate physical locations. One person alone shouldn't be able to access both. Docs: Manage emergency access accounts.

Use phishing-resistant auth. Passkeys (FIDO2 security keys) are the strongest option: phishing-resistant and no dependency on a phone or authenticator app that might be unavailable during an emergency. If you already run PKI, certificate-based auth is another viable option. The key is diversity across your two accounts so a single authentication method failure doesn't lock out both. Docs: Enable FIDO2 security key sign-in.

Exclude at least one account from ALL Conditional Access policies. This is the account that guarantees access if a bad CA policy locks everyone out. Microsoft recommends excluding at least one break-glass account from every CA policy. The second account can optionally have phishing-resistant MFA enforced via CA, giving you a safer fallback for non-federation emergencies.

Assign Global Administrator permanently. Not through PIM. Break-glass accounts need immediate access. PIM activation requires the normal auth flow, which defeats the purpose in an emergency.

Monitor every sign-in. Set up alerts in Azure Monitor or Microsoft Sentinel for any authentication from a break-glass account. If these accounts show activity outside an emergency, investigate immediately.

Test quarterly. Actually sign in with the break-glass accounts on a schedule. Verify the credentials work, the FIDO2 keys work, and the monitoring alert fires. Don't wait for a real emergency to discover something is broken.

The pre-production governance checklist

Before deploying workloads into your new hierarchy, verify:

All subscriptions are nested under the correct MG (not dangling under Tenant Root Group)
Baseline policies applied at the company MG and verified with Get-AzPolicyAssignment
PIM configured with appropriate activation duration (4-8 hours max)
P2 licenses assigned to every user eligible for PIM activation, plus approvers and reviewers
Two break-glass accounts exist, tested, and monitored
At least one break-glass account excluded from all Conditional Access policies
Budget alerts set on every subscription (80% and 100% thresholds)
Resource locks on Terraform state, Log Analytics workspace, and Key Vault
MSDN spending limit verified ON (or budget alert set if removed)
Diagnostic settings routing all activity logs to the central Log Analytics workspace

Where this fits in the governance journey

If you're building Azure governance from zero, here's my recommended reading order:

Demystifying Microsoft Entra ID, Tenants and Azure Subscriptions - understand what tenants, subscriptions, and Entra ID actually are
Azure has three permission systems, and you're probably confusing them - the identity, resource, and billing planes
This post - design your management group hierarchy
Role Structures, Anti-Patterns, and the 10 Governance Principles - RBAC patterns and what not to do
Introducing the Startup-Scale Landing Zone - the full reference architecture

Your Azure VM went down and nobody knew why. Here's how to fix that.

rmmartins — Wed, 22 Apr 2026 15:49:46 GMT

If you've ever had a production VM go unhealthy on Azure and found yourself scrambling to figure out what happened, you're not alone. I work with startups running production workloads on Azure, and this is one of the most common patterns I see: something goes wrong, the team opens a support ticket, and then everyone waits for a root cause while the CTO asks "how do we make sure we know about this before our customers do next time?"

The good news: Azure already gives you the tools to answer both questions. Most teams just haven't set them up yet.

Scope note: This post covers platform health and maintenance signals for Azure VMs. We're not covering guest OS metrics, application telemetry, or Azure Monitor/VM Insights here. If you don't have a dedicated SRE team, these are the highest-leverage Azure-native checks to set up first.

Let's get into it.

Step 1: Figure out what actually happened (Resource Health)

Before you open a support ticket, check Resource Health. It's the fastest way to determine whether your VM went down because of something Azure did (platform event) or something on your side (user-initiated or config issue).

Go to your VM in the Azure portal > Resource Health blade. You'll see:

Current status: Available, Unavailable, Degraded, or Unknown
Health history: 30 days of state transitions with annotations explaining each one
Root cause: For platform-initiated outages on VMs, Azure automatically publishes root cause details within 72 hours, directly in this blade

The annotations often tell you what kind of event occurred: live migration, host reboot, planned maintenance, degraded hardware, etc. In many cases, you get this information without filing a support ticket.

If your VM was affected by a live migration, the annotation will show it was a platform-initiated event. Live migration is a memory-preserving operation that causes a brief pause, typically no more than 5 seconds (docs). But if your application is sensitive to even short freezes, or if you're seeing them frequently, that's worth investigating further.

Docs: Resource Health overview

Step 2: Get notified when it happens (Service Health + Resource Health Alerts)

Checking the portal after an incident is fine. Getting an alert when the incident happens is better.

Service Health Alerts

These notify you about service issues, planned maintenance, health advisories, and security advisories for the Azure services and regions you're actually using. Service Health is best for subscription-level and region-level awareness. If there's a regional maintenance wave driving elevated live migrations, this is how you'd know about it proactively.

Set them up to notify your ops channel via email, SMS, webhook (Slack, PagerDuty, Teams), or automation via Logic Apps or Azure Functions.

Docs: Create Service Health alerts | PagerDuty integration

Resource Health Alerts

These fire when a specific resource (or all resources in a resource group) changes health status. The alert includes health-change details such as status, cause type (platform vs. user-initiated), and descriptive event text, so you get more than a generic "VM is unhealthy" notification.

This is the "never be surprised again" alert. If you only set up one thing from this post, make it this.

Docs: Create Resource Health alerts

Step 3: See it coming (Scheduled Events API)

This is the part most teams don't know about, and it's the most powerful tool for handling live migrations gracefully.

Azure exposes an Instance Metadata Service (IMDS) endpoint on every VM that gives your application advance notice of upcoming maintenance events. Live migrations show up as EventType: "Freeze". In typical cases, you get up to ~15 minutes between the event appearing and Azure proceeding with the operation, though exact timing varies and some failures (like hardware issues) can bypass the advance notification entirely.

Note: Most Azure VM families support live migration, but G, L, N, and H series VMs do not. If you run GPU or HPC workloads on these SKUs, you won't see Freeze events. You'll still get Reboot or Redeploy events for other maintenance types.

The endpoint is available from inside the VM at:

http://169.254.169.254/metadata/scheduledevents?api-version=2020-07-01

Here's an example response when a live migration is scheduled:

{
  "DocumentIncarnation": 1,
  "Events": [
    {
      "EventId": "602d9444-d2cd-49c7-8624-8643e7171297",
      "EventType": "Freeze",
      "ResourceType": "VirtualMachine",
      "Resources": ["my-production-vm"],
      "EventStatus": "Scheduled",
      "NotBefore": "Mon, 22 Apr 2026 19:17:47 GMT",
      "Description": "Virtual machine is being paused for a memory-preserving Live Migration operation.",
      "EventSource": "Platform",
      "DurationInSeconds": 5
    }
  ]
}

You can poll this endpoint and use the lead time to:

Drain connections so active users aren't affected
Checkpoint application state to recover faster
Remove the VM from your load balancer temporarily
Log the event so you have a record of migration frequency

Here's a simple polling script in Python:

import requests
import json
import time

ENDPOINT = "http://169.254.169.254/metadata/scheduledevents"
HEADERS = {"Metadata": "true"}
PARAMS = {"api-version": "2020-07-01"}

def get_scheduled_events():
    response = requests.get(ENDPOINT, headers=HEADERS, params=PARAMS)
    return response.json()

def handle_events(data):
    for event in data.get("Events", []):
        print(f"[{event['EventType']}] {event.get('Description', 'No description')}")
        print(f"  Status: {event['EventStatus']}, Not Before: {event['NotBefore']}")
        print(f"  Duration: {event['DurationInSeconds']}s, Source: {event['EventSource']}")
        # Your graceful drain/checkpoint logic here

def approve_event(event_id):
    """Acknowledge the event so Azure can proceed immediately."""
    payload = json.dumps({"StartRequests": [{"EventId": event_id}]})
    requests.post(ENDPOINT, headers=HEADERS, params=PARAMS, data=payload)

# Poll frequently - the official docs recommend every 1 second for production.
# Adjust based on your workload sensitivity.
while True:
    data = get_scheduled_events()
    handle_events(data)
    time.sleep(1)

Or a quick check in Bash:

curl -s -H "Metadata:true" --noproxy "*" \
  "http://169.254.169.254/metadata/scheduledevents?api-version=2020-07-01" | jq .

Event approval: Once your application has drained connections or checkpointed state, it can approve the event by POSTing back with the EventId. This tells Azure your app is ready, and the platform can proceed without waiting for the full timeout. If you don't explicitly approve, Azure proceeds when the NotBefore time is reached.

If you're seeing elevated frequency of live migrations, this data lets you quantify the pattern (how often, what times, what durations) and bring hard numbers to a support conversation instead of "it feels like it's happening a lot."

Docs: Scheduled Events for VMs

Step 4: Check your overall posture (Azure Advisor)

While you're at it, check Azure Advisor's Reliability recommendations for your VMs. It flags things like:

VMs not deployed in availability zones
Deprecated VM images that need updating
Missing backup configurations
Other resiliency gaps that make you more susceptible to availability issues

Advisor won't explain a past incident, but it can help prevent the next one.

Docs: Azure Advisor Reliability recommendations

A quick note on resilience

These tools improve your visibility and response time, but they don't eliminate downtime by themselves. If a VM is truly critical, pair this monitoring with basic resilience patterns: multiple instances behind a load balancer, availability zones, health probes, regular backups, and cross-region recovery where needed. Monitoring tells you what's happening. Architecture determines whether it matters.

The setup checklist

Quick wins (15 minutes)

#	What	Why	Time
1	Check Resource Health on your production VMs	See if there are past events you didn't know about	2 min
2	Create a Service Health alert for your regions/services	Get notified about platform issues proactively	3 min
3	Create Resource Health alerts for your VM resource groups	Get notified when any VM changes health state	3 min
4	Review Azure Advisor Reliability tab	Fix any posture gaps	2 min

Advanced hardening (1+ hours depending on your app)

#	What	Why
5	Deploy the Scheduled Events polling script on critical VMs	Get advance notice of live migrations and maintenance
6	Implement drain/checkpoint logic tied to Scheduled Events	Gracefully handle maintenance with zero user impact
7	Wire event approvals into your automation	Control the timing of when Azure proceeds with maintenance

Wrapping up

The pattern I keep seeing is teams treating Azure VM monitoring as something they'll get to "later." Then an incident happens, the RCA takes longer than anyone wants, and everyone wishes they had visibility sooner.

The tools are already there. Resource Health tells you what happened. Service Health and Resource Health alerts tell you when it's happening. Scheduled Events tells you before it happens. And Advisor helps you make sure your setup is resilient in the first place.

Fifteen minutes of setup for the quick wins, and you're in a fundamentally better place than most teams running VMs on Azure today.

$17,493 in Undisclosed Marketplace Charges with No Cost Visibility, No Recourse, No Accountability

chrisbaker2000 — Mon, 13 Apr 2026 14:01:05 GMT

I'm a co-founder of a 13-person startup in the Microsoft for Startups Founders Hub program. I'm posting here because after two months of support tickets, calls, and emails across both Microsoft and Anthropic, I have been unable to get anyone with decision-making authority to address this issue. I'm hoping this reaches someone at Microsoft who can help, and that other affected founders in the program will share their experiences.

What happened:

In February 2026, we deployed Claude Opus 4.6 and Sonnet 4.6 through Azure AI Foundry as part of a migration of our AI infrastructure. We are active users of our Azure sponsorship credits and assumed, as anyone would, that these models were covered the same way Azure OpenAI models are. There was no indication otherwise during deployment.

In early March, we received our first invoice: $1,078.07 (invoice G144899694, billing period 02/01–02/28). We were shocked, but we paid it immediately and removed all Anthropic model deployments from our account to prevent further charges.

It didn't matter. On April 9, we received a second invoice: $16,414.94 (invoice G151890529, billing period 03/01–03/31). Despite removing the deployments in mid-March, charges had already accumulated for the first half of the month. We are unable to pay this invoice. Our total exposure across both invoices is $17,493.01.

Why we had no way to prevent this:

No billing distinction at deployment. Azure AI Foundry presents all models (Microsoft-native and third-party) in the same unified interface. There is no warning, label, or confirmation step indicating that certain models are excluded from sponsorship credits.
No cost visibility whatsoever. The Azure AI Foundry monitoring dashboard has an "Estimated Cost" section that is completely blank for these models, with a disclaimer: "Cost monitoring is available for Foundry Models sold directly by Azure only." We could see token counts but had zero visibility into what we were being charged.
Token counts that don't explain the charges. The dashboards show our Claude Opus 4.6 deployment used 63.4M tokens and our Sonnet 4.6 deployments used roughly 170M tokens combined. At published rates, that should be in the low thousands, not $17,500. My analysis shows the dashboard hides billions of cached tokens (prompt caching reads and writes) that are invisible in the monitoring UI but account for the vast majority of the bill. There is no view in Azure that provides a breakdown of these charges by token type.
No alerts or notifications. There were no cost alerts, no threshold warnings, and no notifications at any point.
No indication in any Azure portal that charges were hitting our credit card. There was no line item, no pending charge, no Marketplace spend summary - nothing anywhere in the Azure ecosystem that showed dollars accumulating against our payment method for these deployments.

What happened when we asked for help:

Azure Support (TrackingID#2603090040002936): After a month-long wait, a support engineer told us Microsoft cannot issue credits for Marketplace charges and directed us to Anthropic. The first version of the response email referenced "Azure DDoS Protection Standard" instead of our actual issue, suggesting the volume of similar cases in the queue.
Anthropic: Their AI support bot responded within one minute with a blanket statement and closed the ticket four hours later. I escalated, but have still not received a response over a month later.
Microsoft for Startups Team (TrackingID#2604070040009778): Told us they cannot apply Marketplace charges against sponsorship credits and referred us to a Marketplace billing contact.
Azure Marketplace billing contact: Pending response.

The pattern is clear: Microsoft directs us to Anthropic. Anthropic directs us to Microsoft. The Microsoft for Startups team directs us to Azure Marketplace billing. No one takes responsibility.

What I'm asking for:

A full refund of $17,493.01 across invoices G144899694 and G151890529
That Microsoft implement clear billing warnings in Azure AI Foundry before deploying models that are excluded from sponsorship credits.
That Microsoft provide actual cost visibility in the monitoring dashboard for all models deployed through AI Foundry, not just first-party models.

To other founders in the program:

If you have experienced this same issue, please reply to this thread. I know I'm not alone because this has been covered by The Register, InfoWorld, and Computerworld, and at least 20 founders have signed a Change.org petition about it. The more founders who come forward with specific amounts and case numbers, the harder this is to ignore.

We joined Microsoft for Startups because the program was supposed to help early-stage companies manage infrastructure costs during the most financially vulnerable period of our growth. Instead, the program's own platform generated charges within 2 weeks that exceed the total sponsorship credits we've consumed over the past year, with no visibility, no warning, and no path to resolution.

With Further Inc. | Microsoft for Startups Founders Hub

Azure Support TrackingID #2603090040002936

Startup Support TrackingID #2604070040009778

Role Structures, Anti-Patterns, and the 10 Governance Principles

rmmartins — Thu, 09 Apr 2026 21:25:00 GMT

Part 3 of 3: The implementation playbook for engineering, finance, and security teams

In Part 1, we established Azure's three-plane model: Entra for identity, RBAC for resources, Commerce for billing. In Part 2, we explored where those planes collide: Marketplace governance, Managed Identity, and ABAC.

Now it's time to get practical. This post covers the patterns that work, the anti-patterns that don't, and the governance principles that every digital-native company should adopt before they're forced to adopt them after an incident.

7 anti-patterns to avoid

These seven anti-patterns appear repeatedly across AI, SaaS, and digital-native customers. Every one of them has caused real incidents — surprise invoices, accidental deletions, compliance failures, or governance breakdowns.

❌ Anti-Pattern 1: Giving engineers billing permissions

What happens: Engineers are given Billing Reader or Billing Contributor roles "so they can see costs." They can now see MACC credits, private offer terms, commercial discounts, and Marketplace purchase history, none of which they need.

Symptoms: Engineers purchasing Marketplace SaaS without oversight. Surprise invoices. Procurement loses visibility into vendor commitments.

Fix: Engineers need Cost Management Reader (RBAC) for usage-based cost visibility. They do not need billing roles. If they need to understand MACC impact, create a reporting process, don't give them the keys.

❌ Anti-Pattern 2: Giving finance subscription owner access

What happens: Finance teams are given Owner or Contributor roles on subscriptions "so they can track spending." They now have the ability to deploy, modify, and delete production resources.

Symptoms: Massive over-permissioning. Finance can accidentally delete production resources. Audit risk, regulators will flag this.

Fix: Finance roles belong in the Billing plane, not the resource plane. Give finance Billing Reader for credit and invoice visibility. If they also need resource cost data, add Cost Management Reader (RBAC) scoped to the appropriate subscriptions — that's a read-only, resource-plane role.

❌ Anti-Pattern 3: Too many subscription owners

What happens: Every senior engineer, team lead, and sometimes product managers get Owner on subscriptions. The logic: "they need to unblock themselves."

Symptoms: No accountability, when everyone is Owner, nobody is. High blast radius. Hard to trace role assignments when troubleshooting.

Fix: Maximum 2–3 Owners per subscription: Platform Lead, SRE Lead, and optionally the Cloud Architect. Everyone else gets Contributor or scoped roles. Use PIM for emergency elevation.

❌ Anti-Pattern 4: Believing Entra Global Admin = Azure Owner

What happens: Leadership assumes Global Admin has universal access: subscriptions, resources, billing. They don't. Global Admin controls the identity plane only.

Symptoms: Security teams thinking they can see all resources (they can't). Incorrect governance designs that assume Entra = RBAC.

Fix: Train leadership explicitly: Entra ≠ RBAC ≠ Billing. Three planes, three sets of roles, zero overlap. A Global Admin who needs resource access must be separately granted RBAC roles.

❌ Anti-Pattern 5: Deploying marketplace SaaS without finance

What happens: Engineers purchase Marketplace tools directly because they have billing permissions (see Anti-Pattern 1) or because the org hasn't restricted Marketplace purchases.

Symptoms: Incorrect MACC burn. Licensing duplicates. Vendor lock-in without legal review. Private offer terms not applied.

Fix: Require finance approval for all paid Marketplace purchases. Follow the five-step workflow from Part 2: Engineer requests → Finance reviews → Billing executes → Engineering deploys → Cost monitoring activated.

❌ Anti-Pattern 6: Mixed dev/test/prod in one subscription

What happens: To save time, teams put all environments in one subscription.

Symptoms: Can't isolate production costs. A Contributor on the sub can modify both dev and prod. Can't enforce stricter policies on prod without affecting dev. Compliance teams can't get clean boundaries.

Fix: Separate subscriptions by environment. Pattern: 1 subscription per environment per workload (or at minimum per environment). Use cross-subscription networking via Hub & Spoke or Landing Zones.

❌ Anti-Pattern 7: Not using Azure Policy

What happens: Teams deploy freely with no guardrails. Over time: VMs in unapproved regions, GPU SKUs in non-production, storage accounts without encryption, missing tags, public IP drift.

Symptoms: Inconsistent regions. Wrong VM families. Missing tags make cost attribution impossible. Non-compliant configurations.

Fix: Adopt Azure Policy early, at Management Group scope. Critical policies: allowed locations, allowed VM SKUs, enforce HTTPS, enforce private endpoints, enforce tagging (environment, owner, cost-center).

Recommended role structure

Based on experience with dozens of digital-native customers, here's the role structure that works across the three planes.

Engineering plane (RBAC)

2–3 subscription Owners: Platform Lead, SRE Lead, Cloud Architect
Platform/SRE team as Contributors: deploy and manage infrastructure
Developers as RG-scoped Contributors or Readers: limited to their workload's resource group
Cost Management Reader for budget owners: usage visibility without deployment rights
Azure Policy for guardrails: VM SKUs, regions, encryption, tags
Management Groups for organizational structure

Finance plane (Commerce)

Billing Account Owner = CFO or Finance Director
Billing Contributor = Finance Operations
Billing Reader = FP&A and financial analysts
All Marketplace-paid offers require finance approval
MACC visibility restricted to finance roles

Identity/Security plane (Entra)

2–4 Global Admins (break-glass accounts included)
PIM enforced for all privileged roles, no permanent admin access
Conditional Access for all admin roles (MFA, compliant device, block legacy auth)
Groups used for RBAC assignment, never assign RBAC to individual users
Workload identities (Managed Identity) preferred over service principals

Role mapping templates

Copy these into your onboarding documentation.

Engineering Team

Role	Azure Role	Plane	Allowed actions
Cloud Architect	Owner (2–3 per sub)	RBAC	Govern workloads, assign roles, manage infrastructure
Platform / SRE	Contributor	RBAC	Deploy and manage infrastructure
Developer	Contributor or Reader (RG-scoped)	RBAC	Deploy to specific resource groups
Budget Owner	Cost Management Reader	RBAC	View usage-based cost, manage budgets — not billing

Finance Team

Role	Azure Role	Plane	Allowed actions
Finance Lead	Billing Account Owner	Billing	View and manage credits, invoices, MACC, payment methods
Finance Analyst	Billing Reader	Billing	Read-only billing visibility
FP&A	Billing Reader	Billing	Read-only; no deployments, no resource access

Leadership

Role	Azure Role	Plane	Actions
CTO / VP Engineering	Reader or Cost Mgmt Reader	RBAC	Visibility into platform and resource costs
CFO	Billing Reader	Billing	Visibility into credits, invoices, MACC, commitments

RACI Matrix

Adapted from the Microsoft Cloud Adoption Framework.

Function	Accountable	Responsible	Consulted	Informed
Billing account roles & access	Finance Lead	Finance Ops	Cloud Architect	Engineering
Subscription role assignments	Cloud Architect	Platform / SRE	Finance, Security	Engineering
Cost monitoring & budgets	Finance	Engineering	Leadership	All teams
Marketplace purchases	Finance Lead	Finance Ops	Engineering, Legal	CFO
IaC / Deployment governance	Platform Lead	Engineers	Security	Finance
Policies & guardrails	Security / Cloud Architect	Platform Team	Engineering	Leadership
Identity & access governance	Security Lead	Identity Admin	Cloud Architect	All teams
PIM & Conditional Access	Security Lead	Identity Admin	Platform Lead	Engineering
MACC tracking & credit visibility	Finance Lead	Finance Ops	Cloud Architect	Leadership

Include this template in your onboarding documentation and review it quarterly.

Best Practices

Use Entra Groups for RBAC assignment, never assign directly to users

Benefits: clear separation of identity and resource planes, easy onboarding/offboarding, predictable RBAC inheritance, enables PIM for group-based elevation.

Naming pattern:

grp-sub-<SubscriptionName>-Owner
grp-sub-<SubscriptionName>-Contributor
grp-rg-<WorkloadName>-Reader

Assign the group to the role, not individual users.

Enforce PIM + Conditional Access for all privileged roles

Key CA policies: MFA required for all admins, compliant device requirement, block legacy authentication, block sign-in from high-risk locations, require phishing-resistant MFA.

No permanent admin access. Use time-based elevation for every privileged operation.

Separate subscriptions by environment and workload

Subscriptions are a security boundary. Pattern: 1 subscription per environment per workload. Platform teams get their own subscription. Use Hub & Spoke or Landing Zones for cross-subscription networking.

Keep billing data confidential

Only Billing roles should see credits, commitments, discounts, invoices, and MACC balance. Engineers should never have access to commercial data.

The 10 Principles of Azure Governance

After working with digital natives across AI, SaaS, and infrastructure companies, I can summarize Azure governance into these principles:

#	Principle	Summary
1	Separate identity, resources, and billing. Always.	Never mix roles across planes. An engineer should never hold billing roles. A finance analyst should never hold subscription Owner.
2	Engineering owns the resource plane.	Give them Contributor and Cost Management Reader. Don't burden them with billing or identity administration.
3	Finance owns the billing plane.	Credits, MACC, invoices, private offers. Every Marketplace purchase flows through Finance.
4	Security owns identity and governance.	PIM, Conditional Access, Azure Policy. Identity decisions should not be made by engineering or finance.
5	Keep subscription Owners scarce.	Maximum 2–3 per subscription. Use PIM for emergency elevation. Everyone else gets Contributor or scoped roles.
6	Lock down Marketplace.	Every SaaS purchase approved by Finance. No exceptions. Use the five-step workflow.
7	Use Infrastructure as Code.	Manual deployments don't scale and can't be audited. Use Bicep, Terraform, or Pulumi.
8	Use budgets early.	Set budgets at Management Group, Subscription, and Resource Group levels. Configure alerts to email, Teams, or automation.
9	Use Management Groups from day one.	Every startup that scales beyond a single subscription regrets not using them. Recommended hierarchy: Tenant Root → OrgName → Platform / Production / NonProduction / Sandbox / Shared Services.
10	Build governance before scale.	The companies that scale successfully treat Azure governance as infrastructure, not bureaucracy.

References

Closing thoughts

Azure's three permission planes aren't a problem to solve, they're a framework to leverage.

The confusion happens when teams try to treat Azure as if it has a single permission system. It doesn't, and it never will. Because identity, billing, and resource deployment are fundamentally different domains that must be operated and secured differently.

But when organizations understand these three planes and structure their roles accordingly, something powerful happens:

Engineering moves faster. Clear RBAC scopes mean teams deploy without waiting for approvals they don't need.
Finance gains real oversight. Billing roles provide full commercial visibility without the risk of touching production resources.
Security gets a clean, enforceable boundary model. Entra controls identity; PIM and Conditional Access control elevation; Azure Policy controls the guardrails.
Leadership sees clarity instead of chaos. The right roles in the right planes mean dashboards, reports, and alerts actually reflect what each stakeholder needs.

Good governance doesn't slow down innovation. Bad governance does.

The companies that scale successfully, whether AI-native, SaaS platforms, or global digital-first organizations, are the ones that adopt a clean, intentional model early. They treat Azure governance as infrastructure, not bureaucracy.

The model is simple: Entra for who. RBAC for what. Commerce for how you pay. Start with that, and everything else becomes easier.

This concludes the 3-part series on Azure Governance for Digital Natives. For the full model, start with Part 1: The Three Permission Planes. For collision points and Managed Identity, read Part 2: Marketplace Governance and the Cross-Plane Bridge.

rss.livelink.threads-in-node

From reactive firefighting to proactive operations: custom skills for Azure SRE Agent

The conversation that started it all

Azure SRE Agent: what it is (and what it isn't)

Custom skills: the extensibility model

The proactive operations skills pack

The skills at a glance

How they differ from Azure Advisor

Example: Digital Native Governance in action

Combining skills with scheduled investigations

Getting started

What's next

Next steps

Azure AI Foundry: from zero to production

The starting point

What is Azure AI Foundry?

The decisions you will face

1. Model selection

2. Model lifecycle

3. Deployment type: Standard, Priority, or PTU

4. Where Priority Processing fits

5. How PTU actually works

6. The cost math (this is where it gets interesting)

7. Spillover architecture

Part 2: production hardening

APIM as your AI Gateway

Reference architecture for agentic workloads

Anti-patterns to avoid

Production checklist

Security and network

Reliability

Observability

Cost governance

Model governance

The typical progression

Where to start

Next steps

Your startup doesn't have an SRE team. Now what?

The problem with reactive operations

Custom skills: teaching the agent your priorities

Design decisions that matter

Scoring over pass/fail

Inline documentation links

Tiered scheduling

No write operations, ever

What the output looks like

Lessons from building this

How to use this

How this differs from Azure Advisor

What this is not

Scaling - Startup Onboarding - Microsoft Family Company

Your H100s on Azure won't do distributed training. Here's probably why.

The naming trap

When InfiniBand matters (and when it doesn't)

Making the VMs talk to each other

You can't retrofit this without downtime

Images and drivers

Before you start training

Further reading

Azure Service Health has four alert types. You probably configured one.

The four types

"But won't I get spammed?"

The scope trap in the portal

You probably have more subscriptions than you think

Why this matters more for AI startups

Setup checklist

Further reading

Business verification auto failed after 5 minutes, can not find the founders hub or support in Azure

Azure OpenAI quota approved but GPT-5.5 still shows 0 quota in Azure AI Foundry (Founders Hub)

Microsoft for Startups automated verification repeatedly fails for a legally registered company

Cannot access startup verification flow

From Error Log to Closed Ticket, Without Leaving Your Terminal

The common challenge

How the lifecycle works

The Solution

Getting started

In short

Try it, or read the code

Azure Startup Credit Issue

Urgent Guidance Needed - Azure Credits Exhausted During Active AI Infrastructure Scaling