AI on Azure: platforms for apps and agents

Microsoft

Oct 30, 2026

AI on Azure series, Part 3 of 5

A model endpoint is one component of an AI application. The application still needs to decide how to retrieve information, call tools, handle user identity, moderate content, evaluate results, and recover when a dependency fails.

This article maps the main Azure building blocks around that application. They are layers you can combine, not competing products in one category. For model hosting, see Part 2: Models as a service. For training and serving models you own, see Part 4: Managed ML.

Product details were checked against Microsoft Learn on October 29, 2026. Preview labels and retirement notices matter; verify them again before a production decision.

Figure 1. Choose one runtime path. It can call the model and any optional services shown below; the application still owns access and evaluation.

1. Start with the application boundary

A basic AI application may call a model directly. Add an orchestration layer when the product needs multiple steps, tool calls, shared state, or an explicit workflow. Add retrieval when the model needs access to current or private information. Use a task-specific API when a focused service already does the job.

Microsoft Foundry brings model discovery and application-development capabilities into one platform. Foundry Agent Service provides managed runtime options for prompt agents, voice-based prompt agents, and hosted agents. A prompt agent keeps instructions and tools in a Foundry agent resource. A hosted agent lets you bring your own code and framework as a container while Foundry manages the endpoint, scaling, and identity. The service also documents tracing, metrics, and evaluations for observing agent behavior.

The managed runtime does not make the application design automatic. You still need to decide which tools an agent may call, which data it may retrieve, how a user is authorized, and what happens when the agent returns an incomplete or unsafe result.

2. Choose a runtime or a framework

These are different choices:

Foundry Agent Service is a managed place to run supported agents and connect them to Foundry models and tools.
Microsoft Agent Framework is a code framework for building agents and workflows. It is the direct successor to Semantic Kernel and AutoGen, according to its Learn overview.
Semantic Kernel is worth considering when you already have applications and integrations built with it. Microsoft's migration guides can help compare an existing implementation with Agent Framework; do not assume the transition is a drop-in package change.

The release status can differ by language and integration. The current Agent Framework overview marks Go as public preview, and its .NET quickstart uses prerelease packages. Check the specific runtime and provider before adopting it for a production system.

Use an agent when the task is open-ended and needs model-driven tool selection. Use an ordinary function or a deterministic workflow when the steps are known in advance. A language model does not improve a process just because it has been added to it.

The Azure Machine Learning overview says Prompt Flow is no longer recommended for new development and is scheduled to retire on April 20, 2027. If an existing project depends on it, review Microsoft's migration guidance rather than building a new workflow around it.

3. Ground answers with Azure AI Search

Retrieval-augmented generation, or RAG, lets an application find relevant material and provide it to a model as context. Azure AI Search supports both keyword and vector retrieval. A hybrid query combines them, which is useful when a user may phrase a question differently from the source document.

There are two patterns to understand:

Classic RAG gives the application direct control over indexing, query construction, result filtering, and the handoff to the model. It is a simpler path when the team needs a generally available feature set or already has orchestration code.
Agentic retrieval uses an LLM to plan multiple searches and return structured grounding results. The current Azure AI Search documentation marks this capability as Preview. It can suit complex conversational queries, but do not treat preview status as a production commitment.

Search quality depends on the data pipeline as much as the model. Decide how documents are chunked and updated, which fields are searchable, how permissions are enforced, and which results can be sent to the model. Retrieval is not a substitute for authorization.

4. Add content safety deliberately

Azure AI Content Safety provides APIs for moderating text and images. Prompt Shields can detect direct user prompt attacks and document attacks. They do not determine whether content is accurate or grounded. When you use the standalone API, your application decides whether to block, allow, or escalate the result. These checks do not replace application authorization, threat modeling, or human review for high-impact decisions.

Microsoft has announced that several Content Safety preview features will retire on April 30, 2027. The list includes groundedness detection, Analyze Multimodal, custom categories (rapid), connected or disconnected containers, and the Embedded Content Safety SDK. Before designing around one of these, read its retirement and migration guidance. Content Safety capabilities do not all have the same lifecycle status, so avoid describing every feature as a GA "guardrail."

5. Use task APIs for focused jobs

Not every AI feature needs a generative model. Foundry Tools include APIs built for specific tasks:

Need	Service to evaluate
Transcribe speech, synthesize speech, or work with spoken audio	Azure Speech
Analyze images or extract visual information	Azure Vision
Extract text, tables, and structure from documents	Azure Document Intelligence
Classify or analyze text for language tasks	Azure Language
Translate text between languages	Azure Translator
Extract transcript, translation, and other insights from video	Azure AI Video Indexer

These services have their own supported inputs, regions, quotas, pricing, and lifecycle details. Pick the service by the task, then verify its current documentation and test it with representative data. A task API may be simpler and easier to evaluate than asking a general-purpose model to do the same work.

6. Put the pieces together

For a document assistant, the request might pass through an application that authenticates the user, retrieves permitted passages from Azure AI Search, calls a model, and checks the response before presenting it. A voice product may use Speech or a voice-based agent. A document extraction workflow may start with Document Intelligence and send selected fields to a model for a separate reasoning task.

Assign owners for the application and agent instructions, the search index and source permissions, model and tool access, output evaluation, and cost and service limits. In production, trace the whole request path rather than looking only at model token counts.