In the era of AI, development is advancing. Developers now need to combine various components like data, models, prompts, vector stores, and agents to create compelling AI-powered experiences. Traditional coding methods are no longer sufficient; models are non-deterministic, output quality is subjective, and handling an unknown number of user queries is essential.
Today, we announced Azure AI Foundry, our unified AI platform that includes both our Azure AI Foundry portal (formerly Azure AI Studio), and the Azure AI Foundry SDK, our unified SDK with pre-built app templates enabling developers to:
- Access our most popular models through a single interface
- Easily integrate Azure AI into their apps
- Evaluate, debug and improve application quality and safety across development, testing, and production environments.
The Azure AI Foundry SDK provides a simplified coding experience that enables developers to integrate components together to build AI applications, wherever they build, whether it’s GitHub, Visual Studio, or Copilot Studio. It supports the development journey from idea to code to cloud.
The Azure AI Foundry SDK is available today in Python and C#, with a JavaScript version coming soon. In this initial release, the SDK includes Azure OpenAI, AI model inferencing, Azure AI Search, Azure AI Agent Service, Evaluation, Tracing, and AI app templates. Check out the following video for a quick overview of the SDK:
Explore models with Visual Studio and GitHub
The fastest way to get started with AI is to play with models. GitHub Models enables any developer to start for free. You can also dive right in from Visual Studio by installing the AI Toolkit for VS Code.
The AI Toolkit provides a model playground for GitHub models directly within your editor:
When you get something working in the playground, you can copy code from GitHub models to transition to your AI application.
Start Building with an AI Project
Azure AI projects consolidate everything you need to build AI apps in one place. Start by creating a project in Azure AI Foundry.
Install the projects package:
pip install azure-ai-projects azure-identity
Connect to your project by copying the connection string from the portal, simplifying configuration:
Using the project client, you can access all the capabilities available to build an AI application.
Use the Azure OpenAI Service
The Azure OpenAI Service provides access to OpenAI's models including the GPT-4o, GPT-4o mini, GPT-4, GPT-4 Turbo with Vision, DALLE-3, Whisper, and Embeddings model series with the data residency, scalability, safety, security and enterprise capabilities of Azure.
If you have code that uses the OpenAI SDK, you can easily target your code to use the Azure OpenAI service. First, install the OpenAI SDK:
pip install openai
Create an Azure OpenAI client from the project to leverage the full capabilities of the Azure OpenAI Service:
If you’re already using the Azure OpenAI Service directly, the project provides a convenient way to use Azure OpenAI Service alongside the rest of the AI Foundry capabilities.
Inference Across Popular Models
The model inference package provides a consistent interface for accessing all models supported by our Azure AI model inference service. Combine and experiment with different models to optimize your app, including models from OpenAI, Microsoft, Meta, Mistral AI, Cohere, and AI21 Labs.
To get started with inferencing, install the inference package:
pip install azure-ai-inference
Using project.inference, you can access chat and embedding models deployed to the Azure AI model catalog. Start with the popular GPT-4o model:
… and switch to a Phi3 model by changing the model name:
Prompt templates allow you to easily construct prompts from different inputs:
Agents Unlock Another Level of Intelligence
Many developers manually combine prompts, outputs, and function calls to build intelligent agents that automate business processes. The Azure AI Agent Service simplifies this process with a user-friendly interface and connected tools.
Agents are built into the project, accessible via the project client. Create an agent that implements file search:
Run the agent with user input:
The AI Agent Service meets enterprise needs by allowing Developers to use their own compute and storage. With a rich ecosystem of models from the Azure AI model catalog, knowledge sources such as Microsoft Bing, Microsoft SharePoint, Microsoft Fabric, and Azure AI Search, and more than 1,400 actions connectors with Azure Logic Apps, the service simplifies the process of building agents that increase productivity, enhance customer experience, and improved decision-making. Using the Microsoft 365 Agents SDK, deploy your agent app and API to over 15 Microsoft channels.
"Core42 plans to integrate Azure AI Agent Service APIs into Compass, its managed AI platform, enabling customers to make AI more actionable across their enterprise use cases through plug-and-play integration with various data sources and enterprise applications." – Raghu Chakravarthi, EVP, Engineering & GM US, Core42
Tracing for Observability and Debuggability
Adding tracing to your app provides observability, helping you debug issues and improve performance.
Enable instrumentation of the Azure AI Foundry SDK and other popular packages like OpenAI and LangChain:
Traces can be printed to your local console or uploaded to your AI project:
Evaluate Quality and Safety
Assessing your application’s outputs is critical for production quality and safety at all stages of development: from initial model selection, during development and all the way to post-production. Human feedback can also be incorporated through every step of evaluation to inform application decision-making.
The Azure AI Evaluation SDK provides a comprehensive set of capabilities to accelerate the process of evaluating your application at scale:
- Built-in evaluators allow you to leverage highly tuned evaluation metrics developed by Microsoft (e.g. groundedness, relevance, hate, and unfairness) and the industry (e.g. ROUGE, BLEU, F1 score)
- Custom evaluators enable you to create evaluations specific to your application’s scenarios
- Synthetic data generation gets you started quickly by generating domain-specific evaluation data sets
- User simulators allow you to simulate your target users’ interactions with your app
- Adversarial simulations assess whether your application is vulnerable to security risks, jailbreaks, harmful content generation, or using protected (i.e copyrighted) materials
To get started, install the Azure AI Evaluation package with the remote extra:
pip install azure-ai-evaluation[remote]
To help you evaluate iteratively at every step of your development process, the evaluation package supports:
- Single row of query/response or single/multi-turn conversations
- Batch evaluation (locally and in the cloud) on datasets of query/response pairs or list of conversations, and
- Online evaluation using production traces
You can run an evaluation to score a single query and response pair:
Then you can run a batch evaluation on a dataset or target and log those results to your project:
View, track, and compare your results in the Azure AI Foundry portal:
AI App Templates for Easy Deployment and GenAIOps
Deploying your app requires provisioning cloud infrastructure and automating deployments. The AI Application Templates use infrastructure-as-code to simplify this process.
Use our basic application template to deploy your code to a web app connected to your project resources:
Customize the application code to your liking!
Beyond the basics, more comprehensive templates such as contoso-chat and contoso-creative-writer make it easy to scale AI applications with CI/CD, monitoring and GenAIOps.
The Azure AI Evaluation GitHub action enables you to run evaluations on every code commit as part of your CI/CD process:
After your code passes automated evaluation checks, you will soon be able to automatically create A/B experiments and flight changes to production:
Be sure to sign up for the private preview of our experimentation service!
What Customers have coded with Azure AI Foundry SDK
More than 60,000 customers around the world use Azure AI to develop and scale their AI innovations to reinvent customer experiences and reshape business processes including, Dentsu, C.H. Robinson, and Ontada.
Dentsu reduces time to media insights using Azure AI Foundry
Dentsu used Azure AI Foundry SDK and Azure OpenAI Service to build a predictive analytics copilot that uses conversational chat, that draws on deep expertise in media, forecasting, budgeting, and optimizations, cutting time to insight by 90% and reducing analyst costs. Watch the video.
“With Microsoft, we’re turning our media expertise into a competitive advantage—and harnessing data to build brands and drive business growth.” - Callum Anderson, Dentsu Global Director for DevOps and SRE
“It was great to get a head start with the scripts that were part of the Azure AI Foundry prompt SDK.” - Simon Ranson, Dentsu Lead DevOps Engineer
C.H. Robinson overcomes decades-old barriers to automate the logistics industry using Azure AI Foundry
C.H. Robinson used GitHub, Azure AI Foundry, and Azure OpenAI Service to build a generative AI solution that automates email processing and improves employee productivity and customer satisfaction. Read their story.
“By using the Microsoft platform, we could better ensure data privacy and compliance while interoperating seamlessly with our existing systems.” - Arun Rajan, C.H. Robinson Chief Strategy and Innovation Officer
“Big picture, this tech makes it possible to automate virtually any kind of email transaction and capture efficiencies in global supply chains that just couldn’t be achieved before building this Azure-based generative AI solution. Incorporating Azure AI in our tech has helped put us on pace to achieve another 15% increase in productivity this year, creating happier customers, from one-truck shippers to some of the largest companies in the world.” - Marck Albrecht, C.H. Robinson Vice President of Data Science
Ontada transforms 150 million unstructured oncology documents with Azure OpenAI Service technology
Using Microsoft Azure AI Foundry and Azure OpenAI Service, Ontada implemented large language models to target nearly 100 critical oncology data elements across 39 different cancer types, delivering access to quality data from unstructured oncology documents four times faster. Watch the video.
"We knew the Azure data stack, security, and developer tools could help Ontada uncover and deliver the life-saving biopharma breakthroughs we knew were possible." - Sagran Moodley, Ontada SVP and Chief Information Technology Officer
Try it Out
The Azure AI Foundry SDK provides everything you need to build modern, intelligent applications from prototype to production. We’re just getting started, with much more on the way.
Are you ready to create the future?
- Explore Azure AI Foundry
- Start using the Azure AI Foundry SDK
- Review the Azure AI Foundry documentation
- Take the Azure AI Learn courses
Microsoft
