Build multi-agent orchestration for Azure AI Agent Service with AutoGen / Semantic Kernel.
At Microsoft Ignite 2024, Microsoft released Azure AI Agent Service. In Azure AI Foundry, you can directly build AI Agents based on intelligent business through Azure AI Foundry. Compared with Azure OpenAI Assistants API, Azure AI Agent Service allows you to have more flexible models, such as directly calling open source LLM such like Llama 3, Mistral and Cohere, better enterprise function access (Microsoft Fabric, Microsoft SharePoint, Azure AI Search, Microsoft Bing, etc.), stronger enterprise security mechanisms and data storage methods.
Azure AI Agent Service is in the Public Preview stage. Using Azure AI Foundry SDK, developers can quickly build agents based on Azure AI Agent Service using Python or C#. Enterprises will have different AI Agents based on their business, so how should these AI Agents be combined in the workflow? We need to use AutoGen or Semantic Kernel to orchestrate these three AI Agents.
Tips for creating AI Agents in Azure AI Foundry SDK
When creating a project in Azure AI Foundry, it is recommended to use this template
Note
- Azure AI Agent Service is currently in Public Preview, so we need to use it in specific regions and models.
You can also use the Azure AI Foundry model catalog
-
- Llama 3.1-70B-instruct
- Mistral-large-2407
- Cohere command R+
2. Azure AI Foundry SDK now supports Python / C# versions
Python for Azure AI Foundry SDK
pip install azure-ai-projects
pip install azure-identity
.NET for Azure AI Foundry SDK
dotnet add package Azure.AI.Projects --version 1.0.0-beta.1
3. If you want to learn how to create an AI Agent with Azure AI Foundry SDK, please refer to https://learn.microsoft.com/en-us/azure/ai-services/agents/quickstart
The following introduces a blog writing scenario. This scenario involves three AI agents: one for content collection, another for writing assistance, and a third for content storage and management. These agents can be seamlessly orchestrated using AutoGen or Semantic Kernel.
Azure AI Agent Service defines a single agent
Azure AI Agent Service defines more single agents for specific intelligent tasks. For example, you can define an agent for data mining, an agent that links to Azure Function, or an agent that uses Function Calling to access third-party email services. For the agent mentioned in the above scenario, we can define it like this
🔍Content collection agent
The content collection agent mainly searches for the content of the blog outline through search engines to add more detailed content to the blog. Azure AI Agent Service provides Grounding with Bing service for extension, and you can easily access Bing search.
Samples
- Python https://github.com/kinfey/MultiAIAgent/blob/main/03.AzureAIAgentWithAutoGen01.ipynb
- C# https://github.com/kinfey/MultiAIAgent/blob/main/08.AzureAIAgentWithSK01.ipynb
📖Writer Agent
Agents who write skills can directly complete the written content through LLM.
🛠️Save Agent
The agent that saves content completes the work of saving content through the code-interpreter of Azure AI Agent Service.
Samples:
- Python https://github.com/kinfey/MultiAIAgent/blob/main/01.AzureAIAgentCode.ipynb
- C# https://github.com/kinfey/MultiAIAgent/blob/main/05.AzureAIAgentCodedotNET.ipynb
Use AutoGen / Semantic Kernel to orchestrate multi-agents
In enterprise scenarios, we have different single agents to help us complete different tasks. Based on different tasks, we can combine different agents to complete related tasks. If we need to send a blog, we can combine the above three agents to complete the task. AutoGen / Semantic Kernel both support the orchestration of agents. We can use AutoGen's Function Calling / Semantic Kernel's Plugin to complete the binding of Azure AI Agent Service to complete the orchestration of the above three agents.
The following is the specific architecture
AutoGen's orchestration
Sample - click
Semantic Kernel's orchestration
Sample - click
By orchestrating multiple agents, we can efficiently handle a wide variety of workflows. The collaboration between these agents enables us to quickly complete diverse tasks. As LLM/SLM capabilities continue to advance, integration with enterprise business functions will become even more seamless. The emergence of AI Agents as a Service will enhance the synergy among agents, boosting productivity and paving the way for the realization of the true AI era.
Resources
- Learning about Azure AI Agent Service https://learn.microsoft.com/en-us/azure/ai-services/agents/
- Learning about Microsoft AutoGen https://microsoft.github.io/autogen/dev/
- Learning about Microsoft Semantic Kernel https://github.com/microsoft/semantic-kernel
Updated Jan 07, 2025
Version 3.0kinfey
Microsoft
Joined September 17, 2021
Educator Developer Blog
Follow this blog board to get notified when there's new activity