Blog Post

Azure AI Foundry Blog
15 MIN READ

Building a multimodal, multi-agent system using Azure AI Agent Service and OpenAI Agent SDK

robertrita's avatar
robertrita
Icon for Microsoft rankMicrosoft
Mar 23, 2025

By the end of this blog you will learn how to build a structured, modular conversational AI system using a multi-agent architecture with Azure AI Agent Service and OpenAI Agent SDK. You'll understand how to set up specialized agents—such as Triage, FAQ, Account Management, and Live Agent—to efficiently handle diverse customer interactions by delegating tasks based on user input.

In the rapidly evolving landscape of artificial intelligence (AI), the development of systems that can autonomously interact, learn, and make decisions has become a focal point. A pivotal aspect of t...
Updated Mar 23, 2025
Version 2.0
azure ai search
azure ai studio
azure openai service
suarezlea's avatar
suarezlea
Copper Contributor
Apr 17, 2025


Hi Robert, very good post. Regarding multimodal support, I haven’t been able to get the agent to recognize images as you suggest. Are you sure GPT-4o has multimodal support through the agents API? The Azure Foundry UI doesn’t allow uploading images or videos to a thread. Thank you very much. Best regards.

robertrita's avatar
robertrita
Icon for Microsoft rankMicrosoft
Apr 21, 2025

Yes, multimodal support is implemented on Account Management Agent as shown in the code snippet below where the image path is dynamically added. You can easily tweak this by converting to base64 image if you want it for analysis.

The Azure AI Agent is only used for FAQ.