Use Azure OpenAI and APIM with the OpenAI Agents SDK

The OpenAI Agents SDK provides a powerful framework for building intelligent AI assistants with specialised capabilities. In this blog post, I'll demonstrate how to integrate Azure OpenAI Service and Azure API Management (APIM) with the OpenAI Agents SDK to create a banking assistant system with specialised agents.

The OpenAI Agents SDK

The OpenAI Agents SDK is a powerful toolkit that enables developers to create AI agents with specialised capabilities, tools, and the ability to work together through handoffs. It's designed to work seamlessly with OpenAI's models, but can be integrated with Azure services for enterprise-grade deployments.

Setting Up Your Environment

To get started with the OpenAI Agents SDK and Azure, you'll need to install the necessary packages:

pip install openai openai-agents python-dotenv

You'll also need to set up your environment variables. Create a `.env` file with your Azure OpenAI or APIM credentials:

For Direct Azure OpenAI Connection:

# .env file for Azure OpenAI
AZURE_OPENAI_API_KEY=your_api_key
AZURE_OPENAI_API_VERSION=2024-08-01-preview
AZURE_OPENAI_ENDPOINT=https://your-resource-name.openai.azure.com/
AZURE_OPENAI_DEPLOYMENT=your-deployment-name

For Azure API Management (APIM) Connection:

# .env file for Azure APIM
AZURE_APIM_OPENAI_SUBSCRIPTION_KEY=your_subscription_key
AZURE_APIM_OPENAI_API_VERSION=2024-08-01-preview
AZURE_APIM_OPENAI_ENDPOINT=https://your-apim-name.azure-api.net/
AZURE_APIM_OPENAI_DEPLOYMENT=your-deployment-name

Connecting to Azure OpenAI Service

The OpenAI Agents SDK can be integrated with Azure OpenAI Service in two ways: direct connection or through Azure API Management (APIM).

Option 1: Direct Azure OpenAI Connection

from openai import AsyncAzureOpenAI
from agents import set_default_openai_client
from dotenv import load_dotenv
import os

# Load environment variables
load_dotenv()

# Create OpenAI client using Azure OpenAI
openai_client = AsyncAzureOpenAI(
    api_key=os.getenv("AZURE_OPENAI_API_KEY"),
    api_version=os.getenv("AZURE_OPENAI_API_VERSION"),
    azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
    azure_deployment=os.getenv("AZURE_OPENAI_DEPLOYMENT")
)

# Set the default OpenAI client for the Agents SDK
set_default_openai_client(openai_client)

Option 2: Azure API Management (APIM) Connection

from openai import AsyncAzureOpenAI
from agents import set_default_openai_client
from dotenv import load_dotenv
import os

# Load environment variables
load_dotenv()

# Create OpenAI client using Azure APIM
openai_client = AsyncAzureOpenAI(
    api_key=os.getenv("AZURE_APIM_OPENAI_SUBSCRIPTION_KEY"),  # Note: Using subscription key
    api_version=os.getenv("AZURE_APIM_OPENAI_API_VERSION"),
    azure_endpoint=os.getenv("AZURE_APIM_OPENAI_ENDPOINT"),
    azure_deployment=os.getenv("AZURE_APIM_OPENAI_DEPLOYMENT")
)

# Set the default OpenAI client for the Agents SDK
set_default_openai_client(openai_client)

 

Creating Agents with the OpenAI Agents SDK

Once you've set up your Azure OpenAI or APIM connection, you can create agents using the OpenAI Agents SDK:

from agents import Agent, OpenAIChatCompletionsModel
from openai.types.chat import ChatCompletionMessageParam

# Create a banking assistant agent
banking_assistant = Agent(
    name="Banking Assistant",
    instructions="You are a helpful banking assistant. Be concise and professional.",
    model=OpenAIChatCompletionsModel(
            model="gpt-4.1", # This will use the deployment specified in your Azure OpenAI/APIM client
            openai_client=openai_client
        )
    tools=[check_account_balance]  # A function tool defined elsewhere
)

The OpenAI Agents SDK automatically uses the Azure OpenAI or APIM client you've configured, making it seamless to switch between different Azure environments or configurations.

Implementing Tracing with Azure OpenAI

The OpenAI Agents SDK includes powerful tracing capabilities that can help you monitor and debug your agents. When using Azure OpenAI or APIM, you can implement two types of tracing:

 

1. Console Tracing for Development

Console logging is rather verbose, if you would like to explore the Spans then enable do it like below:

from agents import Agent, HandoffInputData, Runner, function_tool, handoff, trace, set_default_openai_client, set_tracing_disabled, OpenAIChatCompletionsModel, set_tracing_export_api_key, add_trace_processor
from agents.tracing.processors import ConsoleSpanExporter, BatchTraceProcessor

# Set up console tracing
console_exporter = ConsoleSpanExporter()
console_processor = BatchTraceProcessor(exporter=console_exporter)
add_trace_processor(console_processor)

2. OpenAI Dashboard Tracing

Currently the spans are being sent to https://api.openai.com/v1/traces/ingest 

from agents import Agent, HandoffInputData, Runner, function_tool, handoff, trace, set_default_openai_client, set_tracing_disabled, OpenAIChatCompletionsModel, set_tracing_export_api_key, add_trace_processor

set_tracing_export_api_key(os.getenv("OPENAI_API_KEY"))

 

Tracing is particularly valuable when working with Azure deployments, as it helps you monitor usage, performance, and behavior across different environments.

Note that at the time of writing this article, there is a ongoing bug where OpenAI Agent SDK is fetching the old input_tokens, output_tokens instead of the new prompt_tokens & completion_tokens returned by newer ChatCompletion APIs. 

Thus you would need to manually update in agents/run.py file to make this work per https://github.com/openai/openai-agents-python/pull/65/files

Running Agents with Azure OpenAI

To run your agents with Azure OpenAI or APIM, use the Runner class from the OpenAI Agents SDK:

from agents import Runner
import asyncio

async def main():
    # Run the banking assistant
    result = await Runner.run(
        banking_assistant, 
        input="Hi, I'd like to check my account balance."
    )
    
    print(f"Response: {result.response.content}")

if __name__ == "__main__":
    asyncio.run(main())

Practical Example: Banking Agents System

Let's look at how we can use Azure OpenAI or APIM with the OpenAI Agents SDK to create a banking system with specialized agents and handoff capabilities.

1. Define Specialized Banking Agents

We'll create several specialized agents:

• General Banking Assistant: Handles basic inquiries and account information
• Loan Specialist: Focuses on loan options and payment calculations
• Investment Specialist: Provides guidance on investment options
• Customer Service Agent: Routes inquiries to specialists

2. Implement Handoff Between Agents

from agents import handoff, HandoffInputData
from agents.extensions import handoff_filters

# Define a filter for handoff messages
def banking_handoff_message_filter(handoff_message_data: HandoffInputData) -> HandoffInputData:
    # Remove any tool-related messages from the message history
    handoff_message_data = handoff_filters.remove_all_tools(handoff_message_data)
    return handoff_message_data

# Create customer service agent with handoffs
customer_service_agent = Agent(
    name="Customer Service Agent",
    instructions="""You are a customer service agent at a bank.
    Help customers with general inquiries and direct them to specialists when needed.
    If the customer asks about loans or mortgages, handoff to the Loan Specialist.
    If the customer asks about investments or portfolio management, handoff to the Investment Specialist.""",
    handoffs=[
        handoff(loan_specialist_agent, input_filter=banking_handoff_message_filter),
        handoff(investment_specialist_agent, input_filter=banking_handoff_message_filter),
    ],
    tools=[check_account_balance],
)

3. Trace the Conversation Flow

from agents import trace

async def main():
    # Trace the entire run as a single workflow
    with trace(workflow_name="Banking Assistant Demo"):
        # Run the customer service agent
        result = await Runner.run(
            customer_service_agent,
            input="I'm interested in taking out a mortgage loan. Can you help me understand my options?"
        )
        
        print(f"Response: {result.response.content}")

if __name__ == "__main__":
    asyncio.run(main())

Benefits of Using Azure OpenAI/APIM with the OpenAI Agents SDK

Integrating Azure OpenAI or APIM with the OpenAI Agents SDK offers several advantages:

• Enterprise-Grade Security: Azure provides robust security features, compliance certifications, and private networking options
• Scalability: Azure's infrastructure can handle high-volume production workloads
• Monitoring and Management: APIM provides additional monitoring, throttling, and API management capabilities
• Regional Deployment: Azure allows you to deploy models in specific regions to meet data residency requirements
• Cost Management: Azure provides detailed usage tracking and cost management tools

Conclusion

The OpenAI Agents SDK combined with Azure OpenAI Service or Azure API Management provides a powerful foundation for building intelligent, specialized AI assistants. By leveraging Azure's enterprise features and the OpenAI Agents SDK's capabilities, you can create robust, scalable, and secure AI applications for production environments.

Whether you choose direct Azure OpenAI integration or Azure API Management depends on your specific needs for API management, security, and monitoring. Both approaches work seamlessly with the OpenAI Agents SDK, making it easy to build sophisticated agent-based applications.