Exploring Azure AI Foundry's Model Router: How It Automatically Optimizes Costs and Performance

A few days ago, I stumbled upon Azure AI Foundry's Model Router (preview) and was fascinated by its promise: a single deployment that automatically selects the most appropriate model for each query. As a developer, this seemed revolutionary no more manually choosing between GPT ( at the moment only work with OpenAI family), or the new o-series reasoning models.

I decided to conduct a comprehensive analysis to truly understand how this intelligent router works and share my findings with the community.

What is Model Router?

Model Router is essentially a "meta-model" that acts like an orchestra conductor. When you send it a query, it evaluates in real-time factors such as:

• Query complexity
• Whether deep reasoning is required
• Necessary context length
• Request parameters

It then routes your request to the most suitable model, optimizing both cost and performance.

Test

I developed a Python script that performs over 50 different tests, grouped into 5 main categories. Here's what I discovered (I´m form Spain, so i tested in Spanish. Sorry for that)

The router proved to be surprisingly intelligent. For simple questions like "What is the capital of France?", it consistently selected more economical . But when I posed complex math or programming problems, it automatically scaled up to GPT-4 or even o-series reasoning models.

Advantages I Found:

Automatic cost optimization: Significant savings by using economical models when possible
No added complexity: A single endpoint for all your needs
Better performance: o-series models activate automatically for complex problems
Transparency: You can always see which model was used in response.model

Billing information

When you use model router today, you're only billed for the use of the underlying models as they're recruited to respond to prompts: the model routing function itself doesn't incur any extra charges. Starting August 1, the model router usage will be charged as well.

You can monitor the costs of your model router deployment in the Azure portal.