Orchestrate multi-LLM workflows with Azure Managed Redis
Authors: Roberto Perez, George von Bülow & Roy de Milde Key challenge for building effective LLMs In the age of generative AI, large language models (LLMs) are reshaping how we build applications — from chatbots to intelligent agents and beyond. But as these systems become more dynamic and multi-modal, one key challenge stands out: how do we route requests efficiently to the right model, prompt, or action at the right time? Traditional architectures struggle with the speed and precision required to orchestrate LLM calls in real-time, especially at scale. This is where Azure Managed Redis steps in — acting as a fast, in-memory data layer to power smart, context-aware routing for LLMs. In this blog, we explore how Redis and Azure are enabling developers to build AI systems that respond faster, think smarter, and scale effortlessly. Across industries, customers are hitting real limitations. AI workloads often need to track context across multiple interactions, store intermediate decisions, and switch between different prompts or models based on user intent — all while staying responsive. But stitching this logic together using traditional databases or microservice queues introduces latency, complexity, and cost. Teams face challenges like keeping routing logic fast and adaptive, storing transient LLM state without bloating backend services, and coordinating agent-like behaviors across multiple components. These are exactly the pain points AMR was built to address — giving developers a low-latency, highly available foundation for real-time AI orchestration and more. How to use Azure Managed Redis as a Semantic Router Semantic routing uses AI to route user queries to the right service, model or endpoint, based on their intent and context. Unlike rule-based systems, it leverages Generative AI to understand the meaning behind requests, enabling more accurate and efficient decisions. Importantly, the semantic router itself does not forward the query—it only selects the appropriate route. Your application is responsible for taking that routing decision and sending the query to the correct agent, model, or human. The users sends a query, which is passed to the system for processing The query is analyzed by an embedding model to understand its semantic intent and context The semantic router evaluates the user’s intent and context to choose the optimal route: A specific model for further processing An agent to handle the query A default response if applicable Escalation to a human for manual handling, if needed Valid queries go through the RAG pipeline to generate a response The final response is sent back to the user Code examples + Architecture Example: Jupyter Notebook with Semantic Router Let’s look at a Jupyter Notebook example that implements a simple Semantic Router with Azure Managed Redis and the Redis Vector Library. First, we install the required Python packages and define a connection to an AMR instance: pip install -q "redisvl>=0.6.0" sentence-transformers dotenv Define the Azure Managed Redis Connection. import os import warnings warnings.filterwarnings("ignore") from dotenv import load_dotenv load_dotenv() REDIS_HOST = os.getenv("REDIS_HOST") # ex: "gvb-sm.uksouth.redis.azure.net" REDIS_PORT = os.getenv("REDIS_PORT") # for AMR this is always 10000 REDIS_PASSWORD = os.getenv("REDIS_PASSWORD") # ex: "giMzOzIP4YmjNBGCfmqpgA7e749d6GyIHAzCaF5XXXXX" # If SSL is enabled on the endpoint, use rediss:// as the URL prefix REDIS_URL = f"redis://:{REDIS_PASSWORD}@{REDIS_HOST}:{REDIS_PORT}" Next, we create our first Semantic Router with an allow/block list: from redisvl.extensions.router import Route, SemanticRouter from redisvl.utils.vectorize import HFTextVectorizer vectorizer = HFTextVectorizer() # Semantic router blocked_references = [ "things about aliens", "corporate questions about agile", "anything about the S&P 500", ] blocked_route = Route(name="block_list", references=blocked_references) block_router = SemanticRouter( name="bouncer", vectorizer=vectorizer, routes=[blocked_route], redis_url=REDIS_URL, overwrite=False, ) To prevent users from asking certain categories of questions, we can define example references in a list of blocked routes using the Redis Vector Library function SemanticRouter(). While it is also possible to implement blocking at the LLM level through prompt engineering (e.g., instructing the model to refuse answering certain queries), this approach still requires an LLM call, adding unnecessary cost and latency. By handling blocking earlier with semantic routing in Azure Managed Redis, unwanted queries can be intercepted before ever reaching the model, saving LLM tokens, reducing expenses, and improving overall efficiency. Let’s try it out: user_query = "Why is agile so important?" route_match = block_router(user_query) route_match The router first vectorizes the user query using the specified Hugging Face text vectorizer. It finds a semantic similarity with route reference “corporate question sabout agile” and returns the matching route ‘block_list`. Note the returned distance value – this indicates the degree of semantic similarity between the user query and the blocked reference. You can fine-tune the Semantic Router by specifying a minimum threshold value that must be reached to count as a match. For full details and more complex examples, you can explore the Jupyter Notebooks in this GitHub repository. How do customers benefit? For customers, this technology delivers clear and immediate value. By using Azure Managed Redis as the high-performance backbone for semantic routing and agent coordination, organizations can significantly reduce latency, simplify infrastructure, and accelerate time-to-value for AI-driven experiences. Instead of building custom logic spread across multiple services, teams get a centralized, scalable, and fully managed in-memory layer that handles vector search, routing logic, and real-time state management — all with enterprise-grade SLAs, security, and Azure-native integration. The result? Smarter and faster LLM interactions, reduced operational complexity, and the flexibility to scale AI use cases from prototypes to production without re-architecting. Whether you're building an intelligent chatbot, orchestrating multi-agent workflows, or powering internal copilots, this Redis-backed technology gives you the agility to adapt in real time. You can dynamically route based on user intent, past interactions, or even business rules — all while maintaining low-latency responses that users expect from modern AI applications. And because it’s fully managed on Azure, teams can focus on innovation rather than infrastructure, with built-in support for high availability, monitoring, and enterprise governance. It’s a future-proof foundation for AI systems that need to be not just powerful, but precise. Try Azure Managed Redis today If you want to explore how to route large language models efficiently, Azure Managed Redis provides a reliable and low-latency solution. You can learn more about the service on the Azure Managed Redis page and find detailed documentation in the Azure Redis overview. For hands-on experience, check out the routing optimization notebook and other examples in the Redis AI resources repository and GitHub - loriotpiroloriol/amr-semantic-router. Give it a try to see how it fits your LLM routing needs.Building faster AI agents with Azure Managed Redis and .NET Aspire
AI is evolving fast—and so are the tools to build intelligent, responsive applications. In our recent Microsoft Reactor session, Catherine Wang (Principal Product Manager at Microsoft) and Roberto Perez (Microsoft MVP and Senior Global Solutions Architect at Redis) shared how Azure Managed Redis helps you create Retrieval-Augmented Generation (RAG) AI agents with exceptional speed and consistency. Why RAG agents? RAG applications combine the power of large language models (LLMs) with your own data to answer questions accurately. For example, a customer support chatbot can deliver precise, pre-approved answers instead of inventing them on the fly. This ensures consistency, reduces risk, and improves customer experience. Where Azure Managed Redis fits with agentic scenarios In this project, Azure Managed Redis is used as a high-performance, in-memory vector database to support Agentic Retrieval-Augmented Generation (RAG), enabling fast similarity searches over embeddings to retrieve and ground the LLM with the most relevant known answers. Beyond this, Azure Managed Redis is a versatile platform that supports a range of AI-native use cases, including: Semantic Cache – Cache and reuse previous LLM responses based on semantic similarity to reduce latency and improve reliability. LLM Memory – Persist recent interactions and context to maintain coherent, multi-turn conversations. Agentic Memory – Store long-term agent knowledge, actions, and plans to enable more intelligent and autonomous behavior over time. Feature Store – Serve real-time features to machine learning models during inference for personalization and decision-making. These capabilities make Azure Managed Redis a foundational building block for building fast, stateful, and intelligent AI applications. Demo highlights In the session, the team demonstrates how to: Deploy a RAG AI agent using .NET Aspire and Azure Container Apps. Secure your Redis instance with Azure Entra ID, removing the need for connection strings. Use Semantic Kernel to orchestrate agents and retrieve knowledge base content via vector search. Monitor and debug microservices with built-in observability tools. Finally, we walk through code examples in C# and Python, demonstrating how you can integrate Redis search, vector similarity, and prompt orchestration into your own apps. Get Started Ready to explore? ✅ Watch the full session replay: Building a RAG AI Agent Using Azure Redis ✅ Try the sample code: Azure Managed Redis RAG AI SampleGet started with Azure Managed Redis today: a step-by-step guide to deployment
At Microsoft Build 2025, we announced the general availability of Azure Managed Redis, a fully-managed, first-party service built in partnership with Redis. Ready for production workloads globally, Azure Managed Redis marks a major milestone for developers looking to build high-performance, real-time applications with the speed and reliability of Redis, fully managed on Azure. Call to action: get started with Azure Managed Redis in the Azure Portal. Key updates: Up to 15x performance improvements over Azure Cache for Redis 99.999% availability with multi-region Active‑Active replication Support for Redis 7.4 (with Redis 8 coming soon) New modules including RedisJSON, vector search, bloom filters, and time-series Flexible SKUs that let you scale memory and compute independently Navigate the new Azure Managed Redis in the Azure Portal Azure Managed Redis also comes with an updated Azure Portal experience which simplifies how you create, configure, and manage your Redis instances. Whether experimenting or deploying to production, the portal gives you full control with a few clicks. Step-by-step guide to deploying in the Azure Portal Want to see Azure Managed Redis in action? This quick walkthrough video shows how to set up Azure Managed Redis inside the Azure Portal: 👉 Watch on YouTube In this tutorial, you’ll learn how to: How to configure your Active-Active instance for high availability and low latency Setting up geo-replication across regions for 99.999% availability SLA Key tips and best practices to get started quickly No code required — just the Azure Portal and a few minutes of your time! Azure Managed Redis is perfect for cloud architects, developers, and IT pros looking to build resilient, globally available Redis-backed applications on Azure. Whether you're building AI-powered applications, speeding up your web services, or just getting started with Redis, now’s the time to explore what Azure Managed Redis can do. To learn more, head to our product page for more information or contact your Microsoft sales representative. To get started, provision Azure Managed Redis in the Azure Portal today. Resources Azure Managed Redis product page Azure Managed Redis pricing page Create an Azure Managed Redis instance Watch the Microsoft Build 2025 session on AMR Explore the documentationOn-Demand Backups in Azure Database for PostgreSQL – Flexible Server Now Generally Available
We’re excited to announce the General Availability of On-Demand Backups in Azure Database for PostgreSQL – Flexible Server! In today’s dynamic data management landscape, ensuring the protection and recoverability of your databases is essential. Azure Database for PostgreSQL – Flexible Server streamlines this responsibility through comprehensive backup management, including automated, scheduled storage volume snapshots encompassing the entire database instance and all associated transaction logs. With the introduction of On-demand backups you now have the flexibility to initiate backups at any time, supplementing the existing scheduled backups. This capability is particularly valuable in scenarios involving high-risk operations, such as system upgrades or schema modifications, or when performing periodic data refreshes that do not align with the standard backup schedule. Benefits Instant Backup Creation: Trigger a full backup of your server on demand—no more waiting for the automated backup schedule. Cost Optimization: While Azure manages automated backups that cannot be deleted until the retention window is met, on-demand backups provide greater control over storage costs. Delete these backups once their purpose is served to avoid unnecessary storage expense. Enhanced Control & Safety: Take backups before schema changes, major deployments, or periodic refresh activities to meet your business requirements. Seamless Integration: Accessible via Azure Portal, Azure CLI, ARM templates, and REST APIs. Azure Database for PostgreSQL Flexible Server provides a comprehensive, user-friendly backup solution, giving you the confidence to manage your data effectively and securely. Let us explore how on-demand backups can elevate your database management strategy and provide peace of mind during high-stakes operations. Automated Backups vs On-Demand Backups Feature Automated Backups On-Demand Backups Creation Scheduled by Azure Manually initiated by the user Retention Based on the backup policy Based on the backup policy Deletion Managed by Azure User-controlled Use Cases Regular data protection High-risk operations, ad-hoc needs How to take On-Demand Backups using the portal. In the Azure portal, choose your Azure Database for PostgreSQL flexible server. Click Settings from the left panel and choose Backup and Restore. Click Backup and provide your backup name. Click Backup. A notification is shown that an On-demand backup trigger has been initiated. For more information: How to perform On-demand backups using Portal How to take On-Demand Backups using CLI. You can run the following command to perform an on-demand backup of a server. az postgres flexible-server backup create --resource-group <resource_group> --name <server> --backup-name <backup> Example: For more information: How to perform On-demand backups using CLI How to list all on-demand backups using CLI You can list currently available on-demand backups of a server via the az postgres flexible-server backup list command. az postgres flexible-server backup list --resource-group <resource_group> --name <server> --query "[?backupType=='Customer On-Demand']" --output table For more information: How to list all backups using Portal What's Next Once you have taken an on-demand backup based on your business needs, you can retain it until your high-risk operation is complete or use it to refresh your reporting or non-production environments. You can delete the backups to optimize storage costs when the backup is no longer needed. To restore or delete on-demand backups, you can use the Azure portal, CLI, or API for seamless management. Limitations & Considerations: SKU Support: On-demand backups are available for General Purpose and Memory-Optimized SKUs. Burstable SKUs are not supported. Storage Tier Compatibility: Currently, only the SSDv1 storage tier is supported. Support for SSDv2 is on our roadmap and will be introduced in a future update. You can take up to 7 on-demand backups per flexible server. This limit is intentional to help manage backup costs, as on-demand backups are meant for occasional use. The managed service already provides support for up to 35 backups in total, excluding on-demand backups. Take Control of Your Database Protection Today! The ability to create on-demand backups is critical for managing and safeguarding your data. Whether you're preparing for high-risk operations or refreshing non-production environments, this feature puts flexibility and control in your hands. Get started now: Create your first on-demand backup using the Azure Portal or CLI. Optimize your storage costs by deleting backups when no longer needed. Restore with ease to keep your database resilient and ready for any challenge. Protect your data effectively and ensure your database is always prepared for the unexpected. Learn more about Azure Database for PostgreSQL Flexible Server and explore the possibilities with on-demand backups today! You can always find the latest features added to Flexible server in this release notes page. We are eager to hear all the great scenarios this new feature helps you optimize, and look forward to receiving your feedback at https://aka.ms/PGfeedback.
