Baseline Agentic AI Systems Architecture

Agentic AI Systems are designed to resolved complex problems with limited direct human supervision [1]. These systems are composed of multiple conversable agents that converse with each other and can be orchestrated centrally or self-organize in a decentralized manner [1, 2]. As the usage of multi-agents systems increases in the enterprise to automate complex processes or solve complex tasks, we would like to take a closer look at what the architecture of such systems could look like.

These agents possess capabilities such as planning, allowing them to predict future states and select optimal actions to achieve specific goals. They also incorporate memory, enabling them to recall past interactions, experiences, and knowledge, which is crucial for maintaining continuity in tasks and refining strategies. Additionally, agents can utilize various tools, including APIs and external software, to execute code, query databases, and interact with other systems [1, 3]. This tool usage extends their functionality and enables them to perform a wide range of actions.

Because agents can take actions, write, and execute code, there is a potential risk of running code that could be malicious or harmful to the host system or other users [3]. Therefore, understanding the architecture of these systems is crucial to sandboxing code execution, restricting or denying access to production data and services, and mitigating failures, vulnerabilities, and abuses.

• Azure AI Studio [5] is a managed cloud service used to train, deploy, automate, and manage machine learning models, including large language models (LLM), small language models (SLM), and multi-modal models used by the agents. The platform provides a comprehensive suite of tools and services to facilitate the end-to-end machine learning lifecycle. Key features of the Azure AI Studio include:

  • Prompt Flow [6] is a development tool designed to streamline the entire development lifecycle of Generative AI applications. It supports creating, testing, and deploying prompt flows, which can be used to generate responses or actions based on given prompts. These prompt flows can be deployed to a Machine Learning Workspace or containerized and deployed to Azure Container Apps or Azure Kubernetes Services [7]. AI Studio can also be used to develop and deploy these prompt flows.

  • Managed Online Endpoints are used by agents and backend services to invoke prompt flows for real-time inference. They provide scalable, reliable, and secure endpoints for deploying machine learning models, enabling real-time decision-making and interactions [7].

  • Azure AI Dependencies include essential Azure services and resources that support the functioning of AI Studio and associated projects [8]:

    • Azure Storage Account stores artifacts for projects, such as prompt flows and evaluation data. It is primarily used by the AI Studio to manage data and model assets.
    • Azure AI Search, a comprehensive cloud search service that supports full-text search, semantic search, vector search, and hybrid search. It provides search capabilities for AI projects and agents and is essential for implementing the Retrieval-Augmented Generation (RAG) pattern. This pattern involves extracting relevant queries from a prompt, querying the AI Search service, and using the results to generate a response using an LLM or SLM model.
    • Azure Key Vault used for securely storing and managing secrets, keys, and certificates required by agents, AI projects, and backend services.
    • Azure Container Registry stores and manages container images of agents, backend APIs, orchestrators, and other components. It also stores images created when using a custom runtime for prompt flows.
    • Azure OpenAI service enables natural language processing tasks like text generation, summarization, and conversation.
    • Azure AI Services offers APIs for vision, speech, language, and decision-making, including custom models.
      • Document Intelligence extracts data from documents for intelligent processing.
      • Azure Speech converts speech to text and vice versa, with translation capabilities.

      • Content AI Safety ensures AI-generated content is ethical and safe, preventing the creation or spread of harmful or biased material.

  These components and services provided by the Azure AI Studio enable seamless integration, deployment, and management of sophisticated AI solutions, facilitating the development and operation of Agentic AI Systems.