Prototyping Agents with visual tools

Introduction

              Agents are gaining wide adoption in the emerging generative AI applications for organizations, transforming the way we interact with technology. Agent development using visual tools provides a low code / no code approach in prototyping agentic behavior. They help in creating preliminary versions of agentic applications, enabling development, testing, refining the functionalities before full-scale deployment.

Prototyping tools for agents typically have the below features:

• Visual tools that allow for rapid creation, management and interaction with agentic applications
• Enable users to define and modify agents and multi-agent workflows through a point-and-click, drag-and-drop interface
• The interface should make it easier to set parameters for agents within a user-friendly environment and modify flows
• Chat interface to create chat sessions and view results in a conversational and interactive interface. This will enable interactive agent development and testing
• Enable adding memory and tools for agents
• Support for popular OSS agentic frameworks like autogen, langflow, llamaindex, etc
• Access to built-in add-ins and connectors to build sophisticated workflows
• Ability to extend the add-ins and build custom connectors
• Enable tracing for visualization, audit and governance of agents
• Ability to generate deployment code or provide an API and deploy the resulting workflows

By leveraging these tools, developers can quickly prototype and iterate on agent designs, ensuring that the final product is robust, efficient, and capable of delivering a seamless user experience. In this blog, we will look at some OSS options for prototyping and developing agents.

AutoGen Studio

AutoGen Studio is a low-code interface built to help you rapidly prototype AI agents, enhance them with tools, compose them into teams and interact with them to accomplish tasks. While it is still not meant to be a production-ready app, AutoGen Studio can help users rapidly create, manage, and interact with agents that can learn, adapt, and collaborate.

• Declaratively define and modify agents and multi-agent workflows through a point and click, drag and drop interface (e.g., you can select the parameters of two agents that will communicate to solve your task).
• Create chat sessions with the specified agents and view results (e.g., view chat history, generated files, and time taken).
• Explicitly add capabilities to your agents and accomplish more tasks.
• Publish chat sessions to a local gallery.

Agent Development Canvas

• Provides a visual interface for creating agent teams through declarative specification (JSON) or drag-and-drop
• Supports configuration of all core components: teams, agents, tools, models, and termination conditions
• Fully compatible with Autogen AgentChat component definitions

Component map

Edit Components:

Code based editor:

Playground

Provides an interactive environment for testing and running agent teams

• Live message streaming between agents
• Visual representation of message flow through a control transition graph
• Interactive sessions with teams using UserProxyAgent
• Full run control with the ability to pause or stop execution

Tracing and audit

Deployment: AutoGen Studio provides options through Docker and python options for depoying the agents.

  

Semantic Workbench

Semantic Workbench is another tool to prototype agents. The workbench provides a user-friendly UI for creating conversations with one or more agents, configuring settings, and exposing various behaviours.

The Semantic Workbench is composed of three main components:

• Workbench Service (Python): The backend service that handles core functionalities.
• Workbench App (React/Typescript): The frontend web user interface for interacting with workbench and assistants.
• Assistant Services (Python, C#, etc.): any number of assistant services that implement the service protocols/APIs, developed using any framework and programming language of your choice.

Designed to be agnostic of any agent framework, language, or platform, the Semantic Workbench facilitates experimentation, development, testing, and measurement of agent behaviours and workflows. Assistants integrate with the workbench via a RESTful API, allowing for flexibility and broad applicability in various development environments.

Dashboard

Provides a view on existing agents added to the workbench.

Agent Development Canvas

Canvas to add and import new assistants to the workbench.

Agent landing page

Option for viewing past conversations, add new conversations to test the flow and assistant configurations.

Configure Agents

Designing instruction prompts, guardrails, etc.,.

Conversation Testing

Interface to test the assistant flow.

Debugging conversations

Logging the conversation trace and using the trace information for debugging

Ironclad- Rivet

Rivet is a visual programming environment for building AI agents with LLMs. Iterate on your prompt graphs in Rivet, then run them directly in your application. With Rivet, teams can effectively design, debug, and collaborate on complex LLM prompt graphs, and deploy them in their own environment.

Agent Development Canvas

Sample Flow

Flow output

Plugins

Prompt Designer

Testing

Letta ADE:

The Letta ADE is a graphical user interface for creating, deploying, interacting and observing with agents. Letta enables developers to build and deploy stateful AI agents - agents that maintain memory and context across long-running conversations. 

The Agent Development Environment (ADE) provides a visual interface for building and monitoring agents, with real-time visibility into agent memory and behavior. Letta’s context management system intelligently manages memory. Post version 0.5.0, the UI interface is not available in local and we are dependent on a web based interface hosted in letta servers – though the backend can still be local.

Letta enables developers to build and deploy stateful AI agents - agents that maintain memory and context across long-running conversations.

Letta Desktop

Letta agents live inside a Letta Server, which persists them to a database. You can interact with the Letta agents inside your Letta Server with the ADE (a visual interface) and connect your agents to external application via the REST API and Python & TypeScript SDKs. Letta Desktop bundles together the Letta Server and the Agent Development Environment (ADE) into a single application

Adding LLM backends

The Letta server can be connected to various LLM API backends

Flowise:

Flowise is an open source low-code tool for developers to build customized LLM orchestration flows & AI agents. Authoring Canvas offers advanced interface with options to visually add langchain and llamaindex objects for chatflow. Some of the key features include

• Authoring canvas for chat flows and agents

• Chains: Manage the flow of interaction with users, providing a framework to design and implement flows tailored to specific tasks or objectives.
• Language Models: Responsible for language generation and understanding, optimized for various needs.
• Prompts: Keywords or patterns that trigger specific actions or responses based on user inputs.
• Output Parsers: Analyze generated data or responses to extract necessary information.
• Supports integration with frameworks like Langchain, llamaindex, litellm
• Offer enterprise plans for SSO support

Flowise also has an advanced interface to build agent flows.

Tracing

Flowise open source repository has a built-in telemetry that collects anonymous usage information.

Marketplace

Flowise has a large number of templates available that can be useful as a  starter template for complex agents.

Langflow:

Langflow is an OSS framework for building multi-agent and RAG applications. It Python-powered, fully customizable, and LLM and vector store agnostic.

Agent Development Canvas

Langflow provides a canvas that can easily connect different components, such as prompts, language models, and data sources to help build agentic applications.

• Each component in a flow is a node that performs a specific task, like an AI model or a data source.
• Each component has a Configuration menu. Code pane shows a component's underlying Python code.
• Components are connected with edges to form flows.

Components

Langflow 1.1 introduced a new agent component, designed to support complex orchestration with built-in model selection, chat memory, and traceable intermediate steps for reasoning and tool-calling actions.

Playground

Langflow provides a dynamic interface designed for real-time interaction with LLMs, allowing users to chat, access memories, and monitor inputs and outputs. Here, users can directly prototype their models, making adjustments and observing different outcomes

 

API

 Langflow provides an API pane for code templates to call flows from applications.

Starter templates:

Langflow has a library of pre-built templates categorized by use case and methodology.

Langflow Store

Langflow has integration and custom connectors for Flows and components that can be downloaded and imported to the workflows.

Feature Comparison:

 

 

References

How to develop AI Apps and Agents in Azure - A Visual Guide | All things Azure

AutoGen Studio — AutoGen

Semantic Workbench for Agentic AI Development

microsoft/semanticworkbench: A versatile tool designed to help prototype intelligent assistants, agents and multi-agentic systems

Ironclad/rivet: The open-source visual AI programming environment and TypeScript library

Introduction to Rivet | Rivet

https://github.com/Ironclad/rivet

letta-ai/letta: Letta (formerly MemGPT) is a framework for creating LLM services with memory.

Agent Development Environment (ADE) — Letta

https://docs.flowiseai.com/

https://github.com/FlowiseAI/Flowise

https://volcano-ice-cd6.notion.site/Introduction-to-Practical-Building-LLM-Applications-with-Flowise-LangChain-03d6d75bfd20495d96dfdae964bea5a5#eeeab3f52f4047aaa218317f9892aa26

https://github.com/langflow-ai/langflow