Exploring the Advanced RAG (Retrieval Augmented Generation) Service

In the ever-evolving landscape of AI, LLM + RAG (Retrieval Augmented Generation) is a typical use scenario. Retrieving accurate related chunked data from complicated docs and then improving LLM response quality becomes challenge. There is no a silver bullet RAG can address all requirements so far. Developers need to verify different advanced RAG techs to find out which is a proper one for their scenarios considering accuracy, response speed, costs, etc. In order to solve this, with Azure Intelligent Document, Azure OpenAI, LlamaIndex, LangChain, Gradio...,  I developed this AdvancedRAG service. This service is encapsulated in a Docker container, offers a streamlined way to experiment with different indexing techniques, evaluate their accuracy, and optimize performance for various RAG use cases. Whether you're building a quick MVP, a proof of concept, or simply exploring different indexing strategies, this service provides a versatile playground.

 

Introduction

 

The Advanced RAG AI Service enables developers to quickly verify different RAG indexing techniques, assessing both accuracy and performance. From index generation to output verification through chat mode and proofreading mode, this service offers comprehensive support. It can run locally in a Docker container or be deployed to Azure Container Apps, providing flexibility for various deployment scenarios.

 

 

 

Docker Deploy Repo: https://github.com/freistli/AdvancedRAG 

 

Key Features

 

• Knowledge Graph  - LlamaIndex
• Recursive Retriever Query   - LlamaIndex
• Tree Mode Summarization  - LlamaIndex
• CSV Query Engine - LlamaIndex
• Semantic Hybrid Search + Sub Query Engine - Azure AI Search
• Microsoft GraphRAG (Local Search + Global Search) 

 

 

 

Quick Start Guide

 

To get started with the Advanced RAG service, follow these steps:

 

Cloning the Repository

 

git clone https://github.com/freistli/AdvancedRAG.git

Setting Up Environment Variables

 

Rename .env_4_SC.sample to .env_4_SC and input the necessary environment variables. Note that an Azure OpenAI resource and an Azure Document Intelligence resource are required. Azure AI Search is optional unless you plan to build or use an Azure AI Search index.

 

Building and Running the Docker Image

 

Build the Docker image:

docker build -t docaidemo .

Run the image locally:

docker run -p 8000:8000 docaidemo

Access the service at http://localhost:8000.

 

Running on Azure Container App

 

1. Publish your Docker image to Azure Container Registry or Docker Hub.
2. Create an Azure Container App, select the published Docker image, and deploy the revision pod without any extra commands.
3. Set environment variables in the Azure Container App if you didn't include .env_4_SC in the image.

 

Building an Index

 

1. Click the "Index Build" tab.
2. Upload a file to the file section. (PDF format is recommended for complex content.)
3. Click "Submit" and wait for the index building to complete. Status updates will appear in real-time on the right pane.

        4. Once you see the completion message, you can download the index for local use.

 

2024-06-13T10:04:54.120027: Index is persisted in /tmp/index_cache/yourfilename
/tmp/index_cache/yourfilename can be used as your Index name.

 

Setup Your Own Index in the Docker Image

 

"rules" Index Name is predefined for Knowledge Graph Index of Japanese proofread demo in this solution. Developers can use their own indexes in other folders for the docker:

To make it work:

1. Move to the folder which contains the AdvancedRAG dockerfile
2. Create a folder to keep the index, for example, index123
3. Extract the index zip file you get from the step 6 in the "Build Index" section, save index files you downloaded into ./index123
4. Build the docker image again.

After this, you can use index123 as index name in the Chat mode.

 

Using the Advanced RAG Service

 

Calling the Service through REST API

 

• Endpoint: https://{BASEURL}/proofreadaddin/run/predict

  •  "rules" Index Name is predefined for Knowledge Graph Index of proofread addin. Please save Default knowledge graph index into ./rules/storage/rules_original

• Method: POST
• Header: Content-Type: application/json
• Sample Data:

{
"data": [ 

"今回は半導体製造装置セクターの最近の動きを分析します。",  

false <--- Streaming

]
}

• Endpoint: https://{BASEURL}/advchatbot/run/chat
• Method: POST
• Header: Content-Type: application/json
• Sample Data:

{
  "data": [
    "When did the Author convince his farther",  <----- Prompt
    "", <--- History Object, don't change it
    "Azure AI Search",  <------ Index Type
    "azuresearch_0",   <------- Index Name or Folder
    "You are a friendly AI Assistant",    <----- System Message
    false <----- Streaming
  ]
}

Consuming the Service REST API in Copilot Studio Power Autoflow Action

 

 

Consuming the Service in Office Add-In

 

For the Proofread Addin use case, refer to: Proofread Addin.

 

 

Chat Mode

 

1. Click "Chat Mode."
2. Choose the index type.
3. Enter the index path or Azure AI Search Index name in the "Index Name" text field.
4. Interact with the document using various system messages if needed.

 

Proofread Mode

 

Proofread mode is tailored for non-English languages and requires a Knowledge Graph Index. Steps to generate this index are the same as for other indices.

 

Viewing the Knowledge Graph Index

 

1. Click the "View Knowledge Graph Index" tab.
2. Enter your Knowledge Graph Index name and click "Submit."
3. After a short wait, click "Download Knowledge Graph View" to see the result.

 

 

More Information

 

Step by Step: Integrate Advanced CSV RAG Service with your own data into Copilot Studio (microsoft.com)

 

Conclusion

 

The Advanced RAG Service is a convenient solution for developers looking to explore and optimize retrieval augmented generation techniques. By providing a flexible, Docker-based environment, it enables rapid experimentation and deployment, making it easier to find the best indexing strategies for specific use cases, and provide REST API Endpoints. Whether you're working locally or deploying to the cloud, this service streamlines the process of developing and testing advanced AI solutions.