RAG GetChatCompletionsAsync function response.

Copper Contributor

I have been dabbling with the RAG architecture for sometime. I liked how it makes one to focus on the grounding data and as a beginner this approach helped me a lot to explore using multiple data sources. All was going well until couple of days back when i revisited my project with some new ideas. 

The function call 

chatCompletionsOptions = new ChatCompletionsOptions()
                Messages =
                    new ChatMessage(ChatRole.System, systemPrompt),
                    new ChatMessage(ChatRole.User, userPrompt),
                Temperature = (float)0.7,
                MaxTokens = 3000,
                NucleusSamplingFactor = (float)0.95,
                FrequencyPenalty = 0,
                PresencePenalty = 0,
                AzureExtensionsOptions = new AzureChatExtensionsOptions()
                    Extensions =
                        new AzureCognitiveSearchChatExtensionConfiguration()
                            SearchEndpoint = new Uri(searchEndpoint),
                            IndexName = searchIndexName,
                            SearchKey = new AzureKeyCredential(searchKey),
                            EmbeddingEndpoint = new Uri(apiBase+"openai/deployments/"+ModelName+"/embeddings?api-version="+version+"),
                            EmbeddingKey = new AzureKeyCredential(apiKey),
                            SemanticConfiguration = semanticConfiguration,
                            QueryType = AzureCognitiveSearchQueryType.Vector,
                            ShouldRestrictResultScope = true
            Response<ChatCompletions> response = await client.GetChatCompletionsAsync(deploymentId, chatCompletionOption);
                var message = response.Value.Choices[0].Message;

 used to follow RAG and return a grounded response. But at least since the last couple of days this is responding with a generic message saying no information found on the documents looked up. I used the search explorer in the index to check if index was corrupted some how but that is not the case. 

I was also able to get response when I used AI search like below. 

var searchOptions = new SearchOptions
                VectorSearch = new()
                    Queries = { new VectorizedQuery(queryEmbeddings.ToArray()) { KNearestNeighborsCount = k, Fields = { "contentVector" } } }
                Filter = filter,
                Select = { "title", "content", "category" },

            SearchResults<SearchDocument> response = await searchClient.SearchAsync<SearchDocument>(null, searchOptions);

 I felt like the issue is the search is not happening when 

GetChatCompletionsAsync is called. 
Did anyone else came across an issue like this? If yes how did you rectify this? All help appreciated.
2 Replies




It seems like the issue might be related to the Azure Cognitive Search extension. The “no information found on the documents looked up” message typically indicates that the search query did not return any results from the specified index in Azure Cognitive Search.

Here are a few things you could check:

  • Index: Ensure that the IndexName specified in the AzureCognitiveSearchChatExtensionConfiguration is correct and that the index contains the documents you’re trying to search.
  • Search Key: Verify that the SearchKey is valid and has the necessary permissions to read from the specified index.
  • Search Endpoint: Check that the SearchEndpoint is correct. It should point to your Azure Cognitive Search service.
  • Documents: Make sure the documents in your index are correctly formatted and searchable. The fields you’re trying to search should be marked as searchable in your index schema.
  • Query: If you’re using a vector query (QueryType = AzureCognitiveSearchQueryType.Vector), ensure that the EmbeddingEndpoint and EmbeddingKey are correctly configured, and that the SemanticConfiguration is appropriate for your use case.

If everything seems to be in order with your configuration and documents, you might want to reach out to Azure support for further assistance. 

Thanks Venkat, I created a brand new index and tried, and that fixed the issue.
The code I shared in query is based on old version of Azure.AI.OpenAI. there are many breaking changes since the release 1.0.0-beta.14 (2024-03-04) for "ChatCompletionsOptions".