Blog Post

Azure Architecture Blog
17 MIN READ

Demystifying Azure OpenAI Networking for Secure Chatbot Deployment

FreddyAyala's avatar
FreddyAyala
Icon for Microsoft rankMicrosoft
Sep 20, 2023
Introduction Azure AI Landing Zones provide a solid foundation for deploying advanced AI technologies like OpenAI's GPT-4 models. These environments are designed to support AI enthusiasts, b...
Updated Nov 09, 2023
Version 2.0
artificial intelligence
infrastructure
well architected
Terran0925's avatar
Terran0925
Copper Contributor
Oct 13, 2023

Hi Freddie,

 

I've been setting up Azure OpenAI Services with Cognitive Search last week and cannot find documentation on how to have OpenAI Services privately access Cognitive Search via the private endpoint. I am able to successfully configure Cognitive Search to use the "Shared private access" configurating under the Networking blade to connect to multiple blob storage accounts but this feature does not appear to be available for OpenAI Services as it only has the "Private endpoint connections" option for inbound access. Is there a way to have OpenAI reach Cognitive Search service via a private network or does it have to go over the internet and connect to Cognitive Search via its public IP? The only way we could get the setup to work is:

 

Virtual Machine in a subnet with postman calls API ---> Private Endpoint of OpenAI ---> OpenAI calls Cognitive Search via public IP (Cognitive Search has "All Networks" configured for "Public access").

 

I've done a bit of reading and some of the options I can think of are:

 

1. Try to lock down Cognitive Search with IPs that OpenAI sends traffic out from the internet with 

2. Front Cognitive Search with an APIM so it can be locked down to specific public IP of the APIM

 

Thanks.

 

**************Update***************

I think I might have found my answer: 

https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/use-your-data#azure-cognitive-search-resources

If you have an Azure Cognitive Search resource protected by a private network, and want to allow Azure OpenAI on your data to access your search service, complete https://aka.ms/applyacsvpnaoaioyd. The application will be reviewed in ten business days and you will be contacted via email about the results. If you are eligible, we will send a private endpoint request to your search service, and you will need to approve the request.