Enhancing Enterprise AI Deployments with Zero Trust Networking

Problem Statement

• Azure OpenAI is publicly accessible by default Azure OpenAI frequently asked questions | Microsoft Learn
• Any application with API access can call it from anywhere
• Violates:
• Enterprise security policies
• Zero Trust architecture Key principles of the Zero Trust network model
• Regulatory compliance

👉 Enterprises require:

• Private connectivity
• Controlled access via VNet
• DNS-based secure resolution

🏗️ Architecture Overview

✅ Key Components

• Azure OpenAI Service
• Azure Virtual Network (VNet)
• Private Endpoint
• Private DNS Zone
• Application (VM / App Service / AKS)

✅ High-Level Flow

1. Application sends request to OpenAI endpoint
2. DNS resolves endpoint → Private IP
3. Traffic routed inside VNet
4. No internet exposure

👉 Private endpoints assign a private IP inside VNet, ensuring secure communication over Azure backbone.

🔹 Architecture Diagram Description

Diagram 01: Architecture Enhancing Enterprise AI Deployments with Zero Trust Networking

End-to-End Flow

• User authenticates via Entra ID (MFA) Microsoft Entra multifactor authentication
• Traffic passes through WAF (threat filtering)
• Enters private Azure VNet
• API Management enforces policies
• AI services are accessed via private endpoints
• Data is securely fetched from private storage/databases
• Monitoring tools track all activity continuously

Key Value Proposition: This architecture ensures:

• 🚫 No public exposure of AI services or data
• 🔐 Identity-based access instead of network trust
• 🌐 Fully private, isolated network communication
• ⚡ Secure and scalable AI workloads
• 🛡️ Defense-in-depth with monitoring and policy enforcement

Note: This architecture demonstrates how enterprises can securely operationalize AI at scale by combining private networking, identity-driven access, and continuous monitoring—fully aligned with Zero Trust principles.

🔍 Critical Concept: Private Endpoint

A Private Endpoint:

• Creates a network interface in your VNet
• Assigns a private IP address
• Maps to Azure OpenAI service
• Redirects traffic internally

👉 Result:

• No public internet usage
• Fully isolated communication

🔍 Critical Concept: DNS Resolution

Why DNS is critical?

• OpenAI endpoint still uses public FQDN
• Must resolve to private IP instead

👉 Without correct DNS:

• Traffic goes to public endpoint
• Security is broken

How it works

1. Public DNS CNAME → Private Link domain
2. Private DNS overrides resolution
3. FQDN resolves to Private Endpoint IP

👉 DNS ensures that traffic routes correctly to private endpoint

🧱 Required Private DNS Zones

For Azure OpenAI:

• privatelink.openai.azure.com
• privatelink.cognitiveservices.azure.com

👉 These zones map:

• OpenAI endpoint → Private IP

👉 Important: Each Private Endpoint must have proper DNS mapping

⚙️ Step-by-Step Configuration

✅ Step 1: Create Virtual Network

• Create VNet with:
• App subnet
• Private Endpoint subnet

👉 Best practice:

Use dedicated subnet for private endpoints

✅ Step 2: Create Azure OpenAI Resource

• Go to Azure Portal
• Create Azure OpenAI
• Select region & resource group

👉 Note:

OpenAI resource doesn’t need same region as VNet (optional)

✅ Step 3: Disable Public Network Access

• Navigate to:
• Networking → Public Access
• Set:

Public Network Access = Disabled

👉 Ensures service is not accessible via internet

✅ Step 4: Create Private Endpoint

• Go to OpenAI → Networking → Private Endpoint

Configure:

👉 This creates:

• Private IP in subnet
• Network interface mapping

✅ Step 5: Configure Private DNS Zone

Create:

privatelink.openai.azure.com

Then:

• Link DNS zone to VNet
• Add A record automatically (or manually)

👉 DNS maps:

<your-openai-name>.openai.azure.com → Private IP

👉 DNS resolution ensures traffic flows internally

✅ Step 6: Validate Connectivity

From VM inside VNet:

nslookup <openai-name>.openai.azure.com

✅ Expected output:

• Private IP (e.g., 10.x.x.x)

Then test API call → should work

✅ Step 7: Application Integration

Your application (AKS / VM / App Service):

• Calls OpenAI endpoint
• Traffic resolves to private IP
• Routed via VNet

👉 Fully secure AI access

🔐 Security Best Practices

✔ Disable public access completely
✔ Use Private Endpoint for all AI services
✔ Use NSG + Firewall for segmentation
✔ Use Managed Identity instead of API keys
✔ Monitor via Azure Monitor

👉 Private endpoints ensure traffic stays inside Azure backbone network

🏢 Real-World Enterprise Use Case

Example:

• Banking application using OpenAI
• Hosted on AKS
• Uses:
• Private Endpoint
• APIM
• DNS resolution

👉 Result:

• No internet exposure
• Compliance with regulations
• Secure data processing

✅ Key Benefits

• 🔒 Zero internet exposure
• 🌐 Private connectivity
• 🛡️ Zero Trust architecture
• ⚡ Reliable and low latency
• 🧩 Seamless app integration

🧾 Conclusion

Azure OpenAI is powerful, but security architecture is critical for enterprise adoption.

By using:

• Private Endpoints
• Private DNS Zones
• VNet integration

You can build a secure, scalable, and compliant AI solution.