This blog explores the key Azure AI Infrastructure required for building a scalable, secure, and efficient custom AI solution like copilot, including compute resources, storage solutions, networking infrastructure, security measures, monitoring tools, Azure OpenAI service, deployment models, and also their costs associated in it.
What is Azure AI Foundry?
Azure AI Studio is now renamed to Azure AI Foundry. Azure AI Foundry is a unified AI development platform where teams can efficiently manage AI projects, deploy and test generative AI models, integrate data for prompt engineering, define workflows, and implement content security filters. This powerful tool enhances AI solutions with a wide range of functionalities. It is a one stop shop for all you need for AI development.
Azure AI Hubs are collaborative workspaces for developing and managing AI solutions. To use AI Foundry's features effectively, you need at least one Azure AI Hub. An Azure AI Hub can host multiple projects. Each project includes the tools and resources needed to create a specific AI solution. For example, you can set up a project to help data scientists and developers work together on building a custom Copilot business application. You can use Azure AI Foundry to create an Azure AI Hub, or you can create a hub while creating a new project. This creates an AI Hub resource in your Azure subscription in the resource group you specify, providing a workspace for collaborative AI development. https://ai.azure.com/
Azure Infrastructure
Azure AI Foundry environment utilizes Azure's robust AI infrastructure to facilitate the development, deployment, and management of AI models across various scenarios.
Below is the list of Azure Infrastructure required to deploy the environment. Make sure the below resource providers are enabled for your subscription to deploy these Azure resources.
|
Azure resource |
Resource provider |
Kind |
Purpose |
|
Azure AI Foundry Hub |
Microsoft.MachineLearningServices/ workspace | Hub |
This resource type, associated with the Azure Machine Learning service workspace, serves as a central hub for managing machine learning experiments, models, and data. It provides capabilities for creating, organizing, and collaborating on AI projects. |
|
Azure AI Foundry Project |
Microsoft.MachineLearningServices/ workspace | Project |
Within an Azure AI Studio Hub, you can create projects. These projects allow you to organize your work, collaborate with others, and track experiments related to specific tasks or use cases. Essentially, it provides a structured environment for your AI development. |
|
Azure AI OpenAI Service |
Microsoft.CognitiveServices/account |
AI Services |
An Azure AI Services as the model-as-a-service endpoint provider including GPT-4/4o and ADA Text Embeddings models. |
|
Azure AI Search |
Microsoft.Search/searchServices |
Search Service |
Creates indexes on your data and provides search capabilities for your projects. |
|
Azure Storage Account |
Microsoft.Storage/storageAccounts |
Storage |
It is associated with the Azure AI Foundry workspace. Stores artifacts for your projects (e.g., flows and evaluations). |
|
Azure Key Vault
|
Microsoft.KeyVault/vaults |
Key Vault |
It is associated with the Azure AI Foundry workspace. Stores secrets like connection strings for resource connections. |
|
Azure Container Registry(optional)
|
Microsoft.ContainerRegistry/ |
Container Registry |
Stores Docker images created when using custom runtime for prompt flow. |
|
Azure Application Insights |
Microsoft.Insights/components |
Monitoring |
An Azure Application Insights instance associated with the Azure AI Foundry workspace. Used for application-level logging in deployed prompts. |
|
Log Analytics Workspace
| Microsoft.OperationalInsights/ workspaces | Monitoring |
Used for log storage and analysis. |
|
Event Grid |
Microsoft.Storage/storage accounts/ |
Event Grid System Topic | Event Grid automates workflows by triggering actions in response to events across Azure services, ensuring dynamic and efficient operations in an Azure AI solution |
AI Foundry Environment
Azure Portal View
AI Foundry Portal View
All dependent resources are connected to the hub and can some resources (Azure OpenAI and Azure AI Search) can be shared across projects.
Pricing
Since Azure AI Foundry is assembled from multiple Azure services ,pricing would depend on architectural decisions and usage. When building your own Azure AI solution, it's essential to consider the associated costs that it accrues in Azure AI Foundry. Below are the areas where the costs incur:
1.Compute Hours and Tokens: Unlike fixed monthly costs, Azure AI hubs, Azure OpenAI and projects are billed based on compute hours and tokens used. Be mindful of resource utilization to avoid unexpected charges.
2.Networking Costs: By default, the hub network configuration is public. But if you want to secure the Azure AI Hub there is costs associated with data transfer.
3.Additional Resources: Beyond AI services, consider other Azure resources like Azure Key Vault, Storage, Application Insights, and Event Grids. These services charge based on transactions and data volume.
AI Foundry Cost Pane
Now in Azure Pricing Calculator you can directly find the upfront monthly cost of the resources under Example Scenarios tab in Azure AI Foundry scenario. This cost calculation feature is GA now. You can also use cost management and Azure resource tags to help with a detailed resource-level cost breakdown.
Please note while adding vector search in AI Search requires an Azure OpenAI embedding model. Azure OpenAI embedding model, text-embedding-ada-002 (Version 2), will be deployed if not already. Adding vector embeddings will incur usage to your account. Vector search is available as part of all Azure AI Search tiers in all regions at no extra charge.
If you require to group costs of these different services together, it is recommend creating hubs in one or more dedicated resource groups and subscriptions in your Azure environment. You can navigate to your resource group cost estimation from view cost of resources in Azure AI Foundry.
Azure Pricing Calculator
To learn more about the pricing of Azure AI Foundry pricing click here -Azure AI Foundry - Pricing | Microsoft Azure
Conclusion
Azure AI Foundry enables a path forward for enterprises serious about AI transformation, not just experiments, but scalable, governable, cost predictable, and responsible AI Systems by leveraging the robust infrastructure of Azure Cloud. This integration helps maintain and cater to business goals while simultaneously providing a competitive edge in an AI-driven market.
Resources and getting started with Azure AI
Azure AI Portfolio
Azure AI Infrastructure
Azure OpenAI Service
- Azure OpenAI Service documentation.
- Explore the playground and customization in Azure AI Foundry Portal .
Copilot Studio
- Stay up to date on Copilot -What's new in Copilot Studio
- GPT 4.5 Model Request MS form link. Please note this is now limited to US region only as Azure AI Infrastructure is undergoing significant advancements, continually evolving to meet the demands of modern technology and innovation.
- Copilot & AI Agents
Microsoft