Get Started with Azure AI Services | Open AI and Deployment Models

ankitankit · ‎May 15 2024

Overview - Azure AI Services

Azure AI services help developers and organizations rapidly create intelligent, cutting-edge, market-ready, and responsible applications with out-of-the-box and prebuilt and customizable APIs and models.
Azure AI services and Azure Machine Learning both have the end-goal of applying artificial intelligence (AI) to enhance business operations, though how each provides this in the respective offerings is different. Generally, the audiences are different:
- Azure AI services are for developers without machine-learning experience.
- Azure Machine Learning is tailored for data scientists.
Azure AI services are earlier termed as Cognitive Services but now Cognitive Services and Applied AI Services are coined as Azure AI Services.

Example applications include natural language processing for conversations, search, monitoring, translation, speech, vision, and decision-making.

Kinds of Azure AI Services

Many of the Azure AI services have a free tier you can use to try the service. To use the free tier, use F0 as the SKU for your resource.

Service	Description
Anomaly Detector (retired)	Identify potential problems early on.
Azure AI Search	Bring AI-powered cloud search to your mobile & web apps.
Azure OpenAI	Perform a wide variety of natural language tasks.
Bot Service	Create bots and connect them across channels.
Content Moderator (retired)	Detect potentially offensive or unwanted content.
Content Safety	An AI service that detects unwanted contents.
Custom Vision	Customize image recognition for your business.
Document Intelligence	Turn documents into intelligent data-driven solutions.
Face	Detect and identify people and emotions in images.
Immersive Reader	Help users read and comprehend text.
Language	Build apps with industry-leading natural language understanding capabilities.
Language understanding (retired)	Understand natural language in your apps.
Metrics Advisor (retired)	An AI service that detects unwanted contents.
Personalizer (retired)	Create rich, personalized experiences for each user.
QnA maker (retired)	Distill information into easy-to-navigate questions and answers.
Speech	Speech to text, text to speech, translation, and speaker recognition.
Translator	Use AI-powered translation technology to translate more than 100 in-use, at-risk, and endangered languages and dialects.
Video Indexer	Extract actionable insights from your videos.
Vision	Analyze content in images and videos.

With Azure and Azure AI services, you have access to a broad ecosystem, such as:
- Automation and integration tools like Logic Apps and Power Automate.
- Deployment options such as Azure Functions and the App Service.
- Azure AI services Docker containers for secure access.
- Tools like Apache Spark, Azure Databricks, Azure Synapse Analytics, and Azure Kubernetes Service for big data scenarios.

Responsible AI Services

Responsible Artificial Intelligence (Responsible AI) is an approach to developing, assessing, and deploying AI systems in a safe, trustworthy, and ethical way.
Responsible AI can help proactively guide several decisions toward more beneficial and equitable outcomes, keeping people and their goals at the center of system design decisions and respecting enduring values like fairness, reliability, and transparency.

Responsible AI Practices - Empowering responsible AI practices | Microsoft AI

Responsible AI Services within Azure AI Services suite

Vision

Language

Speech

Search

Azure AI Search

Other

Responsible AI terms acceptance

Why accept Responsible AI terms? - AI systems involve technology, users, impacts, and deployment context. Crafting effective systems requires understanding technology, capabilities, and context. Microsoft's Transparency Notes clarify our AI's workings, choices, and holistic perspective. They aid in system development, deployment, and communication.
In addition to the Transparency Note, Microsoft offer guidance and resources for responsibly utilizing Azure OpenAI models, aligning with the Microsoft Responsible AI Standard followed by the engineering teams.

Roles to Accept terms - Azure account must have a Cognitive Services Contributor role assigned in order for you to agree to the responsible AI terms and create a resource.
How to Execute? - If you're planning to use Spatial Analysis in Azure AI Vision or Text Analytics for Health in Azure AI Language, then you must create your first Vision or Language resources from the Azure portal so you can review and acknowledge the terms and conditions. Below are few examples of that.
E.g. Computer Vision

E.g. Azure AI Language

Limited Access Features for Azure AI Services

Why Limited Access? - Microsoft vision is to empower developers and organizations to use AI to transform society in positive ways. We encourage responsible AI practices to protect the rights and safety of individuals. To achieve this, Microsoft has implemented a Limited Access policy grounded in our AI Principles to support responsible deployment of Azure services.

How to Access Limited Features? - Limited Access services require registration, and only customers managed by Microsoft—meaning those who are working directly with Microsoft account teams—are eligible for access.
The use of these services is limited to the use case selected at the time of registration. Customers must acknowledge that they've reviewed and agree to the terms of service. Microsoft may require customers to reverify this information.

The following Azure AI services have Limited Access:
- Custom Neural Voice: Pro features
- Custom text to speech avatar: All features
- Speaker Recognition: All features
- Face API: Identify and Verify features
- Azure AI Vision: Celebrity Recognition feature
- Azure AI Video Indexer: Celebrity Recognition and Face Identify features
- Azure OpenAI: Azure OpenAI Service, modified abuse monitoring, and modified content filters

Submit a registration form for accessing each Limited Access services

Azure Cognitive Account - Kind Open AI

Open AI Services
- Azure OpenAI Service provides REST API access to OpenAI's powerful language models including the GPT-3.5-Turbo model series, GPT-4, GPT-4 Turbo with Vision, GPT-4o & GPT-4 Turbo NEW and Embeddings model series.
- These models can be easily adapted to your specific task including but not limited to content generation, summarization, image understanding, semantic search, and natural language to code translation.
- Users can access the service through REST APIs, Python SDK, or our web-based interface in the Azure OpenAI Studio.
- How to Access OPEN AI - Access is currently limited due to high demand, upcoming product improvements, and Microsoft’s commitment to responsible AI. Azure OpenAI requires registration and is currently only available to approved enterprise customers and partners. You can apply here for access: Apply now
- Once deployed in Azure Portal:

Open AI Deployment Models

Once you create an Azure OpenAI Resource, you must deploy a model before you can start making API calls and generating text.
This action can be done using the Deployment APIs. These APIs allow you to specify the model you wish to use.

Azure OpenAI Service is powered by a diverse set of models with different capabilities and price points. Model availability varies by region

Models	Description
GPT-4o & GPT-4 Turbo NEW	The latest most capable Azure OpenAI models with multimodal versions, which can accept both text and images as input.
GPT-4	A set of models that improve on GPT-3.5 and can understand and generate natural language and code.
GPT-3.5	A set of models that improve on GPT-3 and can understand and generate natural language and code.
Embeddings	A set of models that can convert text into numerical vector form to facilitate text similarity.
DALL-E	A series of models that can generate original images from natural language.
Whisper	A series of models in preview that can transcribe and translate speech to text.
Text to speech (Preview)	A series of models in preview that can synthesize text to speech.

The default quota for models varies by model and region. Default quota limits are subject to change. Select the appropriate model along with the correct version to deploy OPEN AI Models.
Model summary table and region availability - Azure OpenAI Service models - Azure OpenAI | Microsoft Learn
Azure OpenAI now supports automatic updates for select model deployments.
Roles required: Roles and permissions
- Cognitive Services Open AI Contributor role required to edit model and deployments
- Cognitive Services Usages Reader role required for Accessing quota for model deployments.
Sample Screenshot of Open AI Deployment Models

Open AI Studio
- Azure AI Studio is a trusted and inclusive platform that empowers developers of all abilities and preferences to innovate with AI and shape the future. You can play with configured deployment models in Azure Open AI Studio and update scale, capacity settings for different models.
- With Azure AI Studio, you can evaluate large language model (LLM) responses and orchestrate prompt application components with prompt flow for better performance.
- The platform facilitates scalability for transforming proof of concepts into full-fledged production with ease like you can build generative AI applications on an enterprise-grade platform and seamlessly explore, build, test, and deploy using cutting-edge AI tools and ML models, grounded in responsible AI practices.
- Sample Screenshot of Azure Open AI Studio accessible from Azure Open AI product

Azure AI studio provides a comprehensive platform that accelerates the development of production-ready copilots to support enterprise chat, content generation, data analysis, and more. You can evaluate model responses and orchestrate prompt application components with prompt flow for better performance. The platform facilitates scalability for transforming proof of concepts into full-fledged production with ease. The web UI is in preview (May 2024), here is the snapshot of the platform:

IaC Deployment (Terraform)

Cognitive Account Implementation

sku_name varies based on different kinds of Azure AI Services
Possible values of Kind are: Academic, AnomalyDetector, Bing.Autosuggest, Bing.Autosuggest.v7, Bing.CustomSearch, Bing.Search, Bing.Search.v7, Bing.Speech, Bing.SpellCheck, Bing.SpellCheck.v7, CognitiveServices, ComputerVision, ContentModerator, ContentSafety, CustomSpeech, CustomVision.Prediction, CustomVision.Training, Emotion, Face, FormRecognizer, ImmersiveReader, LUIS, LUIS.Authoring, MetricsAdvisor, OpenAI, Personalizer, QnAMaker, Recommendations, SpeakerRecognition, Speech, SpeechServices, SpeechTranslation, TextAnalytics, TextTranslation and WebLM.

Few IaC kinds mapped to Azure AI Services:

S.N.	Azure AI Services	Kind
1	Open AI (Limited Access)	OpenAI
2	Azure AI Content Safety	ContentSafety
3	Azure AI Translation	TextTranslation
4	Azure AI Speech	SpeechServices
5	Azure AI Vision (Responsible AI Service)	ComputerVision
6	Azure AI language (Responsible AI Service)	TextAnalytics
7	Azure AI Document Intelligence	FormRecognizer
8	Azure AI services multi-service account	CognitiveServices

Sample Terraform code shared below with Kind as OpenAI

resource "azurerm_cognitive_account" "example" {
  name                = "example-account"
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
  kind                = "OpenAI"

  sku_name = "S0"

  tags = {
    Acceptance = "Test"
  }
}

Cognitive Model Deployment Implementation
- Model Deployment is only supported for Open AI format
- cognitive_account_idis required parameter where you can pass ID of the above created Cognitive Account
- Pass the values of parameters like model (Required), scale (Required)
- Refer list of models and its version available in a region - Azure OpenAI Service models - Azure OpenAI | Microsoft Learn and select the right configuration
- Sample Terraform code shared below with format as OpenAI
```
resource "azurerm_cognitive_deployment" "example" {
  name                 = "example-cd"
  cognitive_account_id = azurerm_cognitive_account.example.id
  model {
    format  = "OpenAI"
    name    = "text-curie-001"
    version = "1"
  }
  scale {
    type = "Standard"
  }
}
```

Cognitive Account Purge

Once you delete a Cognitive Account, you won't be able to create another one with the same name for 48 hours. To create a resource with the same name, you need to purge the deleted resource.
You must have necessary permissions to purge a resource. The least permission a subscription must have - Microsoft.CognitiveServices/locations/resourceGroups/deletedAccounts/delete to purge resources, such as Cognitive Services Contributor or Contributor.
When using Contributor to purge a resource the role must be assigned at the subscription level. If the role assignment is only present at the resource or resource group level, you can't access the purge functionality.
Recover or purge deleted Azure AI services resources - Azure AI services | Microsoft Learn

Important Links

Overview - Azure AI Services

Responsible AI

Limited Access Features

Azure Open AI Service and Deployment Model

Cognitive Account - IaC

Products (49)

Special Topics (26)

Video Hub (462)

Most Active Hubs