In November 2021, Microsoft announced the release of Azure Cognitive Service for Language. This introduced a new unified service for all natural language processing capabilities in Azure's Cognitive Services. The new service included a set of brand-new set features in public preview that used the latest state-of-the-art transformer Z-Code models to empower users to customize language models with their own domain-specific data. These models are born multi-lingual, which allow you to train custom models in one language, and immediately predict them in other languages. In March, support for up to languages was announced for these custom features.
Today, we're excited to announce their general availability!
Conversational language understanding - A custom NLU service to create your own intents and entities used to power chatbots, devices, or virtual assistants. Conversational language understanding is the next generation of LUIS with significant accuracy improvements, built in evaluation, and service side orchestration for complex architectures of multiple projects managed under one endpoint. As part of the announcement, the service now offers two training modes: standard and advanced. The default standard training mode has a shorter training time allowing for quicker iterations of development in English. The advanced training mode provides access to the full range of 96 languages.
Custom text classification - A custom classification service for your longer, more formal documents such as emails, contracts, and reports. This service offers single and multi-label classification to approach different scenarios. Automate processes such as ticket routing by building your own custom classifiers.
Custom Named Entity Recognition (NER) - A custom extraction service that allows you to identify relevant information within documents. Use this service to extract domain-specific terms to enhance search capabilities or enable automatic tagging.
Each of these services is available in the Language Studio where you can upload and add data, define, label, train, deploy and test your custom projects with ease. You can also try the wide variety of prebuilt NLP capabilities available to use out of the box.
You can watch the AI show from November to get an introduction of the Language Studio and service.
Regional availability
The custom capabilities are now available in the following Azure regions:
- Australia East
- East US
- East US 2
- North Europe
- South Central US
- UK South
- West Europe
- West US 2
- West US 3
Pricing
We’re also excited to announce the free tier options for these custom capabilities.
- Training in custom text classification and custom named entity recognition will still be charged at an hourly rate for the paid tier (S) Language resources. The free tier will offer you 1 free hour of training and 5,000 free text records for predictions, shared with the other Language services.
- Conversational language understanding will begin charging for use with the paid tier (S) Language resources starting July 1st, 2022. Its free tier will include 1 free hour of advanced training and 5,000 free text records for predictions, shared with the other Language services. Standard training will be offered free of charge with the free and paid tier resources. Learn more about the new pricing here.
APIs
Language APIs are the entry point to programmatically call all the Language features. Conversational features such as CLU are available under /language/analyze-conversation. Previously, Text Analytics APIs were used to accessing custom text classification and custom NER predictions through the /text/analytics/v3.2-preview.2 API. All text analytics 3.2-preview.2 API users can begin migrating their workloads to /language/analyze-text?api-version=2022-05-01.
|
|
Text Analytics v3.2-preview.2 |
Language (API Version=2022-05-01) |
Language (API Version=2022-05-15-preview) |
|
URL |
https://{endpoint}/text/analytics/v3.2-preview.2/analyze |
https://{endpoint}/language/:analyze-text?api-version=2022-05-01 |
https://{endpoint}/language/:analyze-text?api-version=2022-05-15-preview |
|
Synchronous API calls
(Sentiment, NER, Key Phrase Extraction, PII, etc.) |
Separate API paths / URLs for each feature |
One API where the payload specifies the type of task. |
One API where the payload specifies the type of task. |
|
Asynchronous API calls
(Custom text classification, Custom NER, Healthcare, Sentiment, Key Phrase, etc.) |
One API to submit jobs for TA4H FHIR, another for all other features. |
One API that can include configurations to tasks for all supported features except TA4H FHIR and extraction summarization. |
One API that can include configurations to tasks all prebuilt tasks including TA4H FHIR and extraction summarization. |
A sample request for the new synchronous APIs for Language would be identical to the following sample below. You’ll notice the kind parameter indicating the type of request and response. You can learn more about the APIs here.
We’re eager to see your businesses benefit globally from these features and looking forward to your feedback to further drive improvements.
Get started with the Language services today.