Form Recognizer previews document classification, Azure OpenAI integration and lots more

[Vinod_Kurpad]

Form Recognizer is an Applied AI service for all your document understanding needs. With the latest update Form Recognizer now adds new capabilities like document classification, new prebuilt models like the 1098 form (with a few variants) and using Azure OpenAI models to extend field extraction with queries.

Form Recognizer has three categories of models, general extraction models which include Read, Layout and the General document model to extract content structure and fields from a form or document. Prebuilt models for extracting a defined schema for a specific document type, examples of prebuilt models include invoice, W-2, ID document and many more. Finally, custom models for classifying and extracting fields from document types specific to your scenario or use case.

The current Generally Available version of the service is being enhanced with a new set of preview capabilities.

What’s new in the latest preview?

Document classification

Common challenges in document processing workflows are identifying the document type to support conditional routing and splitting a large file into multiple documents. Until now, the most effective way to classify and route documents was composing multiple models into a single model for analysis and use an implicit classification capability to route a document to the appropriate component model.

With the addition of the document classifier models, this is now an explicit capability with a few advantages.

1. Adding the ability to ignore documents based on document class.
2. Splitting a file into multiple documents
3. Analyzing all documents of a single type when contained in a file.

The Form Recognizer Studio makes it easy to train a custom classifier model in minutes to disambiguate between the different documents your applications need to process. Try building a document classifier model today in the Form Recognizer Studio to improve your document processing capabilities.

Query fields

Sometimes the fields you need to effectively process a document are not recognized as a key value pair by the general document model. With the new premium query fields capability, Form Recognizer now leverages the power of the Azure OpenAI models to identify and extract the specific fields you require to process your documents. Try the new query fields feature in the Form Recognizer Studio. The query fields is a gated feature, only available in the EastUS region, fill out the access request form to get started.

Common name

Key name variations in documents is another challenge that requires constant updates to code or human intervention to your document processing workflows. When processing forms where the key names can vary from document to document, the General Document model now provides a “common name” to map across variations of that key. Your application now only needs to map to the common name when available. Try the new common name feature in the Form Recognizer Studio.

Barcode recognition in Read and Layout

Documents containing barcodes can now be processed more effectively, common scenarios in healthcare and retail require barcode processing. Barcodes are supported in Read and Layout. The different types of barcodes recognized are:

• QR Codes
• Code 39
• Code 128
• UPC (UPC-A & UPC-E)
• PDF417

Try the new barcode extraction in the Form Recognizer Studio.

Add On Features

The query fields are one example of a new set of features that the Form Recognizer API now supports. Other features include:

High resolution images

Some documents like engineering drawings require a higher resolution input to accurately extract text and identify features. With the added support for high resolution images, these documents are now supported in the native high resolution.

Fonts

Detecting and identifying fonts enables recreating the document with higher fidelity based on the Form Recognizer response, this also enables extensions to the semantic segmentation of documents that started with paragraphs and paragraph roles that were introduced in the previous release. Fonts are available with the Read and Layout APIs.

Formula

 LaTeX is common standard for representing mathematical formulas, the Read and Layout APIs now produce a LaTeX representation of formulas extracted from the document. Try the Formula extraction with Read or Layout in the Studio.

New 1098 prebuilt models

Tax processing scenarios just got a little easier with the addition of the 1098 prebuilt models. The new 1098 prebuilt supports a few different variants including the 1098 mortgage statement, the 1098-T for tuition and 1098-E  for student loans. Request access to try the new 1098 model in the Form Recognizer Studio.

Custom Models

Custom neural model language expansion

Neural models perform better than template models in most scenarios specifically with unstructured documents. Neural models now support additional languages.

1. Spanish
2. German
3. French
4. Italian
5. Dutch

Custom neural models provide the flexibility of a single model for document types with a few different templates, try the updated neural models in the Form Recognizer Studio.

Signature detection improvements in custom template models

Custom template models have the capability to detect the presence of a signature, the signature detection capability is significantly improved to address challenges with variations in signature format.

AI quality and Language coverage improvements

In addition to all of these updates there are always improvements to the AI quality across all models. A few significant updates are:

• Receipt model expanded to thermal receipts.
• Receipt model expanded to 10 additional languages.
• The IDs model now supports Australian Key-pass ID.
• Table detection and extraction improvements in the Layout model.
• OCR improvements for single digits or characters and common entities like dates.

 

Get started with the preview features!

The preview updates are available in only a few select regions that include US East, West US2 and West Europe. The query fields feature is available only to resources in East US.

• Train and test the document splitter and classifier in the Form Recognizer Studio in just a few clicks. Follow the how-to guide.
• Learn more about the new query fields feature.
• Visit the what's new page to learn more about all the new capabilities in Form Recognizer