This blog has been authored by Neta Haiby (Principal PM, Form Recognizer) and Prachi Jain (PMM, Azure AI)
Healthcare organizations, hospitals, government agencies are at the forefront tackling COVID-19 responses. As the fight for the pandemic continues, so does the challenges in extracting and processing information, providing quick responses, and maintaining efficiency in processes.
Forms are everywhere and various scenarios require data extraction from forms like unemployment claims, employee sick leave, loan and mortgage applications, COVID-19 relief paperwork, clinical forms and more. Extracting data from these forms today is mostly manual which takes long processing cycles. Automating the data extraction enables companies to quicken the processing time enabling productivity and helps saves cost.
As an example, we have analyzed a credit card authorization form. A credit card authorization form allows a 3rd party to make a payment by using a person’s or companies written consent and credit card information. This can either be for a 1-time charge or recurring (weekly, monthly, etc.) and is used in by insurance companies, on-boarding new patients and more. Automatic extraction of the data from these forms enables companies to speed up on-boarding new customers and reduces the processing time.
The following credit card authorization forms contains fictitious content for illustrative purposes.
To extract data from forms Form Recognizer enables you to get started with 5 forms to train a custom model and label the values of interest to extract the data you need.
Form Recognizer enables you to extract the values of interest from these forms. With Form Recognizer Sample Labeling Tool you can easily and quickly label the values of interest such as name, souse name, phone, address, transcript required and more and train a model to extract the data. You then can use this model to analyze all incoming forms and automatically extract the data as part of your workflow or Robotic Process Automation solution.
How to Label, Train and Analyze Forms –
- You will need a set of at least six forms of the same type (same structure \ format). You'll use this data to train the model and test a form.
- Go to the Azure Portal and create a Form Recognizer resource if you don’t already have one.
- Label your forms using the Form Recognizer Sample Labeling tool. You can use the try out site here or deploy it locally or in the cloud (How to deploy Form Recognizer Sample Labeling Tool). To create a new project see the Train with Labels Quick Start Guide – Setup input data. For more information on how to label forms see also the Train with Labels QuickStart guide – Label Your Forms.
- Train a custom model, click the Train icon on the left pane to open the Training page. Then click the Train button to begin training the model. For more information on how to train a custom model see also the Train with Labels QuickStart guide – Train a Custom Model. You can also train a custom model using the Form Recognizer Train Custom Model API.
5. Analyze a Form using your custom model, Click on the Predict (light bulb) icon on the left to test your model. Upload a form document that you haven't used in the training process. Then click the Predict button on the right to get key/value predictions for the form. The tool will apply tags in bounding boxes and will report the confidence of each tag. You can also analyze using the Form Recognizer Analyze Form API.
Learn more about how customers have built solutions for COVID-19 for data extraction using Microsoft Computer Vision and Form Recognizer –
Ernst & Young
EY US developed an innovative Paycheck Protection Program (PPP) Loan Forgiveness Platform using Microsoft cloud and Form Recognizer. This enables banks at this critical juncture in the US economic recovery to efficiently meet the increasing demands across the end-to-end lending process required by the unique provisions outlined under the CARES Act. Learn more here.
Automation Anywhere
Automation Anywhere has developed a solution to help accelerate the reporting and processing of many complex forms, including CRFs and E28.This highly secure solution, comprised of Automation Anywhere RPA with native Intelligent Document Processing (IDP) and Azure Cognitive Services Computer Vision API and Form Recognizer. Learn more here.
Additional Resources
Get started with deploying Form Recognizer –
- Custom Model – extract text, tables and key value pairs
- QuickStart: Train a Form Recognizer model and extract form data by using the REST API
- QuickStart: Train a Form Recognizer model with labels using the sample labeling tool
- Form Recognizer Sample Labeling Tool
- Try it out: https://fott.azurewebsites.net/
- Open Source project: https://github.com/microsoft/OCR-Form-Tools
- Prebuilt receipts - extract data from USA sales receipts
- Layout - extract text and table structure (row and column numbers) from your documents
- See What’s New