Document Field Extraction with Generative AI

Adoption of Generative AI technologies is accelerating, driven by the transformative potential they offer across various industry sectors. Azure AI enables organizations to create interactive and responsive AI solutions customized to their requirements, playing a significant part helping businesses harness Generative AI effectively. With the new custom field extraction preview, you can leverage generative AI to efficiently extract fields from documents, ensuring standardized output and a repeatable process to support document automation workflows. 

Field Extraction using Large Language Models 

To extract fields from documents using Large Language Models (LLMs) or Generative AI, you typically need to create a complex orchestration workflow, as shown below, that includes multiple services to manage tasks like text extraction, document chunking, vectorization, search index creation, and prompt engineering. 

• Size and Complexity of Prompts: Managing prompts to accommodate variations can be difficult, resulting in a large number of prompts and associated costs.
• Inconsistent Results: Results may vary across multiple runs of the same document, leading to reliability issues.
• Grounding: Ensuring that values are accurately extracted and traceable to address issues with hallucination.
• Lack of Confidence Scores: Absence of confidence scores makes it challenging to automate downstream processes.

Imagine harnessing the benefits of generative AI without the complexities of developing your own workflow. With the new custom field extraction capability, you simply define your schema, let the model extract the necessary fields, and correct any prediction errors. Once model is trained, you can integrate the model into your document processing workflows with a single API call. This approach provides grounded results and confidence scores, offering guardrails to ensure the extracted values align with your business needs.  

Azure AI Document Intelligence   

Azure AI Document Intelligence is an AI service offering a streamlined set of APIs and a studio experience to efficiently extract content, structure (such as tables, paragraphs, sections, and figures), and fields - whether predefined for specific document types or custom-defined for any document or form. With the Document Intelligence APIs, you can easily split, classify, and extract fields or content from any document or form at scale, tailored to meet your business needs. The latest Document field extraction model leverages generative AI to extract user-specified fields from documents across a wide variety of visual templates. This custom extraction model combines the power of document understanding with Large Language Models (LLMs) and the rigor and schema from custom extraction capabilities to create a model with high accuracy in minutes. 

Why Choose Azure Document Field Extraction? 

• Accuracy and Reliability: Our AI models are built to deliver accurate data extraction, reducing errors and improving efficiency. 
• Scalability: Easily scale your document processing capabilities to meet the growing demands of your business. 
• Customizability: Tailor our extraction models to your specific requirements, ensuring the perfect fit for your unique workflows. 
• Grounded results: Localize the data extracted in the documents, ensuring the response is generated from the content, to enable human review workflows. 
• Confidence scores: Maximize efficiency and minimize costs in automation workflows, leveraging confidence scores. 
• Cost Efficiency: With our new pricing, enjoy the best-in-class AI technology at a fraction of the cost. 

Building a Custom Field Extraction Model

The new field extraction model is available in Azure AI Studio under AI Services - Vision + Document. Start by creating a project to work with your documents. 

• Once you select on the project, you should now be in the Define schema window. The files you uploaded are listed and you can use the drop-down option to select files. You can start adding fields by clicking on the Add new field button. Enter a name, description, and type for the field to be extracted. Once all the fields are added, select the save button at the bottom of the screen.

• After the schema is saved, all the uploaded training documents are analyzed, and field values are automatically extracted. The auto extracted fields are tagged as Predicted. Review the predicted values. If the field value is incorrect or isn't extracted, you can hover over the predicted field. Select the edit button to make the changes and after the labels are reviewed and corrected for all the training documents, proceed to build your model.

• On the Build model dialog page, provide a unique model name and, optionally, a description. Select Build to initiate the training process. Generative models train instantly! Refresh the page to select the model once status is changed to succeeded.
• Once the model training is complete, you can test your model by selecting Test button. Upload your test files and select Run Analysis to extract field values from the documents. Validate your model accuracy by evaluating the results for each field.

You can use the REST API or client libraries to submit a document for analysis. The custom generative AI model is highly effective at extracting simple fields from documents without requiring labeled samples. However, providing a few labeled samples can significantly enhance the extraction accuracy for more complex fields and user-defined fields like tables. 

Business Scenarios  

• Loan & Mortgage Applications - Automation of loan and mortgage application process enables banks, lenders, and government entities to process loan and mortgage applications quicker. 
• Financial Services - Analyze complex documents like financial reports and asset management reports, with the new custom field extraction model.  
• Contract Lifecycle Management - Build a custom field extraction model to extract the fields, clauses, and obligations from a wide array of contract types. 
• Expense Management - Receipts and invoices from various retailers and businesses need to be parsed to validate the expenses. Custom field extraction can extract expenses across different formats and documents with varying templates. 

Get Started! 

Custom generative models are available with the 2024-07-31-preview version and later models. To learn how to build and train a custom field extraction model using generative AI, you can follow the instructions here - Use AI Studio to build and train a custom field extraction.  Start building your custom document field extraction models today!