Forum Discussion
Limits for training a Azure AI Document Intelligence Custom Classification Model
I have the following question: The latest 2024-02-29-preview version for training a Custom Classification model, provides great results and as vast improvement compared to the previous currentl...
[Copilot answer]
The current limitations for training a Custom Classification model in Azure AI Document Intelligence are indeed a maximum of 1 GB of training data and a limit of 10,000 pages1. These limits are in place to ensure the efficiency and effectiveness of the model training process1.
As for whether these limits will increase in future versions, it’s not explicitly stated in the available resources. However, Microsoft does mention that features, approaches, and processes may change based on user feedback12. So, it’s possible that these limits could be adjusted in the future.
Regarding the design choices to circumvent this limit, one approach could be to use multiple more specific classification models1. For instance, if you have 300 different document types, you could create several models, each trained on a subset of the document types. This could help you stay within the page limit for each model.
Another important aspect to consider is the support for incremental training starting with the 2024-02-29-preview API1. This allows you to add new samples to existing classes or add new classes by referencing an existing classifier1. This could be a useful feature to leverage when dealing with a large number of document types.
In summary, while the current limitations do pose a challenge for large-scale applications, there are potential workarounds and future updates may bring changes to these constraints. I hope this information is helpful and feel free to ask if you have more questions!
The current limitations for training a Custom Classification model in Azure AI Document Intelligence are indeed a maximum of 1 GB of training data and a limit of 10,000 pages1. These limits are in place to ensure the efficiency and effectiveness of the model training process1.
As for whether these limits will increase in future versions, it’s not explicitly stated in the available resources. However, Microsoft does mention that features, approaches, and processes may change based on user feedback12. So, it’s possible that these limits could be adjusted in the future.
Regarding the design choices to circumvent this limit, one approach could be to use multiple more specific classification models1. For instance, if you have 300 different document types, you could create several models, each trained on a subset of the document types. This could help you stay within the page limit for each model.
Another important aspect to consider is the support for incremental training starting with the 2024-02-29-preview API1. This allows you to add new samples to existing classes or add new classes by referencing an existing classifier1. This could be a useful feature to leverage when dealing with a large number of document types.
In summary, while the current limitations do pose a challenge for large-scale applications, there are potential workarounds and future updates may bring changes to these constraints. I hope this information is helpful and feel free to ask if you have more questions!