Microsoft Syntex AMA: New document processing pay-as-you-go metered services

I assume you are referring specifically to unstructured document processing. The accuracy score during model training is based on the number of predictions that match the labels in the training set. The relevance of this score will therefore be driven by how many files you have in the training set and how diverse they are. The confidence score shown in a library is broadly a product of the proximity that a file has to the all of the labels and explanations in a classifier. A tip here is to not treat this as an objective score. In real world scenarios the score may or may not give an accurate reflection of whether the outcome of the model is successful. We hide the confidence score by default, because it can be misleading as to the success or failure of the model.