SharePoint syntex doesn't detect handwritten form data

Copper Contributor

I have created a SharePoint syntex AI Builder form processing model to extract data from medical insurance claim forms.

I have used a few examples of medical insurance claim forms pdf files. few of them are handwritten forms and the rest are electronic forms.


The issue is:

  • The model can extract data from electronic forms.
  • but for handwritten forms.
    1. It doesn't detect checked checkboxes for all the handwritten forms. DalbirS_2-1661313405584.jpg


    2. For some forms, the model is unable to detect some Text data. DalbirS_1-1661313386167.jpg


    3. and sometimes it detects wrong data.      DalbirS_0-1661313330376.jpg



  • Yellow highlighted: handwritten form data.
  • Green highlighted: extracted data from handwritten form using form processing model.


Click here for pdf files which i am using.

3 Replies
I've used Syntex for handwritten content and results can vary. What I've found is that if I use the "problem records" to do the training I get better results. So in your example where the Amount Paid field comes in blank, use that form as one of the training files.
Also, I sometimes will do PDF cleanup prior to running the models (e.g., convert from color to B&W or use deskew). These practices usually enable me to get very high accuracy (at least as high as I can read with my human eyes - e.g., the Insured's ID Number above could be 1EGYTES or could be 7E64TES) but if I can't tell as a human how could the AI do any better.
Thank you so much for your quick and helpful response.
Yes, I agree with you like in some cases even we can’t clearly read as a human then how could AI do. But in this case, we must have something to handle this situation.
Steps which I have performed:
• I have followed the same thing which you have suggested. I have used the “problem records” for training. but the model was not detected the amount paid ‘45’.
• I have converted color pdf to B&W. Still getting the same result.
Is there any other way or anything which I am missing?
Are you able to extract selected checkbox values from handwritten forms?
I have had great success with checkbox values from handwritten forms (almost 100%) using AI Builder. Not good success with document understanding models (but it sounds like you are using AI Builder). Not sure why you would have these issues. I could try your docs you shared above, but will have to find time in my "day job"