Process Covid vaccine proof with AI and validate using logic apps

Published Nov 02 2021 09:00 AM 823 Views

Using AI to process Covid vaccine card output and validate

Note

  • All the data set here is from Google images
  • All publicly available dataset
  • There is no PII data (all Fake)

Thanks to @Prateek Gandhi 

Architecture

  • End to End process using Azure Form recognizer and Custom Vision cognitive Services using Logic apps
 

vaccinecardarch.jpg

 

Steps

  • Before we can create logic app we need to build 2 cognitive services model

Azure Form recognizer - Extract Covid vaccine information

  • Create a storage account
  • Upload the images
  • for the container get a SAS URL with expiration
  • Go to https://fott-2-1.azurewebsites.net/ - based on new version, URL might change
  • Create a connection to Blob using SAS URL and create a project
 
 
  • Create Tags
  • Assign the appropriate values to tag
  • Go to Model and Train the model with name

Azure Custom Vision - Extract CDC logo

  • Create a new Custom vision service
  • Upload minimum 15 images
 

 

  • All images used are open source fake data images from google
  • Draw bounding boxes and tag then with name
  • Now click Train
 

 

  • Now click Train and see the model performance
  • Will take few minutes to complete the model training

cv5.jpg

 

  • Do a quick test

cv6.jpg

 

Create Flow with Logic App to automate processing

  • Once we have the above 2 model ready then we can now automate the processing end to end

  • I choose logic app to show the process

  • This is not the end complete solution

  • Entire flow

cv7.jpg

 

cv8.jpg

 

cv9.jpg

  • i am using blob SAS with expiration for security purpose
  • Trigger the logic with blob trigger
  • Create a variable to store form recognizer output URL - header - operation-location
  • Using All HTTP rest connector
  • make sure you have the prediction keys for both Form recognizer and custom vision
  • For Form recognizer it's a 2 step process
  • First send the request to analze
  • then wait for 15 seconds
  • send the analyzeresult url from previous header
  • Make sure content-type and Ocp-Apim-Subscription-Key are sent
@{outputs('HTTP')['headers']?['Operation-Location']}
  • the above is to get the output of HTTP header for analyze request
  • Then send the blob SAS URI to custom vision prediction URL
  • Get the prediction URL from customvision.ai web site
  • Make sure the content-type and prediction-key are sent
  • All the output are stored in blob storage as json

cv10.jpg

 

  • we can do further processing based on requirements and use case

  • Form recognizer output

{"status":"succeeded","createdDateTime":"2021-09-28T12:43:04Z","lastUpdatedDateTime":"2021-09-28T12:43:08Z","analyzeResult":{"version":"2.1.0","readResults":[{"page":1,"angle":5.7989,"width":1130,"height":900,"unit":"pixel","selectionMarks":[{"boundingBox":[1037,184,1069,184,1069,221,1037,221],"confidence":0.223,"state":"unselected"}]}],"pageResults":[{"page":1,"tables":[{"rows":8,"columns":4,"cells":[{"rowIndex":0,"columnIndex":0,"rowSpan":2,"text":"Vaccine","boundingBox":[69,385,205,400,198,479,62,467],"isHeader":true},{"rowIndex":0,"columnIndex":1,"text":"Product Name/Manufacturer","boundingBox":[205,400,553,435,550,475,202,440],"isHeader":true},{"rowIndex":0,"columnIndex":2,"rowSpan":2,"text":"Date","boundingBox":[553,435,747,455,742,534,547,514],"isHeader":true},{"rowIndex":0,"columnIndex":3,"rowSpan":2,"text":"Healthcare Professional or Clinic Site","boundingBox":[747,455,1052,485,1048,564,742,534],"isHeader":true},{"rowIndex":1,"columnIndex":1,"text":"Lot Number","boundingBox":[202,440,550,475,547,514,198,479],"isHeader":true},{"rowIndex":2,"columnIndex":0,"rowSpan":2,"text":"1\" Dose COVID-19","boundingBox":[62,467,198,479,190,560,54,547],"isHeader":false},{"rowIndex":2,"columnIndex":1,"text":"","boundingBox":[198,479,547,514,545,553,195,519],"isHeader":false},{"rowIndex":2,"columnIndex":2,"text":"","boundingBox":[547,514,742,534,739,572,545,553],"isHeader":false},{"rowIndex":2,"columnIndex":3,"rowSpan":2,"text":"","boundingBox":[742,534,1048,564,1043,643,736,613],"isHeader":false},{"rowIndex":3,"columnIndex":1,"text":"","boundingBox":[195,519,545,553,541,594,190,560],"isHeader":false},{"rowIndex":3,"columnIndex":2,"text":"mm dd yy","boundingBox":[545,553,739,572,736,613,541,594],"isHeader":false},{"rowIndex":4,"columnIndex":0,"text":"2nd Dose","boundingBox":[54,547,190,560,187,600,51,588],"isHeader":false},{"rowIndex":4,"columnIndex":1,"text":"","boundingBox":[190,560,541,594,538,633,187,600],"isHeader":false},{"rowIndex":4,"columnIndex":2,"text":"","boundingBox":[541,594,736,613,733,652,538,633],"isHeader":false},{"rowIndex":4,"columnIndex":3,"text":"","boundingBox":[736,613,1043,643,1041,681,733,652],"isHeader":false},{"rowIndex":5,"columnIndex":0,"text":"COVID-19","boundingBox":[51,588,187,600,183,640,47,627],"isHeader":false},{"rowIndex":5,"columnIndex":1,"text":"","boundingBox":[187,600,538,633,535,674,183,640],"isHeader":false},{"rowIndex":5,"columnIndex":2,"text":"mm dd yy","boundingBox":[538,633,733,652,730,694,535,674],"isHeader":false},{"rowIndex":5,"columnIndex":3,"text":"","boundingBox":[733,652,1041,681,1039,723,730,694],"isHeader":false},{"rowIndex":6,"columnIndex":0,"text":"Other","boundingBox":[47,627,183,640,178,707,42,693],"isHeader":false},{"rowIndex":6,"columnIndex":1,"text":"","boundingBox":[183,640,535,674,530,740,178,707],"isHeader":false},{"rowIndex":6,"columnIndex":2,"text":"mm dd yy","boundingBox":[535,674,730,694,726,759,530,740],"isHeader":false},{"rowIndex":6,"columnIndex":3,"text":"","boundingBox":[730,694,1039,723,1036,790,726,759],"isHeader":false},{"rowIndex":7,"columnIndex":0,"text":"Other","boundingBox":[42,693,178,707,171,777,35,765],"isHeader":false},{"rowIndex":7,"columnIndex":1,"text":"","boundingBox":[178,707,530,740,525,811,171,777],"isHeader":false},{"rowIndex":7,"columnIndex":2,"text":"mm dd yy","boundingBox":[530,740,726,759,721,821,525,811],"isHeader":false},{"rowIndex":7,"columnIndex":3,"text":"","boundingBox":[726,759,1036,790,1034,821,721,821],"isHeader":false}],"boundingBox":[73,379,1067,450,1029,821,34,748]}]}],"documentResults":[{"docType":"custom:vacinnecard1","modelId":"3dea3bee-1230-47ed-a8d4-7240c6cf36ff","pageRange":[1,1],"fields":{"Firstname":{"type":"string","confidence":0.943},"vacineserial1":{"type":"string","confidence":0.919},"Lastname":{"type":"string","confidence":0.919},"vaccinename1":{"type":"string","confidence":0.919},"vacinedate1":{"type":"string","confidence":0.919},"vaccinename":{"type":"string","valueString":"Lot Number","text":"Lot Number","page":1,"boundingBox":[210.0,450.0,351.0,450.0,351.0,487.0,210.0,487.0],"confidence":0.99},"healthcarename":{"type":"string","confidence":0.97},"date":{"type":"string","confidence":0.974},"healthcareid":{"type":"string","confidence":0.974},"vacineserial":{"type":"string","confidence":0.973}},"docTypeConfidence":0.589}],"errors":[]}}
  • CDC logo output
{"id":"8380b276-314d-40c5-9b04-c8564ca363dd","project":"93aa72a0-1f37-4097-8f95-d9002a14f4c5","iteration":"5903b0e8-9706-4443-a541-4912015208c0","created":"2021-09-28T12:43:23.684Z","predictions":[{"probability":0.9998611,"tagId":"68b532c0-6de5-4aba-8074-8f234099d1e7","tagName":"CDClogo","boundingBox":{"left":0.7381513,"top":0.12175512,"width":0.22355062,"height":0.18526697}},{"probability":0.44562727,"tagId":"68b532c0-6de5-4aba-8074-8f234099d1e7","tagName":"CDClogo","boundingBox":{"left":0.62779665,"top":0.120928116,"width":0.37220335,"height":0.1991472}},{"probability":0.011576932,"tagId":"68b532c0-6de5-4aba-8074-8f234099d1e7","tagName":"CDClogo","boundingBox":{"left":0.69671506,"top":0.044883475,"width":0.30187488,"height":0.33183825}}]}

Do it yourself link - Samples2021/covidvaccineai.md at main · balakreshnan/Samples2021 (github.com)

Medium link  - Covid Vaccine Card processing using Azure Form Recognizer and Custom Vision | by Balamurugan Balakre...

%3CLINGO-SUB%20id%3D%22lingo-sub-2892962%22%20slang%3D%22en-US%22%3EProcess%20Covid%20vaccine%20proof%20with%20AI%20and%20validate%20using%20logic%20apps%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2892962%22%20slang%3D%22en-US%22%3E%3CH2%20id%3D%22toc-hId--320178538%22%20id%3D%22toc-hId--320157519%22%3EUsing%20AI%20to%20process%20Covid%20vaccine%20card%20output%20and%20validate%3C%2FH2%3E%0A%3CH2%20id%3D%22toc-hId--2127633001%22%20id%3D%22toc-hId--2127611982%22%3E%3CA%20id%3D%22user-content-note%22%20class%3D%22anchor%22%20href%3D%22https%3A%2F%2Fgithub.com%2Fbalakreshnan%2FSamples2021%2Fblob%2Fmain%2Fcovidvaccine%2Fcovidvaccineai.md%23note%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%20aria-hidden%3D%22true%22%3E%3C%2FA%3ENote%3C%2FH2%3E%0A%3CUL%3E%0A%3CLI%3EAll%20the%20data%20set%20here%20is%20from%20Google%20images%3C%2FLI%3E%0A%3CLI%3EAll%20publicly%20available%20dataset%3C%2FLI%3E%0A%3CLI%3EThere%20is%20no%20PII%20data%20(all%20Fake)%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3EThanks%20to%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F227022%22%20target%3D%22_blank%22%3E%40Prateek%20Gandhi%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-359879832%22%20id%3D%22toc-hId-359900851%22%3E%3CA%20id%3D%22user-content-architecture%22%20class%3D%22anchor%22%20href%3D%22https%3A%2F%2Fgithub.com%2Fbalakreshnan%2FSamples2021%2Fblob%2Fmain%2Fcovidvaccine%2Fcovidvaccineai.md%23architecture%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%20aria-hidden%3D%22true%22%3E%3C%2FA%3EArchitecture%3C%2FH2%3E%0A%3CUL%3E%0A%3CLI%3EEnd%20to%20End%20process%20using%20Azure%20Form%20recognizer%20and%20Custom%20Vision%20cognitive%20Services%20using%20Logic%20apps%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CDIV%20id%3D%22tinyMceEditorBalamuruganBalakreshnan_0%22%20class%3D%22mceNonEditable%20lia-copypaste-placeholder%22%3E%26nbsp%3B%3C%2FDIV%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22vaccinecardarch.jpg%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F320601i2D4E637C8F8135D3%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22vaccinecardarch.jpg%22%20alt%3D%22vaccinecardarch.jpg%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId--1447574631%22%20id%3D%22toc-hId--1447553612%22%3ESteps%3C%2FH2%3E%0A%3CUL%3E%0A%3CLI%3EBefore%20we%20can%20create%20logic%20app%20we%20need%20to%20build%202%20cognitive%20services%20model%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CH2%20id%3D%22toc-hId-1039938202%22%20id%3D%22toc-hId-1039959221%22%3E%3CA%20id%3D%22user-content-azure-form-recognizer---extract-covid-vaccine-information%22%20class%3D%22anchor%22%20href%3D%22https%3A%2F%2Fgithub.com%2Fbalakreshnan%2FSamples2021%2Fblob%2Fmain%2Fcovidvaccine%2Fcovidvaccineai.md%23azure-form-recognizer---extract-covid-vaccine-information%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%20aria-hidden%3D%22true%22%3E%3C%2FA%3EAzure%20Form%20recognizer%20-%20Extract%20Covid%20vaccine%20information%3C%2FH2%3E%0A%3CUL%3E%0A%3CLI%3ECreate%20a%20storage%20account%3C%2FLI%3E%0A%3CLI%3EUpload%20the%20images%3C%2FLI%3E%0A%3CLI%3Efor%20the%20container%20get%20a%20SAS%20URL%20with%20expiration%3C%2FLI%3E%0A%3CLI%3EGo%20to%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CA%20href%3D%22https%3A%2F%2Ffott-2-1.azurewebsites.net%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3Ehttps%3A%2F%2Ffott-2-1.azurewebsites.net%2F%3C%2FA%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E-%20based%20on%20new%20version%2C%20URL%20might%20change%3C%2FLI%3E%0A%3CLI%3ECreate%20a%20connection%20to%20Blob%20using%20SAS%20URL%20and%20create%20a%20project%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CDIV%20id%3D%22tinyMceEditorBalamuruganBalakreshnan_1%22%20class%3D%22mceNonEditable%20lia-copypaste-placeholder%22%3E%26nbsp%3B%3C%2FDIV%3E%0A%3CDIV%20id%3D%22tinyMceEditorBalamuruganBalakreshnan_2%22%20class%3D%22mceNonEditable%20lia-copypaste-placeholder%22%3E%26nbsp%3B%3C%2FDIV%3E%0A%3CUL%3E%0A%3CLI%3ECreate%20Tags%3C%2FLI%3E%0A%3CLI%3EAssign%20the%20appropriate%20values%20to%20tag%3C%2FLI%3E%0A%3CLI%3EGo%20to%20Model%20and%20Train%20the%20model%20with%20name%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CH2%20id%3D%22toc-hId--767516261%22%20id%3D%22toc-hId--767495242%22%3E%3CA%20id%3D%22user-content-azure-custom-vision---extract-cdc-logo%22%20class%3D%22anchor%22%20href%3D%22https%3A%2F%2Fgithub.com%2Fbalakreshnan%2FSamples2021%2Fblob%2Fmain%2Fcovidvaccine%2Fcovidvaccineai.md%23azure-custom-vision---extract-cdc-logo%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%20aria-hidden%3D%22true%22%3E%3C%2FA%3EAzure%20Custom%20Vision%20-%20Extract%20CDC%20logo%3C%2FH2%3E%0A%3CUL%3E%0A%3CLI%3ECreate%20a%20new%20Custom%20vision%20service%3C%2FLI%3E%0A%3CLI%3EUpload%20minimum%2015%20images%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CDIV%20id%3D%22tinyMceEditorBalamuruganBalakreshnan_3%22%20class%3D%22mceNonEditable%20lia-copypaste-placeholder%22%3E%26nbsp%3B%3C%2FDIV%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3EAll%20images%20used%20are%20open%20source%20fake%20data%20images%20from%20google%3C%2FLI%3E%0A%3CLI%3EDraw%20bounding%20boxes%20and%20tag%20then%20with%20name%3C%2FLI%3E%0A%3CLI%3ENow%20click%20Train%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CDIV%20id%3D%22tinyMceEditorBalamuruganBalakreshnan_4%22%20class%3D%22mceNonEditable%20lia-copypaste-placeholder%22%3E%26nbsp%3B%3C%2FDIV%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3ENow%20click%20Train%20and%20see%20the%20model%20performance%3C%2FLI%3E%0A%3CLI%3EWill%20take%20few%20minutes%20to%20complete%20the%20model%20training%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22cv5.jpg%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F320609i154A3A5502027D40%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22cv5.jpg%22%20alt%3D%22cv5.jpg%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3EDo%20a%20quick%20test%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22cv6.jpg%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F320613i848228E1DC573558%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22cv6.jpg%22%20alt%3D%22cv6.jpg%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-1719996572%22%20id%3D%22toc-hId-1720017591%22%3ECreate%20Flow%20with%20Logic%20App%20to%20automate%20processing%3C%2FH2%3E%0A%3CUL%3E%0A%3CLI%3E%3CP%3EOnce%20we%20have%20the%20above%202%20model%20ready%20then%20we%20can%20now%20automate%20the%20processing%20end%20to%20end%3C%2FP%3E%0A%3C%2FLI%3E%0A%3CLI%3E%3CP%3EI%20choose%20logic%20app%20to%20show%20the%20process%3C%2FP%3E%0A%3C%2FLI%3E%0A%3CLI%3E%3CP%3EThis%20is%20not%20the%20end%20complete%20solution%3C%2FP%3E%0A%3C%2FLI%3E%0A%3CLI%3E%3CP%3EEntire%20flow%3C%2FP%3E%0A%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22cv7.jpg%22%20style%3D%22width%3A%20391px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F320614i3A88E31DC5D757B5%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22cv7.jpg%22%20alt%3D%22cv7.jpg%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22cv8.jpg%22%20style%3D%22width%3A%20380px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F320615i67B4D476F40007DE%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22cv8.jpg%22%20alt%3D%22cv8.jpg%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22cv9.jpg%22%20style%3D%22width%3A%20483px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F320616iD37B10AA43D45CC6%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22cv9.jpg%22%20alt%3D%22cv9.jpg%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3Ei%20am%20using%20blob%20SAS%20with%20expiration%20for%20security%20purpose%3C%2FLI%3E%0A%3CLI%3ETrigger%20the%20logic%20with%20blob%20trigger%3C%2FLI%3E%0A%3CLI%3ECreate%20a%20variable%20to%20store%20form%20recognizer%20output%20URL%20-%20header%20-%20operation-location%3C%2FLI%3E%0A%3CLI%3EUsing%20All%20HTTP%20rest%20connector%3C%2FLI%3E%0A%3CLI%3Emake%20sure%20you%20have%20the%20prediction%20keys%20for%20both%20Form%20recognizer%20and%20custom%20vision%3C%2FLI%3E%0A%3CLI%3EFor%20Form%20recognizer%20it's%20a%202%20step%20process%3C%2FLI%3E%0A%3CLI%3EFirst%20send%20the%20request%20to%20analze%3C%2FLI%3E%0A%3CLI%3Ethen%20wait%20for%2015%20seconds%3C%2FLI%3E%0A%3CLI%3Esend%20the%20analyzeresult%20url%20from%20previous%20header%3C%2FLI%3E%0A%3CLI%3EMake%20sure%20content-type%20and%20Ocp-Apim-Subscription-Key%20are%20sent%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CDIV%20class%3D%22snippet-clipboard-content%20position-relative%20overflow-auto%22%3E%0A%3CPRE%3E%3CCODE%3E%40%7Boutputs('HTTP')%5B'headers'%5D%3F%5B'Operation-Location'%5D%7D%0A%3C%2FCODE%3E%3C%2FPRE%3E%0A%3C%2FDIV%3E%0A%3CUL%3E%0A%3CLI%3Ethe%20above%20is%20to%20get%20the%20output%20of%20HTTP%20header%20for%20analyze%20request%3C%2FLI%3E%0A%3CLI%3EThen%20send%20the%20blob%20SAS%20URI%20to%20custom%20vision%20prediction%20URL%3C%2FLI%3E%0A%3CLI%3EGet%20the%20prediction%20URL%20from%20customvision.ai%20web%20site%3C%2FLI%3E%0A%3CLI%3EMake%20sure%20the%20content-type%20and%20prediction-key%20are%20sent%3C%2FLI%3E%0A%3CLI%3EAll%20the%20output%20are%20stored%20in%20blob%20storage%20as%20json%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22cv10.jpg%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F320617iF4371865F3A401F4%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22cv10.jpg%22%20alt%3D%22cv10.jpg%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3E%3CP%3Ewe%20can%20do%20further%20processing%20based%20on%20requirements%20and%20use%20case%3C%2FP%3E%0A%3C%2FLI%3E%0A%3CLI%3E%3CP%3EForm%20recognizer%20output%3C%2FP%3E%0A%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CDIV%20class%3D%22snippet-clipboard-content%20position-relative%20overflow-auto%22%3E%0A%3CPRE%3E%3CCODE%3E%7B%22status%22%3A%22succeeded%22%2C%22createdDateTime%22%3A%222021-09-28T12%3A43%3A04Z%22%2C%22lastUpdatedDateTime%22%3A%222021-09-28T12%3A43%3A08Z%22%2C%22analyzeResult%22%3A%7B%22version%22%3A%222.1.0%22%2C%22readResults%22%3A%5B%7B%22page%22%3A1%2C%22angle%22%3A5.7989%2C%22width%22%3A1130%2C%22height%22%3A900%2C%22unit%22%3A%22pixel%22%2C%22selectionMarks%22%3A%5B%7B%22boundingBox%22%3A%5B1037%2C184%2C1069%2C184%2C1069%2C221%2C1037%2C221%5D%2C%22confidence%22%3A0.223%2C%22state%22%3A%22unselected%22%7D%5D%7D%5D%2C%22pageResults%22%3A%5B%7B%22page%22%3A1%2C%22tables%22%3A%5B%7B%22rows%22%3A8%2C%22columns%22%3A4%2C%22cells%22%3A%5B%7B%22rowIndex%22%3A0%2C%22columnIndex%22%3A0%2C%22rowSpan%22%3A2%2C%22text%22%3A%22Vaccine%22%2C%22boundingBox%22%3A%5B69%2C385%2C205%2C400%2C198%2C479%2C62%2C467%5D%2C%22isHeader%22%3Atrue%7D%2C%7B%22rowIndex%22%3A0%2C%22columnIndex%22%3A1%2C%22text%22%3A%22Product%20Name%2FManufacturer%22%2C%22boundingBox%22%3A%5B205%2C400%2C553%2C435%2C550%2C475%2C202%2C440%5D%2C%22isHeader%22%3Atrue%7D%2C%7B%22rowIndex%22%3A0%2C%22columnIndex%22%3A2%2C%22rowSpan%22%3A2%2C%22text%22%3A%22Date%22%2C%22boundingBox%22%3A%5B553%2C435%2C747%2C455%2C742%2C534%2C547%2C514%5D%2C%22isHeader%22%3Atrue%7D%2C%7B%22rowIndex%22%3A0%2C%22columnIndex%22%3A3%2C%22rowSpan%22%3A2%2C%22text%22%3A%22Healthcare%20Professional%20or%20Clinic%20Site%22%2C%22boundingBox%22%3A%5B747%2C455%2C1052%2C485%2C1048%2C564%2C742%2C534%5D%2C%22isHeader%22%3Atrue%7D%2C%7B%22rowIndex%22%3A1%2C%22columnIndex%22%3A1%2C%22text%22%3A%22Lot%20Number%22%2C%22boundingBox%22%3A%5B202%2C440%2C550%2C475%2C547%2C514%2C198%2C479%5D%2C%22isHeader%22%3Atrue%7D%2C%7B%22rowIndex%22%3A2%2C%22columnIndex%22%3A0%2C%22rowSpan%22%3A2%2C%22text%22%3A%221%5C%22%20Dose%20COVID-19%22%2C%22boundingBox%22%3A%5B62%2C467%2C198%2C479%2C190%2C560%2C54%2C547%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A2%2C%22columnIndex%22%3A1%2C%22text%22%3A%22%22%2C%22boundingBox%22%3A%5B198%2C479%2C547%2C514%2C545%2C553%2C195%2C519%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A2%2C%22columnIndex%22%3A2%2C%22text%22%3A%22%22%2C%22boundingBox%22%3A%5B547%2C514%2C742%2C534%2C739%2C572%2C545%2C553%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A2%2C%22columnIndex%22%3A3%2C%22rowSpan%22%3A2%2C%22text%22%3A%22%22%2C%22boundingBox%22%3A%5B742%2C534%2C1048%2C564%2C1043%2C643%2C736%2C613%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A3%2C%22columnIndex%22%3A1%2C%22text%22%3A%22%22%2C%22boundingBox%22%3A%5B195%2C519%2C545%2C553%2C541%2C594%2C190%2C560%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A3%2C%22columnIndex%22%3A2%2C%22text%22%3A%22mm%20dd%20yy%22%2C%22boundingBox%22%3A%5B545%2C553%2C739%2C572%2C736%2C613%2C541%2C594%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A4%2C%22columnIndex%22%3A0%2C%22text%22%3A%222nd%20Dose%22%2C%22boundingBox%22%3A%5B54%2C547%2C190%2C560%2C187%2C600%2C51%2C588%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A4%2C%22columnIndex%22%3A1%2C%22text%22%3A%22%22%2C%22boundingBox%22%3A%5B190%2C560%2C541%2C594%2C538%2C633%2C187%2C600%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A4%2C%22columnIndex%22%3A2%2C%22text%22%3A%22%22%2C%22boundingBox%22%3A%5B541%2C594%2C736%2C613%2C733%2C652%2C538%2C633%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A4%2C%22columnIndex%22%3A3%2C%22text%22%3A%22%22%2C%22boundingBox%22%3A%5B736%2C613%2C1043%2C643%2C1041%2C681%2C733%2C652%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A5%2C%22columnIndex%22%3A0%2C%22text%22%3A%22COVID-19%22%2C%22boundingBox%22%3A%5B51%2C588%2C187%2C600%2C183%2C640%2C47%2C627%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A5%2C%22columnIndex%22%3A1%2C%22text%22%3A%22%22%2C%22boundingBox%22%3A%5B187%2C600%2C538%2C633%2C535%2C674%2C183%2C640%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A5%2C%22columnIndex%22%3A2%2C%22text%22%3A%22mm%20dd%20yy%22%2C%22boundingBox%22%3A%5B538%2C633%2C733%2C652%2C730%2C694%2C535%2C674%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A5%2C%22columnIndex%22%3A3%2C%22text%22%3A%22%22%2C%22boundingBox%22%3A%5B733%2C652%2C1041%2C681%2C1039%2C723%2C730%2C694%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A6%2C%22columnIndex%22%3A0%2C%22text%22%3A%22Other%22%2C%22boundingBox%22%3A%5B47%2C627%2C183%2C640%2C178%2C707%2C42%2C693%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A6%2C%22columnIndex%22%3A1%2C%22text%22%3A%22%22%2C%22boundingBox%22%3A%5B183%2C640%2C535%2C674%2C530%2C740%2C178%2C707%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A6%2C%22columnIndex%22%3A2%2C%22text%22%3A%22mm%20dd%20yy%22%2C%22boundingBox%22%3A%5B535%2C674%2C730%2C694%2C726%2C759%2C530%2C740%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A6%2C%22columnIndex%22%3A3%2C%22text%22%3A%22%22%2C%22boundingBox%22%3A%5B730%2C694%2C1039%2C723%2C1036%2C790%2C726%2C759%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A7%2C%22columnIndex%22%3A0%2C%22text%22%3A%22Other%22%2C%22boundingBox%22%3A%5B42%2C693%2C178%2C707%2C171%2C777%2C35%2C765%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A7%2C%22columnIndex%22%3A1%2C%22text%22%3A%22%22%2C%22boundingBox%22%3A%5B178%2C707%2C530%2C740%2C525%2C811%2C171%2C777%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A7%2C%22columnIndex%22%3A2%2C%22text%22%3A%22mm%20dd%20yy%22%2C%22boundingBox%22%3A%5B530%2C740%2C726%2C759%2C721%2C821%2C525%2C811%5D%2C%22isHeader%22%3Afalse%7D%2C%7B%22rowIndex%22%3A7%2C%22columnIndex%22%3A3%2C%22text%22%3A%22%22%2C%22boundingBox%22%3A%5B726%2C759%2C1036%2C790%2C1034%2C821%2C721%2C821%5D%2C%22isHeader%22%3Afalse%7D%5D%2C%22boundingBox%22%3A%5B73%2C379%2C1067%2C450%2C1029%2C821%2C34%2C748%5D%7D%5D%7D%5D%2C%22documentResults%22%3A%5B%7B%22docType%22%3A%22custom%3Avacinnecard1%22%2C%22modelId%22%3A%223dea3bee-1230-47ed-a8d4-7240c6cf36ff%22%2C%22pageRange%22%3A%5B1%2C1%5D%2C%22fields%22%3A%7B%22Firstname%22%3A%7B%22type%22%3A%22string%22%2C%22confidence%22%3A0.943%7D%2C%22vacineserial1%22%3A%7B%22type%22%3A%22string%22%2C%22confidence%22%3A0.919%7D%2C%22Lastname%22%3A%7B%22type%22%3A%22string%22%2C%22confidence%22%3A0.919%7D%2C%22vaccinename1%22%3A%7B%22type%22%3A%22string%22%2C%22confidence%22%3A0.919%7D%2C%22vacinedate1%22%3A%7B%22type%22%3A%22string%22%2C%22confidence%22%3A0.919%7D%2C%22vaccinename%22%3A%7B%22type%22%3A%22string%22%2C%22valueString%22%3A%22Lot%20Number%22%2C%22text%22%3A%22Lot%20Number%22%2C%22page%22%3A1%2C%22boundingBox%22%3A%5B210.0%2C450.0%2C351.0%2C450.0%2C351.0%2C487.0%2C210.0%2C487.0%5D%2C%22confidence%22%3A0.99%7D%2C%22healthcarename%22%3A%7B%22type%22%3A%22string%22%2C%22confidence%22%3A0.97%7D%2C%22date%22%3A%7B%22type%22%3A%22string%22%2C%22confidence%22%3A0.974%7D%2C%22healthcareid%22%3A%7B%22type%22%3A%22string%22%2C%22confidence%22%3A0.974%7D%2C%22vacineserial%22%3A%7B%22type%22%3A%22string%22%2C%22confidence%22%3A0.973%7D%7D%2C%22docTypeConfidence%22%3A0.589%7D%5D%2C%22errors%22%3A%5B%5D%7D%7D%3CBR%20%2F%3E%0A%3C%2FCODE%3E%3C%2FPRE%3E%0A%3C%2FDIV%3E%0A%3CUL%3E%0A%3CLI%3ECDC%20logo%20output%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CDIV%20class%3D%22snippet-clipboard-content%20position-relative%20overflow-auto%22%3E%0A%3CPRE%3E%7B%22id%22%3A%228380b276-314d-40c5-9b04-c8564ca363dd%22%2C%22project%22%3A%2293aa72a0-1f37-4097-8f95-d9002a14f4c5%22%2C%22iteration%22%3A%225903b0e8-9706-4443-a541-4912015208c0%22%2C%22created%22%3A%222021-09-28T12%3A43%3A23.684Z%22%2C%22predictions%22%3A%5B%7B%22probability%22%3A0.9998611%2C%22tagId%22%3A%2268b532c0-6de5-4aba-8074-8f234099d1e7%22%2C%22tagName%22%3A%22CDClogo%22%2C%22boundingBox%22%3A%7B%22left%22%3A0.7381513%2C%22top%22%3A0.12175512%2C%22width%22%3A0.22355062%2C%22height%22%3A0.18526697%7D%7D%2C%7B%22probability%22%3A0.44562727%2C%22tagId%22%3A%2268b532c0-6de5-4aba-8074-8f234099d1e7%22%2C%22tagName%22%3A%22CDClogo%22%2C%22boundingBox%22%3A%7B%22left%22%3A0.62779665%2C%22top%22%3A0.120928116%2C%22width%22%3A0.37220335%2C%22height%22%3A0.1991472%7D%7D%2C%7B%22probability%22%3A0.011576932%2C%22tagId%22%3A%2268b532c0-6de5-4aba-8074-8f234099d1e7%22%2C%22tagName%22%3A%22CDClogo%22%2C%22boundingBox%22%3A%7B%22left%22%3A0.69671506%2C%22top%22%3A0.044883475%2C%22width%22%3A0.30187488%2C%22height%22%3A0.33183825%7D%7D%5D%7D%3C%2FPRE%3E%0A%3C%2FDIV%3E%0A%3CP%3EDo%20it%20yourself%20link%20-%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fgithub.com%2Fbalakreshnan%2FSamples2021%2Fblob%2Fmain%2Fcovidvaccine%2Fcovidvaccineai.md%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3ESamples2021%2Fcovidvaccineai.md%20at%20main%20%C2%B7%20balakreshnan%2FSamples2021%20(github.com)%3C%2FA%3E%3C%2FP%3E%0A%3CP%3EMedium%20link%26nbsp%3B%20-%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fbalabala76.medium.com%2Fcovid-vaccine-card-processing-using-azure-form-recognizer-and-custom-vision-46fc04f7d58f%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3ECovid%20Vaccine%20Card%20processing%20using%20Azure%20Form%20Recognizer%20and%20Custom%20Vision%20%7C%20by%20Balamurugan%20Balakreshnan%20%7C%20Oct%2C%202021%20%7C%20Medium%3C%2FA%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-2892962%22%20slang%3D%22en-US%22%3E%3CP%3EProcess%20Covid%20vaccine%20proof%20with%20AI%20and%20validate%20using%20logic%20apps%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-2892962%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3ECognitive%20Services%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ECustom%20Vision%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EForm%20Recognizer%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Version history
Last update:
‎Oct 30 2021 06:14 AM