Image quality assessment and improvement for performing optical character recognition
Techniques are disclosed for performing optical character recognition (OCR) by assessing and improving quality of electronic documents to perform the OCR. For example a method for identifying information in an electronic document includes obtaining a reference image of the electronic document, distorting the reference image by adjusting different sets of one or more parameters associated with a quality of the reference image to generate a plurality of distorted images, analyzing each distorted image to detect the adjusted set of parameters and corresponding adjusted values, determining an accuracy of detection of the set of parameters and the adjusted values, and training a model based at least on the plurality of distorted images and the accuracy of the detection, wherein the trained model determines at least a first technique for adjusting a set of parameters in a second image to prepare the second image for optical character recognition.
1. A method for improving identification of information in an electronic document, comprising:
obtaining, via a camera of a mobile device, an image of a document;
providing the image of the document to a machine trained model;
determining a set of available image adjustment parameters;
determining, by the machine trained model based on the image of the document, a subset of image adjustment parameters, of the set of available image adjustment parameters, to be adjusted in the image of the document in preparation for an optical character recognition process;
determining a value for each respective image adjustment parameter in the subset of image adjustment parameters determined by the machine trained model;
adjusting the image of the document based on the value for each respective image adjustment parameter in the subset of image adjustment parameters;
performing the optical character recognition process on the adjusted image of the document; determining one or more labels in the document based on the optical character recognition process; determining one or more values in the document based on the optical character recognition process;
for each respective label of the one or more labels in the document, determining a corresponding value of the one or more values in the document based on a region adjacent to the respective label comprising the corresponding value, wherein the respective label indicates a type of data associated with the corresponding value;
determining one of the one or more determined values is not accurately identified by the optical character recognition process; and
providing the adjusted image of the document to the machine trained model in order to determine an updated value for one or more image adjustment parameters in the subset of image adjustment parameters.
2. The method of claim 1 , further comprising: providing the one or more labels in the document and the corresponding values of the one or more values in the document for each respective label to an application running on the mobile device separate from the optical character recognition process.
3. The method of claim 1 , further comprising:
capturing a plurality of images of the document; and
receiving a selection of the image of the document from the plurality of images of the document.
4. The method of claim 1 , further comprising:
transmitting the image of the document to a remote processing system, wherein the remote processing system is configured to:
perform the determining the subset of image adjustment parameters;
perform the determining the value for each respective image adjustment parameter in the subset of image adjustment parameters;
perform the adjusting the image of the document according to the value for each respective image adjustment parameter in the subset of image adjustment parameters;
and perform the optical character recognition process on the adjusted image of the document; and
receiving the one or more values in the document from the remote processing system.
5. The method of claim 1 , further comprising:
receiving a corrected value for one of the one or more determined values in the document; and
updating the machine trained model based on the corrected value.
6. The method of claim 1 , wherein the subset of image adjustment parameters comprises at least one of rotation, skew, shadow, luminosity, blur, or color density.
7. The method of claim 1 , further comprising populating fields of an interface of the mobile device with one of the one or more labels and one of the one or more values.
8. An electronic device, comprising
a non-transitory memory comprising computer-executable instructions;
a processor configured to execute the computer-executable instructions and cause the electronic device to perform a method for improving identification of information in an electronic document, the method comprising:
obtaining, via a camera of a mobile device, an image of a document;
providing the image of the document to a machine trained model;
determining a set of available image adjustment parameters;
determining, by the machine trained model based on the image of the document, a subset of image adjustment parameters, of the set of available image adjustment parameters, to be adjusted in the image of the document in preparation for an optical character recognition process;
determining a value for each respective image adjustment parameter in the subset of image adjustment parameters determined by the machine trained model;
adjusting the image of the document based on the value for each respective image adjustment parameter in the subset of image adjustment parameters;
performing the optical character recognition process on the adjusted image of the document; determining one or more labels in the document based on the optical character recognition process; determining one or more values in the document based on the optical character recognition process;
for each respective label of the one or more labels in the document, determining a corresponding value of the one or more values in the document based on a region adjacent to the respective label comprising the corresponding value, wherein the respective label indicates a type of data associated with the corresponding value;
determining one of the one or more determined values is not accurately identified by the optical character recognition process; and
providing the adjusted image of the document to the machine trained model in order to determine an updated value for one or more image adjustment parameters in the subset of image adjustment parameters.
9. The electronic device of claim 8 , wherein the method further comprises: providing the one or more labels in the document and the corresponding values of the one or more values in the document for each respective label to an application running on the mobile device separate from the optical character recognition process.
10. The electronic device of claim 8 , wherein the method further comprises:
capturing a plurality of images of the document; and
receiving a selection of the image of the document from the plurality of images of the document.
11. The electronic device of claim 8 , wherein the method further comprises:
transmitting the image of the document to a remote processing system, wherein the remote processing system is configured to:
perform the determining the subset of image adjustment parameters;
perform the determining the value for each respective image adjustment parameter in the subset of image adjustment parameters;
perform the adjusting the image of the document according to the value for each respective image adjustment parameter in the subset of image adjustment parameters;
and perform the optical character recognition process on the adjusted image of the document; and
receiving the one or more values in the document from the remote processing system.
12. The electronic device of claim 8 , wherein the method further comprises:
receiving a corrected value for one of the one or more determined values in the document; and
updating the machine trained model based on the corrected value.
13. The electronic device of claim 8 , wherein the subset of image adjustment parameters comprises at least one of rotation, skew, shadow, luminosity, blur, or color density.
14. The electronic device of claim 8 , wherein the method further comprises populating fields of an interface of the mobile device with one of the one or more labels and one of the one or more values.
15. A non-transitory computer-readable medium comprising instructions that, when executed by a processor of an electronic device, cause the electronic device to perform a method for improving identification of information in an electronic document, the method comprising:
obtaining, via a camera of a mobile device, an image of a document;
providing the image of the document to a machine trained model;
determining a set of available image adjustment parameters;
determining, by the machine trained model based on the image of the document, a subset of image adjustment parameters, of the set of available image adjustment parameters, to be adjusted in the image of the document in preparation for an (OCR) process;
determining a value for each respective image adjustment parameter in the subset of image adjustment parameters determined by the machine trained model;
adjusting the image of the document according to the value for each respective image adjustment parameter in the subset of image adjustment parameters;
performing the optical character recognition process on the adjusted image of the document; determining one or more labels in the document based on the optical character recognition process; determining one or more values in the document based on the optical character recognition process;
for each respective label of the one or more labels in the document, determining a corresponding value of the one or more values in the document based on a region adjacent to the respective label comprising the corresponding value, wherein the respective label indicates a type of data associated with the corresponding value;
determining one of the one or more determined values is not accurately identified by the optical character recognition process; and
providing the adjusted image of the document to the machine trained model in order to determine an updated value for one or more image adjustment parameters in the subset of image adjustment parameters.
16. The non-transitory computer-readable medium of claim 15 , wherein the method further comprises: providing the one or more labels in the document and the corresponding values of the one or more values in the document for each respective label to an application running on the mobile device separate from the optical character recognition process.
17. The non-transitory computer-readable medium of claim 15 , wherein the method further comprises:
capturing a plurality of images of the document; and
receiving a selection of the image of the document from the plurality of images of the document.
18. The non-transitory computer-readable medium of claim 15 , wherein the method further comprises:
transmitting the image of the document to a remote processing system, wherein the remote processing system is configured to:
perform the determining the subset of image adjustment parameters;
perform the determining the value for each respective image adjustment parameter in the subset of image adjustment parameters;
perform the adjusting the image of the document according to the value for each respective image adjustment parameter in the subset of image adjustment parameters;
and perform the optical character recognition process on the adjusted image of the document; and
receiving the one or more values in the document from the remote processing system.
19. The non-transitory computer-readable medium of claim 15 , wherein the method further comprises:
receiving a corrected value for one of the one or more determined values in the document; and
updating the machine trained model based on the corrected value.
20. The non-transitory computer-readable medium of claim 15 , wherein the method further comprises populating fields of an interface of the mobile device with one of the one or more labels and one of the one or more values.