IP Library Granted Patent US 10,108,883
Granted Patent B2
US 10,108,883 · App. 15/337,285 · Granted Oct 23, 2018

Image quality assessment and improvement for performing optical character recognition

Inventors: Richard J. Becker (Mountain View, CA); Rakesh Kandpal (Mountain View, CA); Priya Kothari (Mountain View, CA); Sheldon Porcina (Alberta, CA); Pavlo Malynin (Alberta, CA)
Assignee: INTUIT INC.
G06K9/6206G06K9/00469G06K9/18G06K9/6255
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,108,883
App. No.
15/337,285
Granted
Oct 23, 2018
Kind
B2
Abstract

Techniques are disclosed for performing optical character recognition (OCR) by assessing and improving quality of electronic documents to perform the OCR. For example a method for identifying information in an electronic document includes obtaining a reference image of the electronic document, distorting the reference image by adjusting different sets of one or more parameters associated with a quality of the reference image to generate a plurality of distorted images, analyzing each distorted image to detect the adjusted set of parameters and corresponding adjusted values, determining an accuracy of detection of the set of parameters and the adjusted values, and training a model based at least on the plurality of distorted images and the accuracy of the detection, wherein the trained model determines at least a first technique for adjusting a set of parameters in a second image to prepare the second image for optical character recognition.

Claims (56)

1. A computer-implemented method for identifying information in an electronic document, comprising:

obtaining a reference image of the electronic document;

distorting the reference image by adjusting parameter values for a plurality of sets of parameters associated with a quality of the reference image to generate a plurality of distorted images;

for each distorted image;

analyzing the distorted image to attempt to detect a first set of parameters from the plurality of sets of parameters and corresponding parameter values used to generate the distorted image;

determining an accuracy of detection of the first set of parameters and the corresponding parameter values used to generate the distorted image, the determining including:

comparing each detected parameter determined as a result of the analyzing the distorted image with the first set of parameters used for generating the distorted image, and

determining the accuracy of the detection based on the comparison; and

training a model based at least on the plurality of distorted images and respective accuracies of the detection to generate a trained model;

obtaining a second image of the electronic document;

determining, based on the trained model, a second set of parameters to be adjusted in the second image and a value corresponding to each parameter in the second set by which the parameter is to be adjusted;

determining, based on the trained model, at least one technique for adjusting each parameter in the second set of parameters in the second image to prepare the second image for optical character recognition (OCR);

preparing the second image for the OCR by adjusting each determined parameter in the second set of parameters by a corresponding determined value based on a corresponding determined technique for the determined parameter to generate a prepared second image; and

performing OCR on the prepared second image.

2. The method of claim 1 , wherein determining the accuracy of detection of the first set of parameters and the corresponding parameter values comprises:

comparing a detected parameter value corresponding to the detected parameter determined as a result of the analyzing the distorted image with a corresponding parameter value of a corresponding parameter from the first set of parameters used for the distorting the image; and

determining the accuracy of detection of the detected parameter value based on the comparison.

3. The method of claim 1 , further comprising associating with the reference image information regarding a set of characteristics of the reference image, wherein the set of characteristics comprises at least one of a type of the reference image, a label associated with at least one region of the reference image or a format of a value associated with each label.

4. The method of claim 1 , wherein the training comprises training the model based on the associated information regarding the set of characteristics.

5. The method of claim 1 , further comprising determining, based on the trained model, whether a quality of the second image can be improved to an acceptable level for the OCR.

6. The method of claim 5 , wherein determining whether the quality of the second image can be improved to an acceptable level comprises determining whether one or more of the second set of parameters can be adjusted by corresponding parameter values that are equal to or above threshold parameter values associated with the one or more of the second set of parameters.

7. The method of claim 1 , further comprising:

determining accuracy of performing the OCR; and

feeding back the prepared second image into the model to improve accuracy of subsequent OCRs of the electronic document.

8. The method of claim 1 , wherein distorting the reference image comprises modeling a distribution of a plurality of parameters using a random process.

9. The method of claim 1 , wherein parameters in the plurality of sets of parameters comprise at least one of rotation, skew, shadow, luminosity, blur, or color density.

10. The method of claim 1 , wherein each of the plurality of sets of parameters include a different combination of the parameters.

11. An apparatus for identifying information in an electronic document, comprising:

at least one processor configured to:

obtain a reference image of the electronic document;

distort the reference image by adjusting parameter values for a plurality of sets of parameters associated with a quality of the reference image to generate a plurality of distorted images;

for each distorted image;

analyze the distorted image to attempt to detect a first set of parameters from the plurality of sets or parameters and corresponding parameter values used to generate the distorted image;

determine an accuracy of detection of the first set of parameters and the corresponding parameter values used to generate the distorted image, wherein the at least one processor determines the accuracy of detection by;

comparing each detected parameter determined as a result of the analyzing the distorted image with the first set of parameters used for generating the distorted image; and

determining the accuracy of the detection based on the comparison; and

train a model based at least on the plurality of distorted images and respective accuracies of the detection to generate a trained model;

obtain a second image of the electronic document;

determine, based on the trained model, a second set of parameters to be adjusted in the second image and a value corresponding to each parameter in the second set by which the parameter is to be adjusted;

determine, based on the trained model, at least one technique for adjusting each parameter in the second set of parameters in the second image to prepare the second image for optical character recognition (OCR)

prepare the second image for the OCR by adjusting each determined parameter in the second set of parameters by a corresponding determined value based on a corresponding technique for the determined parameter to generate a prepared second image; and

perform OCR on the prepared second image; and

a memory coupled to the at least one processor.

12. The apparatus of claim 11 , wherein the at least one processor determines the accuracy of detection of the first set of parameters and the corresponding parameter values by:

comparing a detected parameter value corresponding to the detected parameter determined as a result of the analyzing the distorted image with a corresponding parameter value of a corresponding parameter from the first set of parameters used for the distorting the image; and

determining the accuracy of detection of the detected parameter value based on the comparison.

13. The apparatus of claim 11 , wherein the at least one processor is further configured to associate with the reference image information regarding a set of characteristics of the reference image, wherein the set of characteristics comprises at least one of a type of the reference image, a label associated with at least one region of the reference image or a format of a value associated with each label.

14. The apparatus of claim 11 , wherein the training comprises training the model based on the associated information regarding the set of characteristics.

15. The apparatus of claim 11 , wherein the at least one processor is further configured to determine, based on the trained model, whether a quality of the second image can be improved to an acceptable level for the OCR.

16. The apparatus of claim 15 , wherein the at least one processor is configured to determine whether the quality of the second image can be improved to an acceptable level by determining whether one or more of the second set of parameters can be adjusted by corresponding values that are equal to or above threshold parameter values associated with the one or more of the second set of parameters.

17. The apparatus of claim 11 , wherein the at least one processor is further configured to:

determine accuracy of performing the OCR; and

feed back the prepared second image into the model to improve accuracy of subsequent OCRs of the electronic document.

18. The apparatus of claim 11 , wherein the at least one processor distorts the reference image by modeling a distribution of a plurality of parameters using a random process.

19. The apparatus of claim 11 , wherein parameters in the plurality of sets of parameters comprise at least one of rotation, skew, shadow, luminosity, blur, or color density.

20. The apparatus of claim 11 , wherein each of the plurality of sets of parameters includes a different combination of the parameters.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 10, 2016
From: BECKER, RICHARD J.; KANDPAL, RAKESH; KOTHARI, PRIYA; PORCINA, SHELDON; MALYNIN, PAVLO
To: INTUIT INC.
Reel/Frame 040274/0133 →
Continuity (1)
Related Publication 20180121756A1 · May 3, 2018
Cited By (1)
US 12,425,713