IP Library Granted Patent US 10,068,155
Granted Patent B2
US 10,068,155 · App. 15/275,990 · Granted Sep 4, 2018

Verification of optical character recognition results

Inventor: Aleksey Ivanovich Kalyuzhny (Moscow oblast, RU)
Assignee: ABBYY Development LLC
G06K9/6263G06K9/00469G06K9/18G06K9/20G06K9/6215G06K9/6218
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,068,155
App. No.
15/275,990
Granted
Sep 4, 2018
Kind
B2
Abstract

A method of verifying optical character recognition (OCR) results may involve: performing OCR on one or more initial images of a document and displaying initial OCR results of the document to a user; receiving a feedback from the user regarding an error location in the initial OCR results, the error location being a location of a misspelled character sequence; receiving an additional image of the document, which corresponds to the error location, and performing OCR of the additional image to produce additional OCR results; identifying a cluster of character sequences, which correspond to the error location, using the initial OCR results and the additional OCR results; identifying an order of character sequences in the cluster of character sequences based on their respective probability values; and displaying to the user modified optical character recognition results, which contain in the error location a corrected character sequence.

Claims (46)

1. A method comprising:

performing optical character recognition on one or more initial images of a document to produce initial optical character recognition results and displaying the initial optical character recognition results of the document to a user;

receiving a feedback from the user regarding an error location in the initial optical character recognition results, wherein the error location is a location of a misspelled character sequence in the initial optical character recognition results;

receiving an additional image of the document, wherein the additional image contains a portion of the document, which corresponds to the error location;

performing optical character recognition of the additional image to produce additional optical character recognition results;

identifying a cluster of character sequences corresponding to the error location by matching the initial optical character recognition results and the additional optical character recognition results;

performing, for each of the cluster of characters, a probability evaluation to determine a plurality of probability values for the cluster of characters;

identifying, based on the probability values, a character sequence of the cluster of character sequences as a corrected character sequence; and

displaying to the user modified optical character recognition results, which contain in the error location the corrected character sequence, wherein the corrected character sequence is different from the misspelled character sequence.

2. The method of claim 1 , wherein the additional image differs from the one or more initial images in at least one of: an image noise, an image scale, a shooting angle and an image brightness.

3. The method of claim 1 , wherein said identifying the cluster of character sequences, which correspond to the error location, comprises: identifying in the initial optical character recognition results and the additional optical character recognition results a plurality of common features to identify reference points.

4. The method of claim 3 , further comprising identifying using coordinates of the reference points parameters of a coordinate transformation converting coordinates of the additional optical character recognition results into coordinates of the initial optical character recognition results.

5. The method of claim 1 , wherein said identifying, based on the probability values, the character sequence of the cluster of character sequences as the corrected character sequence comprises determining a pre-determined metric for each character sequence of the cluster and resorting the character sequences of the cluster according to the probability values.

6. The method of claim 5 , wherein the pre-determined metric is a sum of edit distances between a character sequence of the cluster and each of the other character sequences of the cluster and wherein the character sequences are resorted so that a character sequence with a lowest value of the metric goes is the top of the cluster, while a character sequence with a highest value of the metric is on the bottom of the cluster.

7. The method of claim 1 , wherein the one or more initial images and the additional image are obtained in a continuous sequence of images.

8. The method of claim 1 , wherein displaying to user the modified optical character recognition results comprises highlighting in the displayed modified optical character recognition results the corrected character sequence.

9. The method of claim 1 , further comprising, after said receiving the feedback from the user, highlighting the error location in the displayed initial optical character recognition results.

10. The method of claim 1 , which performed by a mobile device.

11. The method of claim 10 , wherein the mobile device is one of a mobile phone, a tablet, a laptop, a smartphone or a PDA.

12. The method of claim 10 , wherein said receiving the feedback from the user comprises receiving a feedback from a location of a touch screen display of the mobile device, which corresponds to the error location.

13. The method of claim 10 , wherein the mobile device comprises a camera and the one or more initial images and the additional image are acquired by said camera.

14. A system comprising:

a memory;

a display; and

a processing device, which is coupled to the memory and the display, the processing device is configured to:

perform optical character recognition on one or more initial images of a document to produce initial optical character recognition results and display the initial optical character recognition results to a user;

receive a feedback from the user regarding an error location in the initial optical character recognition results, wherein the error location is a location of a misspelled character sequence in the initial optical character recognition results;

receive an additional image of the document, wherein the additional image contains a portion of the document, which corresponds the error location;

perform optical character recognition on the additional image to produce additional optical character recognition results;

identify a cluster of character sequences corresponding to the error location by matching the initial optical character recognition results and the additional optical character recognition results;

perform, for each of the cluster of characters, a probability evaluation to determine a plurality of probability values for the cluster of characters;

identify, based on the probability values, a character sequence of the cluster of character sequences as a corrected character sequence; and

display to the user modified optical character recognition results, which contain in the error location the corrected character sequence, wherein the corrected character sequence is different from the misspelled character sequence.

15. The system of claim 14 , which is a mobile device.

16. The system of claim 15 , wherein the mobile device is a one of a mobile phone, a tablet, a laptop, a smartphone or a PDA.

17. The system of claim 14 , wherein the display is a touch-screen display and wherein the processing device receives the feedback from the user regarding the error location by receiving a feedback from a location of the touch-screen display touched by the user.

18. The system of claim 14 , further comprising a camera configured to acquire the one or more images and the next image and to transfer each of the one or more images and the next image to the processing device.

19. A computer-readable non-transitory storage medium comprising executable instructions that, when executed by a processing device, cause the processing device to:

perform optical character recognition on one or more initial images of a document to produce initial optical character recognition results and display the initial optical character recognition results of the document to a user;

receive a feedback from the user regarding an error location in the initial optical character recognition results, wherein the error location is a location of a misspelled character sequence in the initial optical character recognition results;

receive an additional image of the document, wherein the additional image contains a portion of the document, which corresponds to the error location;

perform optical character recognition on the additional image to produce additional optical character recognition results;

identify a cluster of character sequences corresponding to the error location by matching the initial optical character recognition results and the additional optical character recognition results;

perform, for each of the cluster of characters, a probability evaluation to determine a plurality of probability values for the cluster of characters;

identify, based on the probability values, a character sequence of the cluster of character sequences as a corrected character sequence; and

display to the user modified optical character recognition results, which contain in the error location the corrected character sequence, wherein the corrected character sequence is different from the misspelled character sequence.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 25, 2022
From: ABBYY PRODUCTION LLC
To: ABBYY DEVELOPMENT INC.
Reel/Frame 059249/0873 →
MERGER Recorded Dec 31, 2018
From: ABBYY DEVELOPMENT LLC
To: ABBYY PRODUCTION LLC
Reel/Frame 047997/0652 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 29, 2016
From: KALYUZHNY, ALEKSEY IVANOVICH
To: ABBYY DEVELOPMENT LLC
Reel/Frame 039897/0544 →
Priority Claims (1)
RU 2016137136 · Sep 16, 2016 · national
Continuity (1)
Related Publication 20180082154A1 · Mar 22, 2018
Cited By (1)
US 12,225,168