IP Library Granted Patent US 11,164,035
Granted Patent B2
US 11,164,035 · App. 16/178,987 · Granted Nov 2, 2021

Neural-network-based optical character recognition using specialized confidence functions

Inventor: Aleksey Zhuravlev (Yaroslavl, RU)
Assignee: ABBYY Production LLC
G06K9/4671G06K9/6262G06K2209/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,164,035
App. No.
16/178,987
Granted
Nov 2, 2021
Kind
B2
Abstract

Systems and methods for neural-network-based optical character recognition using specialized confidence functions. An example method comprises: receiving a grapheme image; computing, by a neural network, a feature vector representing the grapheme image in a space of image features; and computing a confidence vector associated with the grapheme image, wherein each element of the confidence vector reflects a distance, in the space of image features, between the feature vector and a center of a class of a set of classes, wherein the class is identified by an index of the element of the confidence vector.

Claims (52)

1. A method, comprising:

receiving, by a computer system, a grapheme image;

computing, by a neural network, a feature vector representing the grapheme image in a space of image features, wherein the neural network includes multiple alternating sets of convolutional layers and pooling layers producing a plurality of feature maps, wherein each feature map of the plurality of feature maps corresponds to a particular image feature of a plurality of image features; and

computing a confidence vector associated with the grapheme image, wherein each element of the confidence vector is produced by a piecewise-linear monotonically decreasing function of a distance, in the space of image features, between the feature vector and a center of a class of a set of classes, wherein the class is identified by an index of the element of the confidence vector, wherein the piecewise-linear monotonically decreasing function is based on a series of predefined confidence values, such that intervals between the predefined confidence values increase towards a lowest confidence value, and wherein the intervals are chosen to minimize a number of classification errors produced by the neural network.

2. The method of claim 1 , further comprising:

identifying an element having a maximum value among elements of the confidence vector; and

associating the grapheme image with a grapheme class corresponding to the identified element of the confidence vector.

3. The method of claim 1 , further comprising:

identifying an element having a maximum value among elements of the confidence vector; and

responsive to determining that the maximum value falls below a threshold, returning an error code indicating that the grapheme image is not recognizable.

4. The method of claim 1 , further comprising:

performing, in view of the confidence vector, segmentation of an original image to produce a second grapheme image; and

repeating, for the second grapheme image, operations of computing the feature vector and computing the confidence vector.

5. The method of claim 1 , wherein each class of the set of classes corresponds to a character of an alphabet.

6. The method of claim 1 , wherein the neural network comprises a fully-connected layer employed to produce a set of class weights, such that each class weight characterizes a degree of association of the grapheme image with a certain class of a set of classes, and wherein the method further comprises:

computing, using a normalized exponential transformation, a set of probabilities corresponding to the set of class weights, such that each probability characterizes a hypothesis of the grapheme image representing an instance of a certain class of the set of classes.

7. The method of claim 6 , wherein the confidence vector is determined for a subset of classes associated with highest probability values.

8. The method of claim 1 , further comprising:

training the neural network using a loss function represented by a linear combination of a cross entropy loss function, and a center loss function.

9. A system, comprising:

a memory;

a processor, coupled to the memory, the processor configured to:

receive a grapheme image;

compute, by a neural network, a feature vector representing the grapheme image in a space of image features, wherein the neural network includes multiple alternating sets of convolutional layers and pooling layers producing a plurality of feature maps, wherein each feature map of the plurality of feature maps corresponds to a particular image feature of a plurality of image features;

compute a confidence vector associated with the grapheme image, wherein each element of the confidence vector is produced by a piecewise-linear monotonically decreasing function of a distance, in the space of image features, between the feature vector and a center of a class of a set of classes, wherein the class is identified by an index of the element of the confidence vector, wherein the piecewise-linear monotonically decreasing function is based on a series of predefined confidence values, such that intervals between the predefined confidence values increase towards a lowest confidence value, and wherein the intervals are chosen to minimize a number of classification errors produced by the neural network;

identify an element having a maximum value among elements of the confidence vector; and

associate the grapheme image with a grapheme class corresponding to the identified element of the confidence vector.

10. The system of claim 9 , wherein each class of the set of classes corresponds to a character of an alphabet.

11. The system of claim 9 , wherein the neural network comprises a fully-connected layer employed to produce a set of class weights, such that each class weight characterizes a degree of association of the grapheme image with a certain class of a set of classes, and wherein the processor is further configured to:

compute, using a normalized exponential transformation, a set of probabilities corresponding to the set of class weights, such that each probability characterizes a hypothesis of the grapheme image representing an instance of a certain class of the set of classes.

12. The system of claim 11 , wherein the confidence vector is determined for a subset of classes associated with highest probability values.

13. The system of claim 9 , wherein the processor is further configured to:

train the neural network using a loss function represented by a linear combination of a cross entropy loss function and a center loss function.

14. The system of claim 9 , wherein the processor is further configured to:

perform, in view of the confidence vector, segmentation of an original image to produce a second grapheme image; and

repeat, for the second grapheme image, operations of computing the feature vector and computing the confidence vector.

15. A computer-readable non-transitory storage medium comprising executable instructions that, when executed by a computer system, cause the computer system to:

receive a grapheme image;

compute, by a neural network, a feature vector representing the grapheme image in a space of image features, wherein the neural network includes multiple alternating sets of convolutional layers and pooling layers producing a plurality of feature maps, wherein each feature map of the plurality of feature maps corresponds to a particular image feature of a plurality of image features; and

compute a confidence vector associated with the grapheme image, wherein each element of the confidence vector is produced by a piecewise-linear monotonically decreasing function of a distance, in the space of image features, between the feature vector and a center of a class of a set of classes, wherein the class is identified by an index of the element of the confidence vector, wherein the piecewise-linear monotonically decreasing function is based on a series of predefined confidence values, such that intervals between the predefined confidence values increase towards a lowest confidence value, and wherein the intervals are chosen to minimize a number of classification errors produced by the neural network.

16. The computer-readable non-transitory storage medium of claim 15 , further comprising executable instructions causing the computer system to:

identify an element having a maximum value among elements of the confidence vector; and

associate the grapheme image with a grapheme class corresponding to the identified element of the confidence vector.

17. The computer-readable non-transitory storage medium of claim 15 , further comprising executable instructions causing the computer system to:

identify an element having a maximum value among elements of the confidence vector; and

responsive to determining that the maximum value falls below a threshold, return an error code indicating that the grapheme image is not recognizable.

18. The computer-readable non-transitory storage medium of claim 16 , further comprising executable instructions causing the computer system to:

perform, in view of the confidence vector, segmentation of an original image to produce a second grapheme image; and

repeat, for the second grapheme image, operations of computing the feature vector and computing the confidence vector.

19. The computer-readable non-transitory storage medium of claim 15 , wherein each class of the set of classes corresponds to a character of an alphabet.

20. The computer-readable non-transitory storage medium of claim 15 , further comprising executable instructions causing the computer system to:

train the neural network using a loss function represented by a linear combination of a cross entropy loss function and a center loss function.

Assignments (3)
SECURITY INTEREST Recorded Aug 14, 2023
From: ABBYY INC.; ABBYY USA SOFTWARE HOUSE INC.; ABBYY DEVELOPMENT INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 064730/0964 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 25, 2022
From: ABBYY PRODUCTION LLC
To: ABBYY DEVELOPMENT INC.
Reel/Frame 059249/0873 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 2, 2018
From: ZHURAVLEV, ALEKSEY
To: ABBYY PRODUCTION LLC
Reel/Frame 047394/0273 →