System and method of character recognition using fully convolutional neural networks with attention
Embodiments of the present disclosure include a method that obtains a digital image. The method includes extracting a word block from the digital image. The method includes processing the word block by evaluating a value of the word block against a dictionary. The method includes outputting a prediction equal to a common word in the dictionary when a confidence factor is greater than a predetermined threshold. The method includes processing the word block and assigning a descriptor to the word block corresponding to a property of the word block. The method includes processing the word block using the descriptor to prioritize evaluation of the word block. The method includes concatenating a first output and a second output. The method includes predicting a value of the word block.
1. A computer-implemented method, comprising:
obtaining a digital image;
extracting a word block from the digital image, the word block input including at least one symbol;
processing the word block with a first neural network, the first neural network evaluating a value of the word block against a dictionary, the dictionary containing a collection of most common words;
outputting a prediction equal to a common word when a confidence factor is greater than a predetermined threshold;
processing the word block with a second neural network, the second neural network assigning a descriptor to the word block corresponding to a property of the word block;
processing the word block with a third neural network, the third neural network using the descriptor to prioritize evaluation of the word block;
concatenating an output of the second neural network and an output of the third neural network; and
predicting a value of the word block.
2. The computer-implemented method of claim 1 , further comprising:
resizing the word block to a canonical presentation of fixed height; and
resizing the word block to canonical width.
3. The computer-implemented method of claim 1 , wherein the descriptor includes handwriting style, slant, skew, rotation, blur, marker type, paper type, image fidelity, background threshold, a language, a font, a case style, a document field, or a combination thereof.
4. The computer-implemented method of claim 1 , wherein the first neural network, the second neural network, and the third neural network are a common neural network.
5. The computer-implemented method of claim 1 , further comprising assigning a steering factor to the word block, the steering factor corresponding to a statistical property of the word block.
6. The computer-implemented method of claim 5 , wherein the steering factor is a weight factor comprising at least one of attention weights, attention vectors, and Gaussian distributions.
7. The computer-implemented method of claim 5 , wherein the third neural network utilizes the steering factor to prioritize evaluation of the word block.
8. The computer-implemented method of claim 1 , further comprising:
generating a family of word blocks;
generating a vector of attention weights via the second neural network, the vector of attention weights corresponding to the family of word blocks; and
forming a single resized word block from the family of word blocks, the single resized word block comprising a linear combination of resized word blocks associated with the vector of attention weights.
9. The computer-implemented method of claim 1 , further comprising:
generating a family of word blocks;
generating a parametric distribution of attention weights via the second neural network, the parametric distribution of attention weights corresponding to the family of word blocks; and
forming a single resized word block from the family of word blocks, the single resized word block comprising a linear combination of resized word blocks associated with the parametric distribution of attention weights.