Robust key value extraction
Disclosed herein are system, method, and computer program product embodiments for robust key value extraction. In an embodiment, one or more hierarchical concepts units (HCUs) may be configured to extract key value and hierarchical information from text inputs. The HCUs may use a convolutional neural network, a recurrent neural network, and feature selectors to analyze the text inputs using machine learning techniques to extract the key value and hierarchical information. Multiple HCUs may be used together and configured to identify different categories of hierarchical information. While multiple HCUs may be used, each may use a skip connection to transmit extracted information to a feature concatenation layer. This allows an HCU to directly send a concept that has been identified as important to the feature concatenation layer and bypass other HCUs.
1. A computer-implemented method for extracting key value information in an optical character recognition (OCR) system, comprising:
receiving a text input, wherein symbols of the text input have been identified using an OCR technique;
applying a convolutional neural network (CNN) to the text input to determine a plurality of context features of the text input indicating patterns of symbols of the text input;
selecting a subset of the plurality of context features based on a machine learning algorithm of the CNN;
applying a recurrent neural network (RNN) to the subset to determine a plurality of RNN features, wherein the RNN designates the plurality of RNN features as relevant to extracting a key value from the symbols of the text input; and
transmitting one or more RNN features from the plurality of RNN features to a feature concatenation layer using a skip connection to extract the key value.
2. The computer-implemented method of claim 1 , wherein applying the CNN further comprises:
applying a one-dimensional window during a convolution operation.
3. The computer-implemented method of claim 1 , further comprising:
selecting one or more context features of the plurality of context features; and
transmitting the one or more context features to the feature concatenation layer using a second skip connection.
4. The computer-implemented method of claim 1 , further comprising:
selecting a subset of the plurality of RNN features; and
transmitting the subset of the plurality of RNN features to a second CNN.
5. The computer-implemented method of claim 1 , wherein the transmitting further comprises:
transmitting the one or more RNN features to the feature concatenation layer by bypassing a second CNN and a second RNN.
6. The computer-implemented method of claim 1 , further comprising:
aggregating the one or more RNN features at the feature concatenation layer with a second set of RNN features received from a second RNN or a second set of context features received from a second CNN to generate a set of concatenated concepts.
7. The computer-implemented method of claim 6 , further comprising:
applying a convolutional layer to the set of concatenated concepts to identify one or more key values.
8. The computer-implemented method of claim 7 , further comprising:
applying a fully connected layer to the one or more key values to determine a probability corresponding to the one or more key values.
9. A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:
receiving a text input, wherein symbols of the text input have been identified using an OCR technique;
applying a convolutional neural network (CNN) to the text input to determine a plurality of context features of the text input indicating patterns of symbols of the text input;
selecting a subset of the plurality of context features based on a machine learning algorithm of the CNN;
applying a recurrent neural network (RNN) to the subset to determine a plurality of RNN features, wherein the RNN designates the plurality of RNN features as relevant to extracting a key value from the symbols of the text input; and
transmitting one or more RNN features from the plurality of RNN features to a feature concatenation layer using a skip connection to extract the key value.
10. The non-transitory computer-readable device of claim 9 , wherein applying the CNN further comprises:
applying a one-dimensional window during a convolution operation.
11. The non-transitory computer-readable device of claim 9 , the operations further comprising:
selecting one or more context features of the plurality of context features; and
transmitting the one or more context features to the feature concatenation layer using a second skip connection.
12. The non-transitory computer-readable device of claim 9 , the operations further comprising:
selecting a subset of the plurality of RNN features; and
transmitting the subset of the plurality of RNN features to a second CNN.
13. The non-transitory computer-readable device of claim 9 , wherein the transmitting further comprises:
transmitting the one or more RNN features to the feature concatenation layer by bypassing a second CNN and a second RNN.
14. The non-transitory computer-readable device of claim 9 , the operations further comprising:
aggregating the one or more RNN features at the feature concatenation layer with a second set of RNN features received from a second RNN or a second set of context features received from a second CNN to generate a set of concatenated concepts.
15. The non-transitory computer-readable device of claim 14 , the operations further comprising:
applying a convolutional layer to the set of concatenated concepts to identify one or more key values.
16. The non-transitory computer-readable device of claim 15 , the operations further comprising:
applying a fully connected layer to the one or more key values to determine a probability corresponding to the one or more key values.
17. A system, comprising:
a memory; and
at least one processor coupled to the memory and configured to:
receive a text input, wherein symbols of the text input have been identified using an OCR technique;
apply a convolutional neural network (CNN) to the text input to determine a plurality of context features of the text input indicating patterns of symbols of the text input;
select a subset of the plurality of context features based on a machine learning algorithm of the CNN;
apply a recurrent neural network (RNN) to the subset to determine a plurality of RNN features, wherein the RNN designates the plurality of RNN features as relevant to extracting a key value from the symbols of the text input; and
transmit one or more RNN features from the plurality of RNN features to a feature concatenation layer using a skip connection to extract the key value.
18. The system of claim 17 , wherein the at least one processor is further configured to:
select one or more context features of the plurality of context features; and
transmit the one or more context features to the feature concatenation layer using a second skip connection.
19. The system of claim 17 , wherein to transmit the one or more RNN features, the at least one processor is further configured to:
transmit the one or more RNN features to the feature concatenation layer by bypassing a second CNN and a second RNN.
20. The system of claim 17 , wherein the at least one processor is further configured to:
aggregate the one or more RNN features at the feature concatenation layer with a second set of RNN features received from a second RNN or a second set of context features received from a second CNN to generate a set of concatenated concepts;
apply a convolutional layer to the set of concatenated concepts to identify one or more key values; and
apply a fully connected layer to the one or more key values to determine a probability corresponding to the one or more key values.