Machined learning supporting document data extraction
Improved techniques to access content from documents in an automated fashion. The improved techniques permit content within documents to be retrieved and then used by computer systems operating various software programs (e.g., application programs), such as an extraction program. Documents, especially business transaction documents, often have various descriptors (or tables) and values that form key-value pairs. The improved techniques permit key-value pairs within documents to be recognized and extracted from documents. Consequently, RPA systems are able to accurately understand the content of tables within documents so that users and/or software robots can operate on the documents with increased reliability and flexibility.
1. A computer-implemented method for extracting data from an image of a document, the computer-implemented method comprising:
retrieving object data pertaining to an object that has been detected in the image of the document, the object data denoting at least a portion of the document having the object;
acquiring text pertaining to the portion of the document having the object, the text having been recognized from the image of the document;
determining a key type for the object based on the text and a machine learned model, the machine learned model pertaining to at least a character level neural network model having at least a Character-level Convolutional Network and a Bidirectional Gated Recurrent Units Network;
determining a value for the object based on the determined key type for the object; and
storing the determined key type and the determined value for the object,
wherein the determining of the key type predicts the key type using the character level neural network model, and provides a confidence indication for key type predicted using the character level neural network model,
wherein the key type is predicted using the character level neural network model is accepted and recorded if the confidence indication is greater than a threshold level, and
wherein the determining of the key type predicts the key type using the pattern matching model is predicted and recorded if the confidence indication associated with prediction using the character level neural network model is not greater that the threshold level.
2. A computer-implemented method as recited in claim 1 , wherein the text is recognized through Optical Character Resolution (OCR) of at least a portion of the document.
3. A computer-implemented method as recited in claim 1 , wherein the object is an object block, and the object data is provided within the object block.
4. A computer-implemented method as recited in claim 1 , wherein the object is a key-value block.
5. A computer-implemented method as recited in claim 1 , wherein the key type comprises a textual descriptor.
6. A computer-implemented method as recited in claim 1 , wherein the machine learned model is a Natural Language Processing (NLP) model.
7. A computer-implemented method as recited in claim 1 , wherein the character level neural network model comprises a Keras-based model.
8. A computer-implemented method as recited in claim 1 , wherein a user input is provided to the character level neural network model, and wherein the character level neural network model predicts the key type for the object based at least in part on the user input.
9. A computer-implemented method as recited in claim 1 , wherein the document comprises a business transaction document, and wherein the image of the document is provided in a graphical file format.
10. A data extraction system for extracting data from an image of a document, the data extraction system comprising:
a character level neural network model that receives an object block and recognized text within at least a portion of the document as recognized from the image of the document, the character level neural network model predicting a key type and a value for the object block; and
a pattern matching model that receives an object block and recognized text within at least a portion of the document as recognized from the image of the document, the pattern matching model predicting a key type and a value for the object block,
wherein key type and value predicted using the character level neural network model are accepted and recorded if such prediction is successful,
wherein key type and value predicted using the pattern matching model are accepted and recorded if prediction using the character level neural network model is unsuccessful,
wherein the character level neural network model provides a confidence indication for key type and value predicted using the character level neural network model,
wherein key type and value predicted using the character level neural network model are accepted and recorded if the confidence indication is greater than a threshold level, and
wherein key type and value predicted using the pattern matching model are accepted and recorded if the confidence indication associated with prediction using the character level neural network model is not greater that the threshold level.
11. A data extraction system as recited in claim 10 ,
wherein the object block is a key-value block,
wherein key type comprises a textual descriptor, and
wherein the image of the document is provided in a graphical file format.
12. A data extraction system as recited in claim 10 , wherein the image of the document is provided in a graphical file format.
13. A data extraction system as recited in claim 12 , wherein the object block is a key-value block.
14. A data extraction system as recited in claim 10 , wherein a user input is provided to the character level neural network model, and wherein the character level neural network model predicts a key type and a value for the object block based at least in part on the user input.
15. A data extraction system as recited in claim 10 , wherein a user input is provided to the pattern matching model, and wherein the pattern matching model predicts a key type and a value for the object block based at least in part on the user input.
16. A data extraction system as recited in claim 10 for extracting data from an image of a document, the data extraction system comprising:
a character level neural network model that receives an object block and recognized text within at least a portion of the document as recognized from the image of the document, the character level neural network model predicting a key type and a value for the object block; and
a pattern matching model that receives an object block and recognized text within at least a portion of the document as recognized from the image of the document, the pattern matching model predicting a key type and a value for the object block,
wherein key type and value predicted using the character level neural network model are accepted and recorded if such prediction is successful,
wherein key type and value predicted using the pattern matching model are accepted and recorded if prediction using the character level neural network model is unsuccessful, and
wherein the data extraction system receives or seeks user input to assist with extraction of key type and value if prediction using the character level neural network model is unsuccessful and if prediction using the pattern matching model is unsuccessful.
17. A non-transitory computer readable medium including at least computer program code for extracting data from an image of a document, the computer readable medium comprising:
computer program code for retrieving object data pertaining to an object that has been detected in the image of the document, the object data denoting at least a portion of the document having the object;
computer program code for acquiring text pertaining to the portion of the document having the object, the text having been recognized from the image of the document;
computer program code for determining a key type for the object based on at least the text;
computer program code for determining a value for the object based on the determined key type for the object; and
computer program code for storing the determined key type and the determined value for the object,
wherein the computer program code for determining a key type for the object is based on at least a machine learned model, the machine learned model pertaining to at least a character level neural network model,
wherein the determining of the key type predicts the key type using the character level neural network model, and provides a confidence indication for key type predicted using the character level neural network model,
wherein the key type is predicted using the character level neural network model is accepted and recorded if the confidence indication is greater than a threshold level, and
wherein the determining of the key type predicts the key type using the pattern matching model is predicted and recorded if the confidence indication associated with prediction using the character level neural network model is not greater that the threshold level.
18. A non-transitory computer readable medium as recited in claim 17 , wherein the text is recognized through Optical Character Resolution (OCR) of at least a portion of the document.
19. A non-transitory computer readable medium as recited in claim 17 , wherein the object is an object block, and the object data is provided within the object block.
20. A non-transitory computer readable medium as recited in claim 17 , wherein the object is a key-value block.
21. A non-transitory computer readable medium as recited in claim 17 , wherein the key type comprises a textual descriptor.
22. A non-transitory computer readable medium as recited in claim 17 , wherein the document comprises a business transaction document, and wherein the image of the document is provided in a graphical file format.
23. A non-transitory computer readable medium as recited in claim 17 , wherein the machine learned model is a Natural Language Processing (NLP) model, and wherein the character level neural network model having at least a Character-level Convolutional Network and a Bidirectional Gated Recurrent Units Network.
24. A non-transitory computer readable medium as recited in claim 22 , wherein the text is recognized through Optical Character Resolution (OCR) of at least a portion of the document.
25. A non-transitory computer readable medium as recited in claim 24 , wherein the object is an object block, and the object data is provided within the object block.
26. A non-transitory computer readable medium as recited in claim 25 , wherein the document comprises a business transaction document, and wherein the image of the document is provided in a graphical file format.