Two-dimensional document processing
Disclosed herein are system, method, and computer program product embodiments for processing a document. In an embodiment, a document processing system may receive a document. The document processing system may perform optical character recognition to obtain character information and positioning information for the characters. The document processing system may generate a down-sampled two-dimensional character grid for the document. The document processing system may apply a convolutional neural network to the character grid to obtain semantic meaning for the document. The convolutional neural network may produce a segmentation mask and bounding boxes to correspond to the document.
1. A computer implemented method, comprising:
performing optical character recognition on a document;
generating a character grid using character information obtained from the optical character recognition, wherein the character grid is a two-dimensional down-sampled version of the document;
applying a machine learning algorithm to the character grid;
in response to the applying, generating a segmentation mask depicting semantic data of the document; and
wherein generating the character grid further comprises:
identifying a character of the document;
determining a pixel area for the character;
assigning an index value to represent the pixel area in the character grid; and
down-sampling the document by a factor equal to the pixel area covering a character of the document.
2. The computer implemented method of claim 1 , further comprising:
generating one or more bounding boxes indicating a presence of different instances of a semantic class in the document.
3. The computer implemented method of claim 1 , wherein generating the segmentation mask further comprises:
determining a class corresponding to a word of the document.
4. The computer implemented method of claim 1 , further comprising:
classifying the document using positioning of the semantic data.
5. The computer implemented method of claim 1 , wherein the applying further comprises:
applying a convolutional neural network to the character grid.
6. The computer implemented method of claim 1 , wherein the segmentation mask identifies a background class of the document.
7. The computer implemented method of claim 1 , further comprising:
generating one or more bounding boxes identifying different instances of a line item class.
8. A system, comprising:
a memory; and
at least one processor coupled to the memory and configured to:
perform optical character recognition on a document;
generate a character grid using character information obtained from the optical character recognition, wherein the character grid is a two-dimensional down-sampled version of the document;
apply a machine leaning algorithm to the character grid;
in response to the applying, generate a segmentation mask depicting semantic data of the document; and
wherein to generate the character grid, the at least one processor is further configured to:
identify a character of the document;
determine a pixel area for the character;
assign an index value to represent the pixel area in the character grid; and
down-sample the document by a factor equal to the pixel area covering a character of the document.
9. The system of claim 8 , wherein the at least one processor is further configured to:
generate one or more bounding boxes indicating a presence of different instances of a semantic class in the document.
10. The system of claim 8 , wherein to generate the segmentation mask, the at least one processor is further configured to:
determine a class corresponding to a word of the document.
11. The system of claim 8 , wherein the at least one processor is further configured to:
classify the document using positioning of the semantic data.
12. The system of claim 8 , wherein to apply, the at least one processor is further configured to:
apply a convolutional neural network to the character grid.
13. The system of claim 8 , wherein the segmentation mask identifies a background class of the document.
14. The system of claim 8 , wherein the at least one processor is further configured to:
generate one or more bounding boxes identifying different instances of a line item class.
15. A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:
performing optical character recognition on a document;
generating a character grid using character information obtained from the optical character recognition, wherein the character grid is a two-dimensional down-sampled version of the document;
applying a machine learning algorithm to the character grid;
in response to the applying, generating a segmentation mask depicting semantic data of the document; and
wherein generating the character grid comprises:
identifying a character of the document;
determining a pixel area for the character;
assigning an index value to represent the pixel area in the character grid; and
down-sampling the document by a factor equal to the pixel area covering a character of the document.
16. The non-transitory computer-readable device of claim 15 , the operations further comprising:
generating one or more bounding boxes indicating a presence of different instances of a semantic class in the document.
17. The non-transitory computer-readable device of claim 15 , wherein generating the segmentation mask comprises:
determining a class corresponding to a word of the document.
18. The non-transitory computer-readable device of claim 15 , the operations further comprising:
classifying the document using positioning of the semantic data.
19. The non-transitory computer-readable device of claim 15 , wherein the segmentation mask identifies a background class of the document.
20. The non-transitory computer-readable device of claim 15 , the operations further comprising:
generating one or more bounding boxes identifying different instances of a line item class.