Text-based machine learning extraction of table data from a read-only document
Embodiments of the disclosed technologies provide solutions for automatically reading digital electronic documents that contain tables and correctly extracting table data, rows and columns from the documents with high accuracy and high throughput. Embodiments are capable of converting a table portion of a read-only document to a searchable, editable data record using text rectangle (TR)-level numerical data that indicates probabilities of TRs belonging to canonicals and at least one convolutional neural network (CNN) that processes the TR-level numerical data to produce table-level numerical data.
1. A computer-implemented method, comprising:
extracting text rectangle data from a digital electronic document;
converting the text rectangle data to a feature map that indicates text rectangle-level numerical data and spatial locations of text rectangles in the document;
by at least one convolutional neural network, processing the text rectangle-level numerical data to produce table-level numerical data including a spatial location of a table portion of the document, probabilities of text rectangles belonging to row canonicals, and probabilities of text rectangles belonging to column canonicals;
formatting and storing the table-level numerical data in a searchable, editable data record;
wherein the method is performed by one or more computing devices.
2. The method of claim 1 , wherein the text rectangle-level numerical data indicates probabilities of text rectangles belonging to text canonicals.
3. The method of claim 1 , wherein processing the text rectangle-level numerical data produces table-level numerical data including anchor data comprises determining table location in the document and filtering all non-table text rectangle data out of the text rectangle-level numerical data.
4. The method of claim 1 , the text rectangle data indicating text contained in the document and coordinates of text rectangles that contain the text.
5. The method of claim 1 , the text rectangle-level numerical data indicating probabilities of text contained in text rectangles belonging to text canonicals.
6. The method of claim 1 , further comprising partitioning the document into a grid and projecting the text rectangle-level numerical data onto the grid, where a particular value of the text rectangle-level numerical data maps to a particular cell of the grid and corresponds to a level of pixel brightness associated with the particular cell.
7. The method of claim 6 , comprising applying a fading process to the text rectangle-level numerical data for a text to produce a faded text rectangle-level numerical data and projecting the faded text rectangle-level numerical data onto the grid, where a particular value of the faded text rectangle-level numerical data maps to a particular cell of the grid and corresponds to a level of pixel brightness associated with the particular cell.
8. The method of claim 1 , the at least one convolutional neural network comprising a row convolutional neural network that processes the text rectangle-level numerical data to produce row-level numerical data including anchor data that indicates a spatial location of the table portion within the document and probabilities of text rectangles belonging to row canonicals.
9. The method of claim 1 , the at least one convolutional neural network comprising a column convolutional neural network that processes filtered text rectangle-level numerical data to produce probabilities of text rectangles belonging to column canonicals.
10. The method of claim 9 , the filtered text rectangle-level numerical data produced by filtering the text rectangle-level numerical data using anchor data to exclude portions of the text rectangle-level numerical data that correspond to portions of the document that are outside the table portion.
11. The method of claim 1 , wherein the table-level numerical data includes row-level numerical data and column-level numerical data, that are converted to a structured data format to produce formatted table data.
12. A computer system comprising:
at least one hardware processor;
one or more computer-readable non-transitory storage media coupled to the at least one hardware processor and storing one or more sequences of stored program instructions which, when executed by the at least one hardware processor cause the at least one hardware processor to execute:
receiving, from a network, a message containing a digital document;
extracting text rectangle data from the document;
converting the text rectangle data to a feature map that indicates text rectangle-level numerical data and spatial locations of text rectangles in the document;
by at least one convolutional neural network, processing the text rectangle-level numerical data to produce table-level numerical data including a spatial location of a table portion of the document, probabilities of text rectangles belonging to row canonicals, and probabilities of text rectangles belonging to column canonicals;
formatting, and storing the table-level numerical data in a searchable, editable data record.
13. The system of claim 12 , wherein the text rectangle-level numerical data indicates probabilities of text rectangles belonging to text canonicals.
14. The system of claim 12 , further comprising sequences of stored program instructions which, when executed by the at least one hardware processor cause the at least one hardware processor to process the text rectangle-level numerical data to produce table-level numerical data including anchor data by determining table location in the document and filtering all non-table text rectangle data out of the text rectangle-level numerical data.
15. The system of claim 12 , the text rectangle data indicating text contained in the document and coordinates of text rectangles that contain the text.
16. The system of claim 12 , the text rectangle-level numerical data indicating probabilities of text contained in text rectangles belonging to text canonicals.
17. The system of claim 12 , further comprising sequences of stored program instructions which, when executed by the at least one hardware processor cause the at least one hardware processor to execute: partitioning the document into a grid and projecting the text rectangle-level numerical data onto the grid, where a particular value of the text rectangle-level numerical data maps to a particular cell of the grid and corresponds to a level of pixel brightness associated with the particular cell.
18. The system of claim 17 , further comprising sequences of stored program instructions which, when executed by the at least one hardware processor cause the at least one hardware processor to execute: applying a fading process to the text rectangle-level numerical data for the text to produce a faded text rectangle-level numerical data and projecting the faded text rectangle-level numerical data onto the grid, where a particular value of the faded text rectangle-level numerical data maps to a particular cell of the grid and corresponds to a level of pixel brightness associated with the particular cell.
19. The system of claim 12 , the at least one convolutional neural network comprising a row convolutional neural network that processes the text rectangle-level numerical data to produce row-level numerical data including anchor data that indicates a spatial location of the table portion within the document and probabilities of text rectangles belonging to row canonicals.
20. The system of claim 12 , the at least one convolutional neural network comprising a column convolutional neural network that processes filtered text rectangle-level numerical data to produce probabilities of text rectangles belonging to column canonicals.