Text-based machine learning extraction of table data from a read-only document
Embodiments of the disclosed technologies provide solutions for automatically reading digital electronic documents that contain tables and correctly extracting table data, rows and columns from the documents with high accuracy and high throughput. Embodiments are capable of converting a table portion of a read-only document to a searchable, editable data record using text rectangle (TR)-level numerical data that indicates probabilities of TRs belonging to canonicals and at least one convolutional neural network (CNN) that processes the TR-level numerical data to produce table-level numerical data.
1. A computer-implemented method, comprising:
extracting text rectangle (TR)-level numerical data and grid data from a digital electronic document using one or more neural networks;
generating a feature map by projecting the TR-level numerical data onto a grid defined by the grid data and de-biasing one or more long text rectangle data items of the TR-level numerical data;
generating row-level numerical data by processing the feature map using a first-running convolutional filter;
generating a filtered feature map by filtering the feature map using a grid filter;
generating column-level numerical data by processing the filtered feature map using a second-running convolutional filter; and
converting the row-level numerical data and the column-level numerical data to a structured data format.
2. The computer-implemented method of claim 1 , wherein the TR-level numerical data indicates one or more probabilities of contents of text rectangles belonging to one or more text canonicals.
3. The computer-implemented method of claim 2 , wherein the one or more text canonicals comprises a label or a value.
4. The computer-implemented method of claim 1 , wherein the row-level numerical data indicates top anchors and bottom anchors of a table contained in the digital electronic document.
5. The computer-implemented method of claim 4 , wherein the row-level numerical data indicates probabilities of each row of the table belonging to a set of row canonicals.
6. The computer-implemented method of claim 1 , wherein the grid filter uses anchor data to determine a table location in the digital electronic document.
7. The computer-implemented method of claim 6 , wherein the TR-level numerical data is associated with TR data of the digital electronic document, and wherein the grid filter filters all non-table TR data out of the TR-level numerical data.
8. The computer-implemented method of claim 1 , wherein the column-level numerical data indicates probabilities of each column of a table belonging to a set of column canonicals.
9. The computer-implemented method of claim 1 , wherein the structured data format is stored in a searchable and editable data record.
10. The computer-implemented method of claim 2 , wherein the de-biasing comprises reducing an effect of one or more feature weights associated with the one or more long text rectangle data items on the TR-level numerical data.
11. One or more non-transitory computer-readable storage media, storing instructions which, when executed, cause one or more processors to execute:
extracting text rectangle (TR)-level numerical data and grid data from a digital electronic document using one or more neural networks;
generating a feature map by projecting the TR-level numerical data onto a grid defined by the grid data and de-biasing one or more long text rectangle data items of the TR-level numerical data;
generating row-level numerical data by processing the feature map using a first-running convolutional filter;
generating a filtered feature map by filtering the feature map using a grid filter;
generating column-level numerical data by processing the filtered feature map using a second-running convolutional filter; and
converting the row-level numerical data and the column-level numerical data to a structured data format.
12. The one or more non-transitory computer-readable storage media of claim 11 , wherein the TR-level numerical data indicates one or more probabilities of contents of text rectangles belonging to one or more text canonicals.
13. The one or more non-transitory computer-readable storage media of claim 12 , wherein the one or more text canonicals comprises a label or a value.
14. The one or more non-transitory computer-readable storage media of claim 11 , wherein the row-level numerical data indicates top anchors and bottom anchors of a table contained in the digital electronic document.
15. The one or more non-transitory computer-readable storage media of claim 14 , wherein the row-level numerical data indicates probabilities of each row of the table belonging to a set of row canonicals.
16. The one or more non-transitory computer-readable storage media of claim 11 , wherein the grid filter uses anchor data to determine a table location in the digital electronic document.
17. The one or more non-transitory computer-readable storage media of claim 16 , wherein the TR-level numerical data is associated with TR-data of the digital electronic document, and wherein the grid filter filters all non-table TR data out of the TR-level numerical data.
18. The one or more non-transitory computer-readable storage media of claim 11 , wherein the column-level numerical data indicates probabilities of each column of a table belonging to a set of column canonicals.
19. The one or more non-transitory computer-readable storage media of claim 11 , wherein the structured data format is stored in a searchable and editable data record.
20. The one or more non-transitory computer-readable storage media of claim 12 , wherein the de-biasing comprises reducing an effect of one or more feature weights associated with the one or more long text rectangle data items on the TR-level numerical data.