Systems and methods for generating tables from print-ready digital source documents
Systems and methods are provided for generating tables from print-ready digital source documents. A document is received and one or more text fragments are identified on a rendered page of the document. A wrapping region collection is generated, comprising one or more wrapping regions. A tabular, narrative and label score is generated for each wrapping region. A block type is assigned to each wrapping region based on the scores. A wrapping region group and a block set are generated. One or more tables are generated based on text fragments corresponding to one of the one or more blocks. The text fragments are organized into corresponding fields of the one or more tables.
1. A method comprising:
receiving, by one or more processors, a digital document comprising at least one rendered page;
generating, by the one or more processors, one or more wrapping regions based on the digital document, wherein each wrapping region comprises one or more fragment runs, and wherein each fragment run comprises one or more text fragments of the digital document that are adjacent to one another and within a horizontal separation threshold and a vertical separation threshold;
classifying, by the one or more processors, each of the one or more wrapping regions, wherein classifying each of the one or more wrapping regions comprises assigning each of the one or more wrapping regions a respective type;
generating, by the one or more processors, one or more blocks of wrapping regions based on the classifications of the one or more wrapping regions;
generating, by the one or more processors, one or more tables based on one or more blocks of wrapping regions; and
providing, by the one or more processors, an electronic document comprising the one or more tables.
2. The method of claim 1 , wherein the digital document comprises data distributed on fixed locations on the least one rendered page.
3. The method of claim 1 , wherein at least one of the horizontal separation threshold or the vertical separation threshold is determined by:
determining a default value for at least one of the horizontal separation threshold the vertical separation threshold, and
modifying least one of the horizontal separation threshold the vertical separation threshold based on one or more properties of the one or more wrapping regions.
4. The method of claim 1 , wherein at least one of the horizontal separation threshold or the vertical separation threshold is determined based on:
a number of overlaps between the one or more wrapping regions.
5. The method of claim 1 , wherein at least one of the horizontal separation threshold or the vertical separation threshold is determined based on:
a number of the one or more wrapping regions having a single-line.
6. The method of claim 1 , wherein at least one of the horizontal separation threshold or the vertical separation threshold is determined based on:
a number of the one or more wrapping regions having multiple lines and not overlapping any other ones of the one or more wrapping regions.
7. The method of claim 1 , wherein the horizontal separation threshold is determined by:
calculating, for each of a plurality of candidate horizontal separation thresholds, a respective score for that candidate horizontal separation threshold based on one or more properties of the one or more wrapping regions, and
selecting one of the plurality of candidate horizontal separation thresholds as the horizontal separation threshold based on the scores.
8. The method of claim 1 , wherein the vertical separation threshold is determined by:
calculating, for each of a plurality of candidate vertical separation thresholds, a respective score for that candidate vertical separation threshold based on one or more properties of the one or more wrapping regions, and
selecting one of the plurality of candidate vertical separation thresholds as the vertical separation threshold based on the scores.
9. The method of claim 1 , wherein assigning each of the one or more wrapping regions a respective type comprises assigning each of the one or more wrapping regions at least one of:
a tabular type,
a narrative type, or
a label type.
10. The method of claim 1 , wherein the one or more processors are implemented on a cloud computing system, and wherein the digital document is received by the one or more processors from a computer systems remote from the cloud computing system.
11. A system comprising:
one or more processors;
one or more non-transitory computer-readable media including one or more sequences of instructions which, when executed by the one or more processors, causes the one or more processors to perform operations comprising:
receiving a digital document comprising at least one rendered page;
generating one or more wrapping regions based on the digital document, wherein each wrapping region comprises one or more fragment runs, and wherein each fragment run comprises one or more text fragments of the digital document that are adjacent to one another and within a horizontal separation threshold and a vertical separation threshold;
classifying each of the one or more wrapping regions, wherein classifying each of the one or more wrapping regions comprises assigning each of the one or more wrapping regions a respective type;
generating one or more blocks of wrapping regions based on the classifications of the one or more wrapping regions;
generating one or more tables based on one or more blocks of wrapping regions; and
providing an electronic document comprising the one or more tables.
12. The system of claim 11 , wherein the digital document comprises data distributed on fixed locations on the least one rendered page.
13. The system of claim 11 , wherein at least one of the horizontal separation threshold or the vertical separation threshold is determined by:
determining a default value for at least one of the horizontal separation threshold the vertical separation threshold, and
modifying least one of the horizontal separation threshold the vertical separation threshold based on one or more properties of the one or more wrapping regions.
14. The system of claim 11 , wherein at least one of the horizontal separation threshold or the vertical separation threshold is determined based on:
a number of overlaps between the one or more wrapping regions.
15. The system of claim 11 , wherein at least one of the horizontal separation threshold or the vertical separation threshold is determined based on:
a number of the one or more wrapping regions having a single-line.
16. The system of claim 11 , wherein at least one of the horizontal separation threshold or the vertical separation threshold is determined based on:
a number of the one or more wrapping regions having multiple lines and not overlapping any other ones of the one or more wrapping regions.
17. The system of claim 11 , wherein the horizontal separation threshold is determined by:
calculating, for each of a plurality of candidate horizontal separation thresholds, a respective score for that candidate horizontal separation threshold based on one or more properties of the one or more wrapping regions, and
selecting one of the plurality of candidate horizontal separation thresholds as the horizontal separation threshold based on the scores.
18. The system of claim 11 , wherein the vertical separation threshold is determined by:
calculating, for each of a plurality of candidate vertical separation thresholds, a respective score for that candidate vertical separation threshold based on one or more properties of the one or more wrapping regions, and
selecting one of the plurality of candidate vertical separation thresholds as the vertical separation threshold based on the scores.
19. The system of claim 11 , wherein assigning each of the one or more wrapping regions a respective type comprises assigning each of the one or more wrapping regions at least one of:
a tabular type,
a narrative type, or
a label type.
20. The system of claim 11 , further comprising a cloud computing system comprising the one or more processors, and
wherein the digital document is received by the one or more processors of the cloud computing system from a computer systems remote from the cloud computing system.