Vision-based cell structure recognition using hierarchical neural networks
Methods, systems, and computer program products for vision-based cell structure recognition using hierarchical neural networks and cell boundaries to structure clustering are provided herein. A computer-implemented method includes detecting a style of the given table using at least one style classification model; selecting, based at least in part on the detected style, a cell detection model appropriate for the detected style; detecting cells within the given table using the selected cell detection model; and outputting, to at least one user, information pertaining to the detected cells comprising image coordinates of one or more bounding boxes associated with the detected cells.
1. A computer-implemented method for, given coordinates of cells in a table, inferring row and column structure of the table, the method comprising:
removing one or more cell boxes associated with the cells in the table that do not overlap with any text boxes associated with the table;
expanding one or more remaining cell boxes associated with the cells in the table until each of the one or more remaining cell boxes is expanded to a maximum horizontal width without overlapping with one or more of the other remaining cell boxes;
sampling at the center of each expanded cell box, horizontally and vertically, to determine the number of rows in the table and the number of columns in the table;
determining an alignment for rows and columns of the table based at least in part on the one or more remaining cell boxes prior to said expanding;
using at least one K-means clustering technique on the one or more remaining cell boxes based at least in part on the determined number of rows in the table and the determined number of columns in the table; and
assigning each of the one or more remaining cell boxes to a respective row and a respective column based at least in part on the determined alignment;
wherein the method is carried out by at least one computing device.
2. The computer-implemented method of claim 1 , comprising:
expanding one or more cells into one or more neighboring empty cells upon a determination that portions of text of the one or more cells overlap into the one or more neighboring empty cells.
3. The computer-implemented method of claim 1 , comprising:
splitting one or more multi-text line cells upon a determination that there is at least one empty neighboring cell.
4. The computer-implemented method of claim 3 , comprising:
re-assigning portions of corresponding text from the one or more split cells based at least in part on location of the corresponding text.
5. The computer-implemented method of claim 1 , wherein said expanding the one or more remaining cell boxes associated with the table comprises expanding the one or more remaining cell boxes one at a time, and in a left-to-right and top-to-bottom order.
6. The computer-implemented method of claim 1 , wherein software implementing the method is provided as a service in a cloud environment.
7. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:
remove one or more cell boxes associated with the cells in the table that do not overlap with any text boxes associated with the table;
expand one or more remaining cell boxes associated with the cells in the table until each of the one or more remaining cell boxes is expanded to a maximum horizontal width without overlapping with one or more of the other remaining cell boxes;
sample at the center of each expanded cell box, horizontally and vertically, to determine the number of rows in the table and the number of columns in the table;
determine an alignment for rows and columns of the table based at least in part on the one or more remaining cell boxes prior to said expanding;
use at least one K-means clustering technique on the one or more remaining cell boxes based at least in part on the determined number of rows in the table and the determined number of columns in the table; and
assign each of the one or more remaining cell boxes to a respective row and a respective column based at least in part on the determined alignment.
8. The computer program product of claim 7 , wherein the program instructions executable by the computing device further cause the computing device to:
expand one or more cells into one or more neighboring empty cells upon a determination that portions of text of the one or more cells overlap into the one or more neighboring empty cells.
9. The computer program product of claim 7 , wherein the program instructions executable by the computing device further cause the computing device to:
split one or more multi-text line cells upon a determination that there is at least one empty neighboring cell.
10. The computer program product of claim 9 , wherein the program instructions executable by the computing device further cause the computing device to:
re-assign portions of corresponding text from the one or more split cells based at least in part on location of the corresponding text.
11. The computer program product of claim 7 , wherein said expanding the one or more remaining cell boxes associated with the table comprises expanding the one or more remaining cell boxes one at a time, and in a left-to-right and top-to-bottom order.
12. A system comprising:
a memory; and
at least one processor operably coupled to the memory and configured for:
removing one or more cell boxes associated with the cells in the table that do not overlap with any text boxes associated with the table;
expanding one or more remaining cell boxes associated with the cells in the table until each of the one or more remaining cell boxes is expanded to a maximum horizontal width without overlapping with one or more of the other remaining cell boxes;
sampling at the center of each expanded cell box, horizontally and vertically, to determine the number of rows in the table and the number of columns in the table;
determining an alignment for rows and columns of the table based at least in part on the one or more remaining cell boxes prior to said expanding;
using at least one K-means clustering technique on the one or more remaining cell boxes based at least in part on the determined number of rows in the table and the determined number of columns in the table; and
assigning each of the one or more remaining cell boxes to a respective row and a respective column based at least in part on the determined alignment.
13. The system of claim 12 , wherein the at least one processor operably coupled to the memory is further configured for:
expanding one or more cells into one or more neighboring empty cells upon a determination that portions of text of the one or more cells overlap into the one or more neighboring empty cells.
14. The system of claim 12 , wherein the at least one processor operably coupled to the memory is further configured for:
splitting one or more multi-text line cells upon a determination that there is at least one empty neighboring cell.
15. The system of claim 14 , wherein the at least one processor operably coupled to the memory is further configured for:
re-assigning portions of corresponding text from the one or more split cells based at least in part on location of the corresponding text.
16. The system of claim 12 , wherein said expanding the one or more remaining cell boxes associated with the table comprises expanding the one or more remaining cell boxes one at a time, and in a left-to-right and top-to-bottom order.