Deep learning based table detection and associated data extraction from scanned image documents
The need for extracting information trapped in unstructured document images is becoming more acute. A major hurdle to this objective is that these images often contain information in the form of tables and extracting data from tabular sub-images presents a unique set of challenges. Embodiments of the present disclosure provide systems and methods that implement a deep learning network for both table detection and structure recognition, wherein interdependence between table detection and table structure recognition are exploited to segment out the table and column regions. This is followed by semantic rule-based row extraction from the identified tabular sub-regions.
1. A processor implemented method, comprising:
receiving, via one or more hardware processors, a scanned image document comprising one or more tables and text;
extracting and highlighting, via the one or more hardware processors, at least one of one or more non-numerical values and one or more numerical values from the one or more tables and associated text, using a text recognition technique;
inputting, via the one or more hardware processors, the extracted and highlighted non-numerical and numerical values being appended to the scanned image document, to a deep learning network to obtain a set of learnt features;
generating, via the one or more hardware processors, a masked table region and one or more masked column regions using the extracted and highlighted non-numerical and numerical values based on the set of learnt features;
applying, via the one or more hardware processors, one or more domain-based rules on the masked table region to obtain one or more rows; and
identifying, via the one or more hardware processors, tuples in the one or more tables based on one or more columns from the masked column regions and the one or more obtained rows from the masked table region.
2. The processor implemented method of claim 1 , wherein the step of extracting and highlighting at least one of one or more non-numerical values and one or more numerical values from the one or more tables and associated text, using a text detection and recognition technique, is based on identified semantic information using the text from the scanned image document.
3. The processor implemented method of claim 1 , wherein the masked table region and the one or more masked column regions are generated to determine boundary of the one or more columns and the one or more tables comprised in the scanned image document.
4. The processor implemented method of claim 1 , wherein the step of identifying tuples in the one or more tables is based on an intersection of the one or more columns from the masked column regions and the one or more obtained rows in the masked table region.
5. A system, comprising:
a memory storing instructions;
one or more communication interfaces; and
one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to:
receive, via one or more hardware processors, a scanned image document comprising one or more tables and text;
extract and highlight, via the one or more hardware processors, at least one of one or more non-numerical values and one or more numerical values from the one or more tables and associated text, using a text recognition technique;
input, via the one or more hardware processors, the extracted and highlighted non-numerical and numerical values being appended to the scanned image document, to a deep learning network comprised in the system to obtain a set of learnt features;
generate, via the one or more hardware processors, a masked table region and one or more masked column regions using the extracted and highlighted non-numerical and numerical values based on the set of learnt features;
apply, via the one or more hardware processors, one or more domain-based rules on the masked table region to obtain one or more rows; and
identify, via the one or more hardware processors, tuples in the one or more tables based on one or more columns from the masked column regions and the one or more obtained rows from the masked table region.
6. The system of claim 5 , wherein at least one of one or more non-numerical values and one or more numerical values extracted and highlighted from the one or more tables and associated text, using the text detection and recognition technique, is based on identified semantic information using the text from the scanned image document.
7. The system of claim 5 , wherein the masked table region and the one or more masked column regions are generated to determine boundary of the one or more columns and the one or more tables comprised in the scanned image document.
8. The system of claim 5 , wherein the tuples are identified in the one or more tables based on an intersection of the one or more columns from the masked column regions and the one or more obtained rows in the masked table region.
9. One or more non-transitory machine readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause deep learning based table detection and associated data extraction from scanned image documents by:
receiving, via the one or more hardware processors, a scanned image document comprising one or more tables and text;
extracting and highlighting, via the one or more hardware processors, at least one of one or more non-numerical values and one or more numerical values from the one or more tables and associated text, using a text detection and recognition technique;
inputting, via the one or more hardware processors, the extracted and highlighted non-numerical and numerical values to the deep learning network to obtain a set of learnt features;
generating, via the one or more hardware processors, a masked table region and one or more masked column regions using the set of learnt features;
applying, via the one or more hardware processors, one or more domain-based rules on the masked table region to obtain one or more rows; and
identifying, via the one or more hardware processors, tuples in the one or more tables based on one or more columns from the masked column regions and the one or more obtained rows from the masked table region.
10. The one or more non-transitory machine readable information storage mediums of claim 9 , the step of extracting and highlighting at least one of one or more non-numerical values and one or more numerical values from the one or more tables and associated text, using a text detection and recognition technique, is based on identified semantic information using the text from the scanned image document.
11. The one or more non-transitory machine readable information storage mediums of claim 9 , wherein the masked table region and the one or more masked column regions are generated to determine boundary of the one or more columns and the one or more tables comprised in the scanned image document.
12. The one or more non-transitory machine readable information storage mediums of claim 9 , wherein the step of identifying tuples in the one or more tables is based on an intersection of the one or more columns from the masked column regions and the one or more obtained rows in the masked table region.