Method and system of extracting label:value data from a document
This disclosure provides an exemplary method and system for extracting structured label and value pairwise textual data from a textual document. According to an exemplary method, initially a layout analysis is performed resulting in one or more alternatives for grouping and ordering the textual elements of interest. Next, textual elements are tagged as including a label term, a value term or a label and value term. Finally, a sequence-based method is applied to the tagged elements to generate one or more sequence listings representative of the label and value pairwise data structure(s) and label:value pairwise data is extracted.
1. A computer implemented method of extracting structural label and value pairwise data associated with a digital version of a document, the method comprising:
a) performing a layout analysis of the digital version of the document to generate one or more layout structures associated with the document, each layout structure including a plurality of structural elements vertically or horizontally aligned where each structural element is defined as a typographical box including one or more lines of textual elements associated with the digital version of the document;
b) processing the one or more lines of textual elements to identify and tag textual elements associated with label and value pairwise data;
c) processing the one or more layout structures and associated tagged textual elements to generate one or more respective label:value sequences of tagged textual elements representative of a respective layout structure; and
d) extracting label and value pairwise data from the one or more label:value sequences of tagged elements,
wherein step c) generates the one or more respective label:value sequences of tagged textual element using a sequence-based method to hierarchically segment a sequence of elements associated with the generated one or more layout structures associated with the digital version of the document; and
wherein the sequence-based method includes:
generating a set of n-prams from the sequence of elements, an n-gram including an ordered sequence of n features provided by a sequence of n named elements;
electing sequential n-grams from the set of n-grams, the sequential n-grams defined as similar contiguous n-grams;
selecting a most frequent sequential n-pram from the elected sequential n-grams; and
generating a new sequence of the elements by matching the selected most frequent sequential n-gram against the sequence of elements associated with the document, replacing matched elements of the sequence of elements with a respective node, and associating the matched elements of the sequence of elements as children of the respective node.
2. The computer implemented method of extracting structured label and value pairwise data according to claim 1 , wherein the structural elements are associated with one or more of a line of text, and a block of text, the text including one or more of letters, numbers and symbols.
3. The computer implemented method of extracting structured label and value pairwise data according to claim 1 , wherein the one or more layout structures are associated with one or more of a 1D (Dimensional) structure and a 2D structure.
4. The computer implemented method of extracting structured label and value pairwise data according to claim 1 , wherein prior to step a) text extraction and graphical segment extraction are performed on the digital document.
5. The computer implemented method of extracting structured label and value pairwise data according to claim 1 , wherein a predefined list of terms is used to identify and tag textual elements associated with label data.
6. The computer implemented method of extracting label and value pairwise data according to claim 1 , wherein a predefined set of one or more patterns is used to identify and tag textual elements associated with value data.
7. The computer implemented method of extracting label and value pairwise data according to claim 1 , wherein step b) tags textual elements as one of a label term, a value term and a label and value term.
8. The computer implemented method of extracting label and value pairwise data according to claim 1 , comprising:
e) generating a final label and value data list from the label and value pairwise data extracted in step d), the final label and value data list generated by mapping the extracted label and value pairwise data to a plurality of predefined canonical designations.
9. The computer implemented method of extracting label and value pairwise data according to claim 8 , wherein an edit-distance score calculated for the extracted label and value pairwise data is used to generate the final label and value data list.
10. The computer implemented method of extracting label and value pairwise data according to claim 1 , wherein the label and value pairwise data is associated with an invoice.
11. An image processing system comprising:
a processor configured to receive a digital version of a document, the processor configured to execute instructions to perform a method of extracting structured label and value pairwise data associated with the digital version of the document, the method comprising:
a) performing a layout analysis of the digital version of the document to generate one or more layout structures associated with the document, each layout structure including a plurality of structural elements vertically or horizontally aligned where each structural element is defined as a typographical box including one or more lines of textual elements associated with the digital version of the document;
b) processing the one or more lines of textual elements to identify and tag textual elements associated with label and value pairwise data;
c) processing the one or more layout structures and associated tagged textual elements to generate one or more respective label:value sequences of tagged textual elements representative of a respective layout structure; and
d) extracting label and value pairwise data from the one or more label:value sequences of tagged elements,
wherein step c) generates the one or more respective label:value sequences of tagged textual element using a sequence-based method to hierarchically segment a sequence of elements associated with the generated one or more layout structures associated with the digital version of the document; and
wherein the sequence-based method includes:
generating a set of n-grams from the sequence of elements, an n-gram including an ordered sequence of n features provided by a sequence of n named elements;
electing sequential n-grams from the set of n-grams, the sequential n-grams defined as similar contiguous n-grams;
selecting a most frequent sequential n-gram from the elected sequential n-grams; and
generating a new sequence of the elements by matching the selected most frequent sequential n-gram against the sequence of elements associated with the document, replacing matched elements of the sequence of elements with a respective node, and associating the matched elements of the sequence of elements as children of the respective node.
12. The image processing system according to claim 11 , wherein the structural elements are associated with one or more of a line of text, and a block of text, the text including one or more of letters, numbers and symbols.
13. The image processing system according to claim 11 , wherein the one or more layout structures are associated with one or more of a 1D (Dimensional) structure and a 2D structure.
14. The image processing system according to claim 11 , wherein prior to step a) text extraction and graphical segment extraction are performed on the digital document.
15. The image processing system according to claim 11 , wherein a predefined list of terms is used to identify and tag textual elements associated with label data.
16. The image processing system according to claim 11 , wherein a predefined set of one or more patterns is used to identify and tag textual elements associated with value data.
17. The image processing system according to claim 11 , wherein step b) tags textual elements as one of a label term, a value term and a label and value term.
18. The image processing system according to claim 11 , wherein the method comprises:
e) generating a final label and value data list from the label and value pairwise data extracted in step d), the final label and value data list generated by mapping the extracted label and value pairwise data to a plurality of predefined canonical designations.
19. The image processing system according to claim 18 , wherein an edit-distance score calculated for the extracted label and value pairwise data is used to generate the final label and value data list.
20. The image processing system according to claim 11 , wherein the label and value pairwise data is associated with an invoice.
21. A computer program product comprising:
a non-transitory computer-usable data carrier storing instructions that, when executed by a computer, cause the computer to perform a method of extracting label and value pairwise data associated with a digital version of a document, the method comprising:
a) performing a layout analysis of the digital version of the document to generate one or more layout structures associated with the document, each layout structure including a plurality of structural elements vertically or horizontally aligned where each structural element is defined as a typographical box including one or more lines of textual elements associated with the digital version of the document;
b) processing the one or more lines of textual elements to identify and tag textual elements associated with label and value pairwise data;
c) processing the one or more layout structures and associated tagged textual elements to generate one or more respective label:value sequences of tagged textual elements representative of a respective layout structure; and
d) extracting label and value pairwise data from the one or more label:value sequences of tagged elements,
wherein step c) generates the one or more respective label:value sequences of tagged textual element using a sequence-based method to hierarchically segment a sequence of elements associated with the generated one or more layout structures associated with the digital version of the document; and
wherein the sequence-based method includes:
generating a set of n-grams from the sequence of elements, an n-gram including an ordered sequence of n features provided by a sequence of n named elements;
electing sequential n-grams from the set of n-grams, the sequential n-grams defined as similar contiguous n-grams;
selecting a most frequent sequential n-gram from the elected sequential n-grams; and
generating a new sequence of the elements by matching the selected most frequent sequential n-gram against the sequence of elements associated with the document, replacing matched elements of the sequence of elements with a respective node, and associating the matched elements of the sequence of elements as children of the respective node.
22. The computer program product according to claim 21 , wherein prior to step a) text extraction and graphical segment extraction are performed on the digital document.
23. The computer program product according to claim 21 , wherein step b) tags textual elements as one of a label term, a value term and a label and value term.
24. The computer program product according to claim 21 , the method comprising:
e) generating a final label and value data list from the label and value pairwise data extracted in step d), the final label and value data list generated by mapping the extracted label and value pairwise data to a plurality of predefined canonical designations.
25. The computer program product according to claim 24 , wherein an edit-distance score calculated for the extracted label and value pairwise data is used to generate the final label and value data list.