Method and apparatus for detecting pagination constructs including a header and a footer in legacy documents
A method for identifying header/footer content of a document, in order to sequence text fragments comprising recognizable text blocks as derived from the document. The textual variability of lines comprised of text blocks, including the different kinds of text blocks within the line is analyzed for assessment of textual variability. Header/footer zones are defined by textual content having a low textual variability. An alternative embodiment identifies pagination constructs by comparing selected text-boxes for similarity and proximity and clustering the text boxes satisfying a predetermined similarity value, wherein the clustered text boxes are deemed to comprise pagination constructs.
1. A method for detecting pagination constructs in a document, the method comprising:
providing by a converter page segmentation and positional information for a number of text-boxes in a document;
determining by a processor a value for each of three features including a page proximity, textual content, and a geometric similarity between text-boxes, wherein the determining the geometric similarity includes modifying a coordinate system of each page for bounding all text-boxes on a page such that an origin for the modified coordinate system starts at a top corner of a text-box and moves toward a center of the text-box;
computing a similarity measure for a text-box using the values; and,
in response to the similarity measure meeting a threshold, associating the text-boxes as having pagination elements.
2. The method of claim 1 , wherein the determining the value of the textual content includes:
assigning a value of “1” for a textual similarity measure if the textual content between the at least two text-boxes is the same; and,
assigning a value of “0” for the textual similarity measure if the textual content between the at least two text-boxes is not the same.
3. The method of claim 1 , wherein the determining the value of the geometric similarity includes determining a position of the text-boxes on a page.
4. The method of claim 3 , wherein the position is determined as an area on a page.
5. The method of claim 1 , wherein the page proximity includes a distance between text-boxes.
6. The method of claim 5 , wherein the distance is determined as a number of pages that the text-boxes are on.
7. The method of claim 1 , further including:
assigning a value of “1” for the page proximity if a number of pages situated between pages including two text-boxes is less than a predetermined number; and,
assigning a value of “0” for the page proximity if the number of pages situated between the pages including the two text-boxes is greater than the predetermined number.
8. An apparatus adapted to determine pagination constructs, the apparatus comprising:
a converter for providing positional information for a number of text-boxes in a segregated document;
a fragmenter for breaking the segregated document into a sequence of pages and text-boxes; and,
a processor for determining a value for each of three features including a page proximity, textual content, and a geometric similarity between text-boxes, the processor identifies a position of the text-boxes and computes the geometric similarity based on a ratio of a surface of an intersection between two text-boxes by a surface of a union of the two-text boxes, the processor further computing a similarity measure for a text-box using the values;
wherein the processor is adapted to include in a cluster of pagination constructs the text-boxes having the similarity measure that meets a threshold.
9. The apparatus of claim 8 , wherein the processor assigns a value of “1” for a textual similarity measure if the textual content between the text-boxes is the same and assign a value of “0” for the textual similarity measure if the textual content between the text-boxes is not the same.
10. The apparatus of claim 8 , wherein the processor identifies a first number of pages that each text-box is on, and the processor further computes the page proximity measure based on a second number of pages between two text-boxes, the page proximity value being further based on a threshold.
11. A method for detecting pagination constructs in a document, the method comprising:
providing by a converter page segmentation and positional information for a number of text-boxes in a document;
determining by a processor a value for each of three features including a page proximity, textual content, and a geometric similarity between text-boxes, wherein the determining the geometric similarity includes:
enhancing a border of the text-boxes by a preselected number of pixels, and
computing a ratio of a surface of an intersection between two text-boxes by a surface of a union of the two-text boxes;
computing a similarity measure for a text-box using the values; and,
in response to the similarity measure meeting a threshold, associating the text-boxes as having pagination elements.
12. The method of claim 11 , wherein the determining the value of the textual content includes:
assigning a value of “1” for a textual similarity measure if the textual content between the at least two text-boxes is the same; and,
assigning a value of “0” for the textual similarity measure if the textual content between the at least two text-boxes is not the same.
13. The method of claim 11 , wherein the determining the value of the geometric similarity includes determining a position of the text-boxes on a page.
14. The method of claim 13 , wherein the position is determined as an area on a page.
15. The method of claim 11 , wherein the page proximity includes a distance between text-boxes.
16. The method of claim 15 , wherein the distance is determined as a number of pages that the text-boxes are on.
17. The method of claim 11 , further including:
assigning a value of “1” for the page proximity if a number of pages situated between pages including two text-boxes is less than a predetermined number; and,
assigning a value of “0” for the page proximity if the number of pages situated between the pages including the two text-boxes is greater than the predetermined number.