Method, device, and system, for identifying data elements in data structures
A computer implemented method for identifying one or more data elements in an input data stream formed from an input data structure using processing enabled by a processor-executable instruction can be stored on a non-transitory computerized storage medium. This stored method includes identifying a marker sequence set in the input document string, the marker sequence set including a marker sequence; assigning an eligibility value to each marker sequence, the eligibility value being a number calculated using an eligibility calculation, to obtain at least one eligible marker sequence; generating output data comprising data related to the at least one eligible marker sequence; and providing the output data, such as on a human-readable interface.
1. A computer-implemented method for determining document structure of a document, the method comprising:
generating a document string corresponding to the document;
identifying a first candidate marker at a first position in the document string;
based on determining that the first candidate marker is in a predefined first-element form, adding the first candidate marker to a set of marker sequences associated with the document as a first marker sequence, wherein the first marker sequence corresponds to a first candidate hierarchy for the document structure;
identifying a second candidate marker occurring at a second position in the document string subsequent to the first position;
based on determining that the second candidate marker is in the predefined first-element form, adding the second candidate marker to the set of marker sequences as a second marker sequence, wherein the second marker sequence corresponds to a second candidate hierarchy for the document structure;
identifying a third candidate marker occurring at a third position in the document string subsequent to the second position;
based on determining that the third candidate marker is not in the predefined first-element form, iteratively determining whether the third candidate marker is consecutively congruent to a last element in each respective marker sequence of the set of marker sequences;
based on determining that the third candidate marker is consecutively congruent to the last element in the first marker sequence, adding the third candidate marker to the first marker sequence;
determining at least one eligible marker sequence based on evaluating each marker sequence of the set of marker sequences using at least one marker sequence eligibility criteria; and
returning a determined hierarchy for the document structure based on the at least one eligible marker sequence.
2. The method of claim 1 , wherein each candidate marker is associated with at least one of: a section, a subsection, an element of a numbered list, or an element of an outline numbered list.
3. The method of claim 1 , wherein evaluating each marker sequence is based on sequential markers in the respective marker sequences other than an initial marker.
4. The method of claim 3 , wherein the at least one marker sequence eligibility criteria include one or more rules based on:
a cardinality of a sequential marker set comprising the sequential markers.
5. The method of claim 3 , wherein the at least one marker sequence eligibility criteria include one or more rules based on:
a substring adjacent to the sequential markers, or
a superstring comprising:
at least a portion of a given sequential marker; and
characters adjacent to the given sequential marker.
6. The method of claim 1 , wherein generating the document string comprises:
detecting if at least one removable string is present in text content of the document, wherein a string is determined to be removable if it comprises a page header, a page footer, or a footnote; and
removing the at least one removable string.
7. The method of claim 1 , further comprising:
combining a first eligible marker sequence with a second eligible marker sequence to generate a third eligible marker sequence.
8. The method of claim 1 , wherein determining that the third candidate marker is consecutively congruent to the last element of the first marker sequence is based on determining that the third candidate marker indicates a next section, subsection, or list element associated with the last element.
9. The method of claim 1 , wherein determining that the third candidate marker is consecutively congruent to the last element of the first marker sequence is based on determining that the third candidate marker indicates a child section, subsection, or list element associated with the last element.
10. The method of claim 1 , wherein determining that the third candidate marker is consecutively congruent to the last element of the first marker sequence is based on determining that the third candidate marker indicates a next section, subsection, or list element associated with a parent element of the last element.
11. A document processing apparatus comprising:
at least one processor; and
memory storing instructions that, when executed by the at least one processor, cause the document processing apparatus to:
generate a document string corresponding to a document;
identify a first candidate marker at a first position in the document string;
based on determining that the first candidate marker is in a predefined first-element form, add the first candidate marker to a set of marker sequences associated with the document as a first marker sequence, wherein the first marker sequence corresponds to a first candidate hierarchy for a document structure of the document;
identify a second candidate marker occurring at a second position in the document string subsequent to the first position;
based on determining that the second candidate marker is in the predefined first-element form, add the second candidate marker to the set of marker sequences as a second marker sequence, wherein the second marker sequence corresponds to a second candidate hierarchy for the document structure;
identify a third candidate marker occurring at a third position in the document string subsequent to the second position;
based on determining that the third candidate marker is not in the predefined first-element form, iteratively determine whether the third candidate marker is consecutively congruent to a last element in each respective marker sequence of the set of marker sequences;
based on determining that the third candidate marker is consecutively congruent to the last element in the first marker sequence, add the third candidate marker to the first marker sequence;
determine at least one eligible marker sequence based on evaluating each marker sequence of the set of marker sequences using at least one marker sequence eligibility criteria; and
return a determined hierarchy for the document structure based on the at least one eligible marker sequence.
12. The apparatus of claim 11 , wherein each candidate marker is associated with at least one of: a section, a subsection, an element of a numbered list, or an element of an outline numbered list.
13. The apparatus of claim 11 , wherein the instructions, when executed by the at least one processor, cause the document processing apparatus to evaluate each marker sequence based on sequential markers in the respective marker sequences other than a initial marker, and
wherein the at least one marker sequence eligibility criteria include one or more rules based on a cardinality of a sequential marker set comprising the sequential markers.
14. The apparatus of claim 11 , wherein the instructions, when executed by the at least one processor, cause the document processing apparatus to determine that the third candidate marker is consecutively congruent to the last element of the first marker sequence based on determining that the third candidate marker indicates a next section, subsection, or list element associated with the last element.
15. The apparatus of claim 11 , wherein the instructions, when executed by the at least one processor, cause the document processing apparatus to determine that the third candidate marker is consecutively congruent to the last element of the first marker sequence based on determining that the third candidate marker indicates a child section, subsection, or list element associated with the last element.
16. The apparatus of claim 11 , wherein the instructions, when executed by the at least one processor, cause the document processing apparatus to determine that the third candidate marker is consecutively congruent to the last element of the first marker sequence based on determining that the third candidate marker indicates a next section, subsection, or list element associated with a parent element of the last element.
17. One or more non-transitory computer-readable media storing instructions that, when executed by at least one processor, cause a device to determine document structure of a document by causing the device to perform steps comprising:
generating a document string corresponding to the document;
identifying a first candidate marker at a first position in the document string;
based on determining that the first candidate marker is in a predefined first-element form, adding the first candidate marker to a set of marker sequences associated with the document as a first marker sequence, wherein the first marker sequence corresponds to a first candidate hierarchy for the document structure;
identifying a second candidate marker occurring at a second position in the document string subsequent to the first position;
based on determining that the second candidate marker is not in the predefined first-element form, iteratively determining whether the second candidate marker is consecutively congruent to a last element in each respective marker sequence of the set of marker sequences, wherein each marker sequence corresponds to a respective candidate hierarchy for the document structure;
based on determining that the second candidate marker is consecutively congruent to the last element in the first marker sequence, adding the second candidate marker to the first marker sequence;
determining at least one eligible marker sequence based on evaluating each marker sequence of the set of marker sequences using at least one marker sequence eligibility criteria; and
returning a determined hierarchy for the document structure based on the at least one eligible marker sequence.
18. The computer-readable media of claim 17 , wherein evaluating each marker sequence is based on sequential markers in the respective marker sequences other than an initial marker, and wherein the at least one marker sequence eligibility criteria include one or more rules based on:
a substring adjacent to the sequential markers, or
a superstring comprising:
at least a portion of a given sequential marker; and
characters adjacent to the given sequential marker.
19. The computer-readable media of claim 17 , wherein determining that the second candidate marker is consecutively congruent to the last element of the first marker sequence is based on determining that the second candidate marker indicates a next section, subsection, or list element associated with the last element.
20. The computer-readable media of claim 17 , wherein determining that the second candidate marker is consecutively congruent to the last element of the first marker sequence is based on determining that the second candidate marker indicates:
a child section, subsection, or list element associated with the last element, or
a next section, subsection, or list element associated with a parent element of the last element.