Efficient globally optimal interpretation of documents
A method is provided for parsing a document having a plurality of lines on which items are listed spanning one or more lines. It includes: obtaining a plurality of candidates, representing hypothetical items within the document, each candidate spanning one or more lines and having a local cost representing a confidence in a quality of the candidate compared to a model; determining labeling costs for intervals of the document defined between pairs of lines, each interval containing candidates therein, each labeling cost reflecting a configuration of the candidates within the interval; identifying a best labeling for each interval based on the labeling costs determined for that interval, the best labeling corresponding to one of the configurations of the candidates within the interval; defining a global objective function; and selecting a subset of the candidates such that the global objective function is optimized, based on the identified best labelings.
1. A method for parsing a document having a plurality of lines on which a series of actual items are listed, the listing of each actual item spanning one or more lines, said method comprising:
obtaining a plurality of candidates representing hypothetical items within the document, each candidate spanning one or more lines of the document and having a local cost associate therewith, said local cost representing a confidence in a quality of the candidate with respect to an item model;
determining a plurality of labeling costs for one or more intervals of the document defined between given pairs of lines of the document, each interval containing a plurality of candidates therein, each said labeling cost reflecting a different configuration of the candidates within the interval for which the labeling cost is determined;
identifying a best labeling for each interval based on a plurality of labeling costs determined for that interval, said best labeling corresponding to one of the different configurations of the candidates within the interval for which the best labeling is being identified; defining a global objective function; and selecting a subset of the candidates such that the global objective function is optimize, based on the identified best labeling.
2. The method of claim 1 , said method further comprising:
constraining the selection of the subset such that no two selected candidates in the subset spans the same line of the document.
3. The method of claim 2 , wherein at least two of the candidates overlap one another.
4. The method of claim 3 , said determining further comprising:
computing a labeling cost, associated with each given candidate within the interval for which the given labeling costs are being determined, from the local cost of each given candidate and a value representing a number of lines, within the interval for which the given labeling costs are being determined, that are not spanned by the given candidate.
5. The method of claim 4 , said computing further comprising: weighting the value with a weighting parameter; and
adding the weighted value to the local cost of the given candidate for which the labeling cost is being computed.
6. The method of claim 5 , said identifying further comprising:
finding a minimum labeling cost of the computed labeling costs over the candidates contained within the interval, said minimum labeling cost corresponding to the identified best labeling.
7. The method of claim 6 , further comprising:
defining a quantity T(n,i) as an optimal cost of selecting a number n of candidates up to a line i; and
recursively computing T(n,i) as
T ( n,i )=min j ( T ( n− 1)+ C *( j,i ) for all i<=L
where L is a total number of lines in the document and C*(j,i) is the identified best labeling for the interval defined by the pair of lines j and i.
8. The method of claim 7 , wherein a global optimal cost is given by min T(n,L).
9. A system for parsing a document having a plurality of lines on which a series of actual items are listed, the listing of each actual item spanning one or more lines, said system comprising: a processor operative to:
obtain a plurality of candidates representing hypothetical items within the document, each candidate spanning one or more lines of the document and having a local cost associate therewith, said local cost representing a confidence in a quality of the candidate with respect to an item model;
determine a plurality of labeling costs for one or more intervals of the document defined between given pairs of lines of the document, each interval containing a plurality of candidates therein, each said labeling cost reflecting a different configuration of the candidates within the interval for which the labeling cost is determined;
identify a best labeling for each interval based on a plurality of labeling costs determined for that interval, said best labeling corresponding to one of the different configurations of the candidates within the interval for which the best labeling is being identified; define a global objective function; and select a subset of the candidates such that the global objective function is optimize, based on the identified best labeling.
10. The system of claim 9 , said processor further operative to: constrain the selection of the subset such that no two selected candidates in the subset spans the same line of the document.
11. The system of claim 10 , wherein at least two of the candidates overlap one another.
12. The system of claim 11 , said processor further operative to: compute a labeling cost, associated with each given candidate within the interval for which the given labeling costs are being determined, from the local cost of each given candidate and a value representing a number of lines, within the interval for which the given labeling costs are being determined, that are not spanned by the given candidate.
13. The system of claim 12 , said processor further operative to: weight the value with a weighting parameter; and add the weighted value to the local cost of the given candidate for which the labeling cost is being computed.
14. The system of claim 13 , said processor further operative to: find a minimum labeling cost of the computed labeling costs over the candidates contained within the interval, said minimum labeling cost corresponding to the identified best labeling.
15. The system of claim 14 , said processor further operative to: defining a quantity T(n,i) as an optimal cost of selecting a number n of candidates up to a line i; and recursively compute T(n,i) as T(n,i)=minj_i(T(n−1,j)+C*(j,i) for all i L where L is a total number of lines in the document and C*(j,i) is the identified best labeling for the interval defined by the pair of lines j and i.
16. The system of claim 15 , wherein a global optimal cost is given by min T(n,L).