Systems and methods for field extraction from unlabeled data
A field extraction system that does not require field-level annotations for training is provided. Specifically, the training process is bootstrapped by mining pseudo-labels from unlabeled forms using simple rules. Then, a transformer-based structure is used to model interactions between text tokens in the input form and predict a field tag for each token accordingly. The pseudo-labels are used to supervise the transformer training. As the pseudo-labels are noisy, a refinement module that contains a sequence of branches is used to refine the pseudo-labels. Each of the refinement branches conducts field tagging and generates refined labels. At each stage, a branch is optimized by the labels ensembled from all previous branches to reduce label noise.
1. A method for field extraction from forms with unlabeled data through a field extraction model, the method comprising:
receiving, via a data interface, an unlabeled form comprising a plurality of fields and a plurality of field values;
detecting, by a processor, a set of words and a set of locations within the unlabeled form for the set of words;
identifying, from the set of words and the set of locations, a field value for a field based at least in part on a geometric relationship between the set of words and a key for the field by:
determining a set of phrase candidates from the set of words and a set of corresponding phrase locations from the set of locations by grouping a neighborhood of recognized words within a distance,
computing a key score for each phrase candidate indicating a likelihood that the respective phrase candidate is a key for the field, and
determining the key based on a maximum key score among the set of phrase candidates;
encoding, by an encoder, a pair of a first word and a first location corresponding to the field value into a first representation;
generating, by a classifier, a predicted field classification from the first representation;
computing a first loss objective by comparing the predicted field classification with the field value as a pseudo-label; and
updating the encoder based on the first loss objective via backpropagation.
2. The method of claim 1 , wherein the identifying the field value for the field comprises applying a first rule that one or more words in a form of a key relates to a field name for the field.
3. The method of claim 2 , wherein the identifying the field value for the field comprises applying a second rule that a pair of words that are aligned horizontally or vertically are a key for the field and the field value.
4. The method of claim 3 , wherein the identifying the field value for the field comprises applying a third rule that a word from the set of words that matches a pre-defined key text is the key for the field.
5. The method of claim 1 , wherein the key score is computed based on a string distance between the respective phrase candidate and a pre-defined key.
6. The method of claim 1 , further comprising:
predicting, via a neural model, a respective data type for each phrase candidate;
determining a subset of phrase candidates having data types that match with pre-defined data types for the field;
compute, for each phrase candidate in the subset, a value score indicating a likelihood that the respective phrase candidate is the field value for the field; and
determining the field value based on a maximum value score among the subset of phrase candidates.
7. The method of claim 6 , wherein the value score is computed based on a key score of a localized key corresponding to the field and a geometric relationship metric between the respective phrase candidate and the localized key.
8. The method of claim 7 , wherein the geometric relationship metric is computed based on a string distance and an angle between the respective phrase candidate and the localized key.
9. The method of claim 1 , further comprising:
generating, by a plurality of progressive label ensemble (PLE) branches in parallel, a plurality of predicted labels based on the first representation, respectively; and
computing, at one PLE branch, a loss component by comparing a predicted label at the one PLE branch and predicted labels from previous PLE branches as pseudo-labels,
wherein a first PLE branch from the plurality of PLE branches receives the identified field value for the field as a first pseudo-label.
10. A system for field extraction from forms with unlabeled data through a field extraction model, the method comprising:
a data interface receiving an unlabeled form comprising a plurality of fields and a plurality of field values;
a memory storing a plurality of processor-executed instructions; and
a processor executing the processor-executed instructions to perform operations comprising:
detecting a set of words and a set of locations within the unlabeled form for the set of words;
identifying, from the set of words and the set of locations, a field value for a field based at least in part on a geometric relationship between the set of words and a key for the field by:
determining a set of phrase candidates from the set of words and a set of corresponding phrase locations from the set of locations by grouping a neighborhood of recognized words within a distance,
computing a key score for each phrase candidate indicating a likelihood that the respective phrase candidate is a key for the field, and
determining the key based on a maximum key score among the set of phrase candidates;
encoding, by an encoder, a pair of a first word and a first location corresponding to the field value into a first representation;
generating, by a classifier, a predicted field classification from the first representation;
computing a first loss objective by comparing the predicted field classification with the field value as a pseudo-label; and
updating the encoder based on the first loss objective via backpropagation.
11. The system of claim 10 , wherein the identifying the field value for the field comprises applying a first rule that one or more words in a form of a key relates to a field name for the field.
12. The system of claim 11 , wherein the operation of identifying the field value for the field comprises applying a second rule that a pair of words that are aligned horizontally or vertically are a key for the field and the field value.
13. The system of claim 12 , wherein the operation of identifying the field value for the field comprises applying a third rule that a word from the set of words that matches a pre-defined key text is the key for the field.
14. The system of claim 10 , wherein the key score is computed based on a string distance between the respective phrase candidate and a pre-defined key.
15. The system of claim 10 , wherein the operations further comprise:
predicting, via a neural model, a respective data type for each phrase candidate;
determining a subset of phrase candidates having data types that match with pre-defined data types for the field;
compute, for each phrase candidate in the subset, a value score indicating a likelihood that the respective phrase candidate is the field value for the field; and
determining the field value based on a maximum value score among the subset of phrase candidates.
16. The system of claim 15 , wherein the value score is computed based on a key score of a localized key corresponding to the field and a geometric relationship metric between the respective phrase candidate and the localized key.
17. The system of claim 16 , wherein the geometric relationship metric is computed based on a string distance and an angle between the respective phrase candidate and the localized key.
18. The system of claim 10 , wherein the operations further comprise:
generating, by a plurality of progressive label ensemble (PLE) branches in parallel, a plurality of predicted labels based on the first representation, respectively; and
computing, at one PLE branch, a loss component by comparing a predicted label at the one PLE branch and predicted labels from previous PLE branches as pseudo-labels,
wherein a first PLE branch from the plurality of PLE branches receives the identified field value for the field as a first pseudo-label.