Systems and methods for machine learning-based data extraction
In some aspects, the disclosure is directed to methods and systems for machine learning-based data extraction using multiple string searching models. String extraction logic may differ depending on the type of document received. For documents identified to contain line item structures, broader searching models are applied to the document to account for the increased variability of data in the document inherent in data organized in line item structures. For documents identifier to contain non-line item structures, stricter searching models are applied to the document to account for predictable data in the document associated with data organized in non-line item structures.
1. A method for machine learning-based data extraction, comprising:
receiving, by a computing device, a document comprising one or more strings, the document associated with a document label separate from the one or more strings;
determining, by the computing device, whether the document label indicates a type of document structure;
receiving, by the computing device, a rule associated with the document label comprising a string sequence;
determining, by the computing device, whether the string sequence matches the one or more strings in the document by applying a regular expression parser to match the string sequence with the one or more strings in the document;
increasing, by the computing device, a confidence score in response to the string sequence matching the one or more strings in the document;
determining, by the computing device, a similarity score indicating a similarity between the string sequence and the one or more strings in the document;
increasing, by the computing device, the confidence score in response to the similarity score meeting or exceeding a first score value; and
displaying, by the computing device, on a display, the one or more strings in the document in response to the confidence score meeting or exceeding a second score value.
2. The method of claim 1 , wherein the type of document structure is a line item structure.
3. The method of claim 1 , further comprising:
receiving, by the computing device, a rule constraint associated with the rule;
discarding, by the computer, the one or more strings in response to the one or more strings not satisfying the rule constraint.
4. The method of claim 1 , wherein the similarity score is determined using a fuzzy model.
5. The method of claim 1 , further comprising:
executing, by the computing device, at least one of an optical character recognition model or a natural language processing model to determine the document label.
6. A method for machine learning-based data extraction comprising:
receiving, by a computing device, a document comprising one or more strings, the document associated with a document label separate from the one or more strings;
determining, by the computing device, whether the document label indicates a type of document structure;
applying, by the computing device, a plurality of searching models to the document to generate a corresponding plurality of arrays of one or more strings;
identifying, by the computing device, a first similarity score indicating a similarity between each of the plurality of arrays;
identifying, by the computing device, a second similarity score, different from the first similarity score, indicating a correctness of each of the one or more strings in each of the plurality of arrays, in response to the first similarity score meeting or exceeding a first score value, wherein the second similarity score indicating the correctness of each of the one or more strings in each of the plurality of arrays is determined using a fuzzy model; and
displaying, by the computing device, on a display, the one or more strings in the document in response to the second similarity score meeting or exceeding a second score value.
7. The method of claim 6 , wherein the first similarity score indicating the similarity between each of the plurality of arrays is determined using a similarity measure index.
8. The method of claim 6 , wherein the type of document structure is a non-line item structure.
9. The method of claim 6 , further comprising:
applying, by the computing device, the plurality of searching models to a second document to generate a second corresponding plurality of arrays of one or more strings;
identifying, by the computing device, that a third similarity score indicating a similarity between each of the second corresponding plurality of arrays does not meet or exceed the first score value; and
applying, by the computing device, a second, different plurality of searching models to the second document to generate a corresponding another plurality of arrays of one or more strings.
10. A system for machine learning-based data extraction comprising:
a computing device comprising processing circuitry and a network interface;
wherein the network interface is configured to receive a document comprising one or more strings, the document associated with a document label, separate from the one or more strings, indicating a type of document structure; and
wherein the processing circuitry is configured to:
determine whether the document label indicates a first type of document structure;
receive a rule associated with the document label comprising a string sequence;
determine whether the string sequence matches the one or more strings in the document by applying a regular expression parser to match the string sequence with the one or more strings in the document;
increase a confidence score in response to the string sequence matching the one or more strings in the document;
determine a first similarity score indicating a similarity between the string sequence and the one or more strings in the document;
increase the confidence score based on the similarity score meeting or exceeding a first score value;
display the one or more strings in the document in response to the confidence score meeting or exceeding a second score value.
11. The system of claim 10 , wherein the processing circuitry is further configured to:
determine whether the document label indicates a second type of document structure;
apply a plurality of searching models to the document to generate a corresponding plurality of arrays of one or more strings;
identify a second similarity score indicating a similarity between each of the plurality of arrays; and
identify a third similarity score indicating a correctness of each of the one or more strings in each of the one or more arrays, in response to the second similarity score meeting or exceeding a third score value.
12. The system of claim 10 , wherein the first type of document structure is a line item structure.
13. The system of claim 10 , wherein the processing circuitry is further configured to:
receive a rule constraint associated with the rule; and
discard the one or more strings in response to the one or more strings not satisfying the rule constraint.
14. The system of claim 10 , wherein the first similarity score is determined using a fuzzy model.
15. The system of claim 11 , wherein the second similarity score is determined using a similarity measure index.
16. The system of claim 11 , wherein the processing circuitry is further configured to:
determine whether another document label indicates the second type of document structure, the another document label associated with the document comprising one or more strings;
apply the plurality of searching models to the document to generate the corresponding plurality of arrays of one or more strings;
identify that the second similarity score does not meet or exceed the second score value; and
apply another plurality of searching models to the another document to generate a corresponding another plurality of arrays of one or more strings;
wherein the plurality of searching models comprises a first searching model and a second searching model; and
wherein the another plurality of searching models comprises the first searching model and a third searching model.
17. The system of claim 10 , wherein the processing circuitry is further configured to execute at least one or an optical character recognition model or a natural language processing model to determine the document label.