System and method for automatic learning of functions
A method and system learns functions to be associated with data fields of forms to be incorporated into an electronic document preparation system. The functions are essentially sets of operations required to calculate the data field. The method and system receive form data related to a data field that expects data values resulting from performing specific operations. The method and system utilize machine learning and training set data to generate, test, and evaluate candidate functions to determine acceptable functions.
1. A method performed by one or more processors of a system, the method comprising:
receiving form data having a first data field for which a function needs to be determined;
receiving training set data of a plurality of data values of data fields relating to the first data field, from a database;
generating, for the first data field, one or more dependencies;
generating, for the first data field, two or more distinct candidate functions based on the one or more dependencies and one or more operators from a library of operators, using a machine learning process based on genetic programming;
generating, for each generated candidate function, test data by applying the candidate function to at least a portion of the training set data;
generating matching data indicating how closely the test data matches at least a portion of the training set data;
for at least two candidate functions having desirable fitness function results among the candidate functions:
splitting, of the at least two candidate functions, a first candidate function into at least a first component piece and a second component piece using the machine learning process based on the genetic programming;
assembling at least the first component piece from the first candidate function with at least a portion of a second candidate function, forming a new candidate function using the machine learning process based on the genetic programming; and
iterating between at least the splitting and assembling operations until test data of at least one of the new candidate functions matches the training set data within a predefined margin of error or until the matching data reflects that the candidate functions are not being improved in each iteration, as shown by a relatively constant margin of error using the machine learning process based on the genetic programming; and
incorporating at least one candidate function into an electronic document preparation system.
2. The method of claim 1 , wherein:
the training set data includes field data sets, each field data set including at least a data value for each field-related dependency and a data value for the first data field; and
generating the test data comprises:
substituting each field-related dependency appearing in the candidate function into the candidate function; and
performing operators of the candidate function.
3. The method of claim 2 , wherein generating the matching data comprises:
comparing the test data associated with individual candidate functions with the data values for the first data field to determine whether the test data matches the data value for the first data field; and
assigning points to each candidate function based on how many data values of the test data match the data value for the first data field.
4. The method of claim 1 , wherein the one or more operators include one or more of a mathematical operator, logical operators, a string operator, a Boolean operator, or condition operators.
5. The method of claim 1 , wherein the candidate functions generate numerical data values, Boolean data values, selection of a check box, or a text data value.
6. The method of claim 1 , wherein the one or more dependencies include one or more of a second data field from a form associated with the first data field, multiple data fields from the form associated with the first data field, a data field from a form other than the form associated with the first data field, multiple data fields from multiple different forms, or a constant.
7. The method of claim 1 , wherein the first data field is a field of a new tax form.
8. The method of claim 7 , wherein the training set data includes previously prepared tax returns.
9. The method of claim 7 , wherein the training set data includes fabricated tax returns.
10. A non-transitory computer-readable storage medium comprising instructions that, when executed by one or more processors of a system, cause the system to perform operations comprising:
receiving form data having a first data field for which a function needs to be determined;
receiving training set data of a plurality of data values of data fields relating to the first data field, from a database;
generating, for the first data field, one or more dependencies;
generating, for the first data field, two or more distinct candidate functions based on the one or more dependencies and one or more operators from a library of operators, using a machine learning process based on genetic programming;
generating, for each generated candidate function, test data by applying the candidate function to at least a portion of the training set data;
generating matching data indicating how closely the test data matches at least a portion of the training set data;
for at least two candidate functions having desirable fitness function results among the candidate functions:
splitting, of the at least two candidate functions, a first candidate function into at least a first component piece and a second component piece using the machine learning process based on the genetic programming;
assembling at least the first component piece from the first candidate function with at least a portion of a second candidate function, forming a new candidate function using the machine learning process based on the genetic programming; and
iterating between at least the splitting and assembling operations until test data from at least one of the new candidate functions matches the training set data within a predefined margin of error or until the matching data reflects that the candidate functions are not being improved in each iteration, as shown by a relatively constant margin of error using the machine learning process based on the genetic programming; and
incorporating at least one candidate function into an electronic document management system.
11. A system comprising:
at least one processor; and
at least one memory storing instructions that, when executed by the at least one processor, cause the system to:
receive, with an interface module, form data having a first data field for which a function needs to be determined;
receive, with a data acquisition module, training set data of a plurality of data values of data fields relating to the first data field, from a database;
generate, for the first data field, one or more dependencies;
generate, for the first data field, two or more distinct candidate functions based on the one or more dependencies and one or more operators from a library of operators, using a machine learning process based on genetic programming;
generate, for each generated candidate function, test data by applying the candidate function to at least a portion of the training set data;
generate, matching data indicating how closely the test data matches at least a portion of the training set data;
for at least two candidate functions having desirable fitness function results among the candidate functions:
split, of the at least two candidate functions, a first candidate function into at least a first component piece and a second component piece using the machine learning process based on the genetic programming;
assemble, at least the first component piece from the first candidate function with at least a portion of a second candidate function, forming a new candidate function using the machine learning process based on the genetic programming; and
iterate, between at least the splitting and assembling operations until test data from at least one of the new candidate functions matches the training set data within a predefined margin of error or until the matching data reflects that the candidate functions are not being improved in each iteration, as shown by a relatively constant margin of error using the machine learning process based on the genetic programming; and
incorporate, at least one candidate function into an electronic document management system.
12. The system of claim 11 , wherein:
the training set data includes field data sets, each field data set including at least a data value for each field related dependency and a data value for the first data field; and
generating the test data comprises:
substituting each field-related dependency appearing in the candidate function into the candidate function; and
performing operators of the candidate function.
13. The system of claim 12 , wherein generating the matching data comprises:
comparing, with the machine learning process based on the genetic programming, the test data associated with individual candidate functions with the data value for the first data field to determine whether the test data matches the data value for the first data field; and
assigning, with the machine learning process based on the genetic programming, points to each candidate function based on how many data values of the test data match the data value for the first data field.
14. The system of claim 11 , wherein the one or more operators include one or more of a mathematical operator, logical operators, a string operator, a Boolean operator, or condition operators.
15. The system of claim 11 , wherein the candidate functions generate numerical data values, Boolean data values, selection of a check box, or a text data value.
16. The system of claim 11 , wherein the one or more dependencies include one or more of a second data field from a form associated with the first data field, multiple data fields from the form associated with the first data field, a data field from a form other than the form associated with the first data field, multiple data fields from multiple different forms, or a constant.