System and method for auto-query generation
Various systems and methods provide an intuitive user interface that enables automatic specification of queries and constraints for analysis by ML component. Various implementations provide methodologies for automatically formulating machine learning (“ML”) and optimization queries. The automatic generation of ML and/or optimization queries can be configured to use examples to facilitate formulation of ML and optimization queries. One example method includes accepting input data specifying variables and data values associated with the variables. Within the input data any unspecified data records are identified, and a relationship between the variables specified in the input data and a variable associated with the at least one unspecified data record is automatically determined. The relationship can be automatically determined based on training data contained within the input data. Once a relationship is established a ML problem can be automatically generated.
1. A method, comprising:
accessing, by a computer system, input data comprising:
training data comprising a first plurality of data records, each of the first plurality of data records having numeric data values at least for first and second variables in a plurality of variables; and
a second plurality of data records comprising at least a first not fully specified data record that does not include a first numeric data value for the first variable in the plurality of variables;
generating, by the computer system and using the training data, a representation of a relationship between the first variable and at least the second variable in the plurality of variables; and
obtaining, by the computer system and using the generated representation, at least the first numeric data value for the first variable in the first not fully specified data record.
2. A system, comprising:
using at least one hardware processor to perform:
accessing input data comprising:
training data comprising a first plurality of data records, each of the first plurality of data records having numeric data values at least for first and second variables in a plurality of variables; and
a second plurality of data records comprising at least a first not fully specified data record that does not include a first numeric data value for the first variable in the plurality of variables;
generating, using the training data, a representation of a relationship between the first variable and at least the second variable in the plurality of variables; and
obtaining, by the computer system and using the generated representation, at least the first numeric data value for the first variable in the first not fully specified data record.
3. The method of claim 1 , wherein generating the representation comprises using a machine learning technique to generate a mathematical model representing the relationship between the first variable and at least the second variable.
4. The method of claim 1 , wherein generating the mathematical model representing the relationship between the first variable and at least the second variable is performed by using symbolic regression.
5. The method of claim 1 , wherein accessing the input data is performed using a spreadsheet user interface, and wherein generating the representation of the relationship comprises:
generating one or more candidate query patterns that relate a blank cell in the spreadsheet user interface to its surrounding non-blank cells in the spreadsheet user interface;
for each pattern in the one or more candidate query patterns:
collecting data from the spreadsheet user interface that matches the pattern,
generating, using a machine learning technique and the collected data, a respective candidate mathematical model representing a relationship between a variable associated with the blank cell and one or more variables associated with the surrounding non-blank cells;
determining a confidence associated with the generated candidate mathematical model; and
selecting the generated candidate mathematical model having a highest confidence level as the representation of the relationship.
6. The method of claim 1 , further comprising:
identifying, by the computer system, the first not fully specified data record in the second plurality of data records.
7. The method of claim 1 , wherein accessing the input data comprises:
receiving, within a spreadsheet user interface, a selection of the first plurality of data records and the second plurality of data records.
8. The method of claim 1 , further comprising:
determining a confidence value for the first numeric data value; and
displaying the first numeric data value with an intensity that depends on the confidence value.
9. The method of claim 1 , wherein the first not fully specified data record does include a second numeric data value for the second variable in the plurality of variables and does not include a third numeric data value for a third variable in the plurality of variables, and wherein the obtaining comprises obtaining, using the generated representation and an optimization technique, the first numeric data value and the third numeric data value based on the second numeric data value.
10. The system of claim 2 , wherein generating the representation comprises using a machine learning technique to generate a mathematical model representing the relationship between the first variable and at least the second variable.
11. The system of claim 2 , wherein generating the mathematical model representing the relationship between the first variable and at least the second variable is performed by using symbolic regression.
12. The system of claim 2 , wherein accessing the input data is performed using a spreadsheet user interface, and wherein generating the representation of the relationship comprises:
generating one or more candidate query patterns that relate a blank cell in the spreadsheet user interface to its surrounding non-blank cells in the spreadsheet user interface;
for each pattern in the one or more candidate query patterns:
collecting data from the spreadsheet user interface that matches the pattern,
generating, using a machine learning technique and the collected data, a respective candidate mathematical model representing a relationship between a variable associated with the blank cell and one or more variables associated with the surrounding non-blank cells;
determining a confidence associated with the generated candidate mathematical model; and
selecting the generated candidate mathematical model having a highest confidence level as the representation of the relationship.
13. The system of claim 2 , further comprising:
identifying, by the computer system, the first not fully specified data record in the second plurality of data records.
14. The system of claim 2 , wherein accessing the input data comprises:
receiving, within a spreadsheet user interface, a selection of the first plurality of data records and the second plurality of data records.
15. The system of claim 2 , further comprising:
determining a confidence value for the first numeric data value; and
displaying the first numeric data value with an intensity that depends on the confidence value.
16. The system of claim 2 , wherein the first not fully specified data record does include a second numeric data value for the second variable in the plurality of variables and does not include a third numeric data value for a third variable in the plurality of variables, and wherein the obtaining comprises obtaining, using the generated representation and an optimization technique, the first numeric data value and the third numeric data value based on the second numeric data value.
17. The method of claim 7 , wherein the generating and obtaining occur in response to receiving the selection.
18. The method of claim 9 , wherein the optimization technique comprises a gradient search technique.
19. The system of claim 14 , wherein the generating and obtaining occur in response to receiving the selection.
20. The system of claim 16 , wherein the optimization technique comprises a gradient search technique.