Automatic generation of training and testing data for machine-learning models
Provided are computing systems, methods, and platforms for generating training and testing data for machine-learning models. The operations can include receiving signal extraction information that has instructions to query a data store. Additionally, the operations can include accessing, using Structured Query Language (SQL) code generated based on the signal extraction information, raw data from the data store. Moreover, the operations can include processing the raw data using signal configuration information to generate a plurality of signals. The signal configuration information can have instructions on how to generate the plurality of signals from the raw data. Furthermore, the operations can include joining, using SQL code, the plurality of signals with a first label source to generate training data and testing data. Subsequently, the operations can include processing the training data and the testing data to generate the input data. The input data being an ingestible for a machine-learning pipeline.
1. A computer-implemented method for generating input data for training a machine-learning model, the method comprising:
receiving, from a user input, signal configuration information having instructions to generate a plurality of signals from raw data, wherein the user input includes custom code to be executed using an on-the-fly operation, the custom code defining a first signal and how to generate the first signal using the raw data;
receiving signal extraction information that has instructions to query a data store;
accessing, using Structured Query Language (SQL) code that is generated based on the signal extraction information, the raw data from the data store;
processing the raw data using the signal configuration information to generate the plurality of signals;
determining that the first signal is a new signal because the first signal was not previously generated in the prior iteration of the plurality of signals;
determining to omit a backfilling operation of the new signal because the first signal is directly generated from the raw data;
joining, using the SQL code, the plurality of signals with a first label source to generate training data and testing data; and
processing the training data and the testing data to generate input data, the input data being an ingestible file for a machine-learning pipeline.
2. The method of claim 1 , further comprising:
joining, using SQL code, the plurality of signals and the first label source with a second label source to generate the training data and the testing data, the first label source being a different system than the second label source.
3. The method of claim 2 , wherein the first label source is an annotated data that has human-generated labels, and the second label source is a data source that has feedback information based on user interaction.
4. The method of claim 3 , wherein the first label source is associated with a first sample weight, and the second label source is associated with a second sample weight, and wherein the joining of the plurality of signals and the first label source with the second label source is further based on the first sample weight and the second sample weight.
5. The method of claim 4 , wherein the first sample weight is based on a confidence level associated with the first label source, and the second sample weight is based on a different confidence level associated with the second label source.
6. The method of claim 1 , wherein the first signal is a base signal that was not generated in a prior iteration of the plurality of signals, the base signal being derived by processing a plurality of inputs obtained from the raw data, the method further comprising: backfilling the base signal using an on-the-fly operation, wherein the on-the-fly operation prevents a full-fledged backfill operation.
7. The method of claim 1 , wherein the data store includes a first data source and a second data source, and wherein the signal extraction information includes instructions to query the first data source and the second data source, the first data source being a different type than the second data source.
8. The method of claim 7 , wherein the first data source is a column in a first dataset and the second data source is a human-generated label in a second dataset.
9. The method of claim 1 , wherein generating the training data and the testing data further comprising:
performing, based on a custom request, a custom-split of the training data and the testing data.
10. The method of claim 9 , wherein the custom request is a date reference, and wherein the training data is associated with data prior to the date reference and the testing data is associated with data after the date reference.
11. The method of claim 9 , wherein the machine-learning model is trained on non-independent and identically distributed (IID) data requiring a custom-split of the training data and the testing data.
12. The method of claim 1 , further comprising:
transmitting an alert when an error occurs during the generating of the plurality of signals.
13. The method of claim 1 , further comprising:
transmitting an alert when an error occurs during the generating of the training data or the testing data.
14. The method of claim 1 , wherein the training data is utilized during the training of the machine-learning model, and the testing data is utilized during a validation testing of the machine-learning model.
15. The method of claim 1 , wherein the plurality of signals are stored in a table pointing to a file dump containing all signals utilized by the machine-learning model, and wherein the table is configured, using a table alias generator, to point to the file dump that is current.
16. A computing system, comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
receiving, from a user input, signal configuration information having instructions to generate the plurality of signals from raw data, wherein the user input includes custom code to be executed using an on-the-fly operation, the custom code defining a first signal and how to generate the first signal using the raw data;
receiving signal extraction information that has instructions to query a data store;
accessing, using Structured Query Language (SQL) code that is generated based on the signal extraction information, the raw data from the data store;
processing the raw data using the signal configuration information to generate the plurality of signals;
determining that the first signal is a new signal because the first signal was not previously generated in the prior iteration of the plurality of signals;
determining to omit a backfilling operation of the new signal because the first signal is directly generated from the raw data;
joining, using the SQL code, the plurality of signals with a first label source to generate training data and testing data; and
processing the training data and the testing data to generate input data, the input data being an ingestible file for a machine-learning pipeline.
17. One or more non-transitory computer-readable media that collectively store a machine-learned model, wherein the machine-learned model has been learned by performance of operations, the operations comprising:
receiving, from a user input, signal configuration information having instructions to generate the plurality of signals from raw data, wherein the user input includes custom code to be executed using an on-the-fly operation, the custom code defining a first signal and how to generate the first signal using the raw data;
receiving signal extraction information that has instructions to query a data store;
accessing, using Structured Query Language (SQL) code that is generated based on the signal extraction information, the raw data from the data store;
processing the raw data using the signal configuration information to generate the plurality of signals;
determining that the first signal is a new signal because the first signal was not previously generated in the prior iteration of the plurality of signals;
determining to omit a backfilling operation of the new signal because the first signal is directly generated from the raw data;
joining, using the SQL code, the plurality of signals with a first label source to generate training data and testing data; and
processing the training data and the testing data to generate input data, the input data being an ingestible file for a machine-learning pipeline.