Automated feature engineering for machine learning models
Training computer models by generating time-aware training datasets is provided. A system receives a secondary dataset to be combined with a primary dataset for generation of a training dataset. The primary dataset includes a plurality of data records where at least one data record corresponds to a time-of-prediction value corresponding to a timestamp at which at least one data record was used to generate a prediction. The secondary dataset includes a plurality of features where at least one feature corresponds to a timestamp value. The system selects a feature within the secondary dataset with a timestamp that precedes or matches a time-of-prediction value for a corresponding data record within the primary dataset. The system generates the training dataset that includes the primary dataset and the selected feature. The system trains a model using the generated training dataset.
1 . A method, comprising:
generating, by one or more processors, a training dataset, wherein generating the training dataset includes:
receiving, by the one or more processors, a secondary dataset to be combined with a primary dataset for generation of a training dataset, the primary dataset comprising a plurality of primary data records, wherein each primary data record in the plurality of primary data records includes respective values of a plurality of primary features, the plurality of primary features including a first timestamp feature, a value of the first timestamp feature representing a time-of-prediction associated with the respective primary data record, and the secondary dataset comprising a plurality of secondary data records, wherein each secondary data record in the plurality of secondary data records includes respective values of a plurality of secondary features, the plurality of secondary features including a second timestamp feature, a value of the second timestamp feature representing a time associated with the respective secondary data record;
selecting, by the one or more processors, a plurality of sets of one or more secondary data records of the plurality of secondary data records, wherein for each primary data record of the plurality of primary data records, a respective set of the plurality of sets corresponds to the primary data record and values of the second timestamp feature of the one or more secondary data records in the respective set precede or match a value of the time-of-prediction associated with the respective primary data record;
determining, by the one or more processors, for each primary data record of the plurality of primary data records, a respective value of a training feature based on the values of one or more features of the respective set of one or more secondary data records corresponding to the primary data record;
generating, by the one or more processors, a plurality of training data records of the training dataset, wherein each respective training data record in the plurality of training data records includes the values of the plurality of primary features of a respective primary data record and the value of the training feature corresponding to the respective primary data record;
removing, by the one or more processors, one or more training features from the training dataset based on an analysis of feature impact values of the plurality of training features:
identifying, by the one or more processors, a candidate feature derived from a pair of training features in the training dataset;
adding, by the one or more processors, the candidate feature to the training dataset based on an analysis of feature interactions between the pair of training features, the analysis indicating that an interpretability of the pair of training features satisfies one or more first criteria and a statistical significance of the candidate feature satisfies a threshold value; and
training, by the one or more processors, a model using the training dataset.
2 . The method of claim 1 , wherein the secondary dataset is received from a user computing device.
3 . The method of claim 1 , further comprising:
executing, by the one or more processors, a feature transformation protocol to at least one feature within the secondary dataset before determining for each primary data record, the respective value of the training feature based on the values of one or more features of the secondary data records.
4 . The method of claim 3 , wherein the feature transformation protocol is dependent upon a feature type of the at least one feature.
5 . The method of claim 3 , wherein the one or more processors extract a new feature from the at least one feature within the secondary dataset by executing the feature transformation protocol, wherein the one or more processors add the new feature to the primary dataset.
6 . The method of claim 1 , further comprising:
revising, by the one or more processors, a time limit of at least one feature derivation window for at least one feature within the secondary dataset.
7 . The method of claim 1 , further comprising:
revising, by the one or more processors, a time-of-prediction value for at least one data record within the primary dataset.
8 . The method of claim 1 , further comprising performing the analysis of the feature interactions between the pair of training features, including calculating a value based on:
a correlation between the pair of training features,
a comparison of coefficients for the pair of training features,
a comparison of signs for the pair of training features, and/or
a number of unique values of each feature in the pair of training features.
9 . The method of claim 1 , further comprising:
determining, by the one or more processors, a recipe for deriving a value of an engineered feature within the primary dataset, the recipe comprising at least one of SQL statements or code; and
in response to receiving a request to generate a new prediction for a new set of feature values using the model, deriving, by the one or more processors, the value of the engineered feature by applying the recipe to the new set of feature values.
10 . The method of claim 9 , wherein the one or more processors apply the recipe at the value of the time-of-prediction for the new set of feature values.
11 . The method of claim 9 , further comprising:
identifying, by the one or more processors, at least one engineered feature not used by the model to generate the new prediction; and
removing, by the one or more processors, at least one SQL statement or code associated with the identified at least one engineered feature from the recipe.
12 . The method of claim 9 , further comprising generating, by the one or more processors, the new prediction, wherein generating the new prediction comprises applying at least one engineered feature associated with a previous session to the model.
13 . The method of claim 1 , further comprising:
generating, by the one or more processors, a lineage map comprising a relationship between at least one data record within the secondary dataset and at least one feature within the primary dataset.
14 . The method of claim 13 , further comprising:
presenting, by the one or more processors on a graphical user interface, a graphical representation of the primary dataset and at least one data record of the secondary dataset visually related in accordance with the lineage map.
15 . The method of claim 1 , further comprising removing, by the one or more processors, at least one feature from the training dataset based on:
determining that the at least one feature is missing at least one value,
determining that the at least one feature corresponds to a constant value,
determining that the at least one feature is a duplicate value, and/or
determining that the at least one feature has a data size that is beyond a threshold.
16 . The method of claim 1 , further comprising removing, by the one or more processors, at least one feature from the training dataset based on:
determining that the at least one feature has an impact that satisfies a first threshold, and/or
determining that the at least one feature has a quality score that satisfies a second threshold.
17 . The method of claim 16 , further comprising calculating, by the one or more processors, a score based at least in part on a shapley value, wherein determining whether the at least one feature has the impact that satisfies the first threshold is based on the shapley value.
18 . A computer system comprising one or more processors coupled to a memory, the one or more processors configured to perform operations including:
generating a training dataset, wherein generating the training dataset includes:
receiving a secondary dataset to be combined with a primary dataset for generation of a training dataset, the primary dataset comprising a plurality of primary data records, wherein each primary data record in the plurality of primary data records includes respective values of a plurality of primary features, the plurality of primary features including a first timestamp feature, a value of the first timestamp feature representing a time-of-prediction associated with the respective primary data record, and the secondary dataset comprising a plurality of secondary data records, wherein each secondary data record in the plurality of secondary data records includes respective values of a plurality of secondary features, the plurality of secondary features including a second timestamp feature, a value of the second timestamp feature representing a time associated with the respective secondary data record;
selecting a plurality of sets of one or more secondary data records of the plurality of secondary data records of the plurality of secondary data records, wherein for each primary data record of the plurality of primary data records, a respective set of the plurality of sets corresponds to the primary data record and values of the second timestamp feature of the one or more secondary data records in the respective set precede or match a value of the time-of-prediction associated with the respective primary data record;
determining, for each primary data record of the plurality of primary data records, a respective value of a training feature based on the values of one or more features of the respective set of one or more secondary data records corresponding to the primary data record;
generating a plurality of training data records of the training dataset, wherein each respective training data record in the plurality of training data records includes the values of the plurality of primary features of a respective primary data record and the value of the training feature corresponding to the respective primary data record;
removing one or more training features from the training dataset based on an analysis of feature impact values of the plurality of training features;
identifying a candidate feature derived from a pair of training features in the training dataset;
adding the candidate feature to the training dataset based on an analysis of feature interactions between the pair of training features, the analysis indicating that an interpretability of the pair of training features satisfies one or more first criteria and a statistical significance of the candidate feature satisfies a threshold value; and
train a model using the training dataset.
19 . The computer system of claim 18 , wherein the secondary dataset is received from a user computing device.
20 . A non-transitory computer-readable storage device storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations including:
generating a training dataset, wherein generating the training dataset includes:
receiving a secondary dataset to be combined with a primary dataset for generation of a training dataset, the primary dataset comprising a plurality of primary data records, wherein each primary data record in the plurality of primary data records includes respective values of a plurality of primary features, the plurality of primary features including a first timestamp feature, a value of the first timestamp feature representing a time-of-prediction associated with the respective primary data record, and the secondary dataset comprising a plurality of secondary data records, wherein each secondary data record in the plurality of secondary data records includes respective values of a plurality of secondary features, the plurality of secondary features including a second timestamp feature, a value of the second timestamp feature representing a time associated with the respective secondary data record;
selecting a plurality of sets of one or more secondary data records of the plurality of secondary data records, wherein for each primary data record of the plurality of primary data records, a respective set of the plurality of sets corresponds to the primary data record and values of the second timestamp feature of the one or more secondary data records in the respective set precede or match a value of the time-of-prediction associated with the respective primary data record;
determining, for each primary data record of the plurality of primary data records, a respective value of a training feature based on the values of one or more features of the respective set of one or more secondary data records corresponding to the primary data record;
generating a plurality of training data records of the training dataset, wherein each respective training data record in the plurality of training data records includes the values of the plurality of primary features of a respective primary data record and the value of the training feature corresponding to the respective primary data record;
removing one or more training features from the training dataset based on an analysis of feature impact values of the plurality of training features;
identifying a candidate feature derived from a pair of training features in the training dataset;
adding the candidate feature to the training dataset based on an analysis of feature interactions between the pair of training features, the analysis indicating that an interpretability of the pair of training features satisfies one or more first criteria and a statistical significance of the candidate feature satisfies a threshold value; and
training a model using the training dataset.