Integrated feature engineering
A feature engineering application receives a plurality of data sets from different data sources for training a model for making a prediction based on new data. The feature engineering application generates primitives based on the data sets. A primitive is to be applied to a variable in the data sets to synthesize a feature. The feature engineering application also receives a temporal parameter that specifies a temporal value for generating time-based features. After the primitives are generated and the temporal parameter is received, the feature engineering application aggregates the plurality of data entities based on primary variables in the plurality of data entities and generate an entity set based on the aggregation. The feature engineering application then synthesize features, including the time-based features, based on the entity set, at least some of the primitives, and the temporal parameter.
1 . A method comprising:
receiving a plurality of data entities from different data sources;
generating a pool of primitives based on the plurality of data entities, the pool of primitives including hundreds of primitives and each primitive in the pool of primitives including an algorithm that, when applied to one of more of the plurality of data entities, synthesizes a feature having an associated value;
generating an entity set by aggregating the plurality of data entities;
synthesizing a plurality of features by:
generating a pool of features by applying the pool of primitives to the entity set;
updating the pool of features over a plurality of iterations, each of the plurality of iterations comprising:
determining a usefulness score for each feature in the pool of features using a portion of the entity set that is different from a portion of the entity set used in a different iteration; and
removing at least one feature from the pool of features based on the usefulness scores; and
outputting the updated pool of features as the plurality of features in response to a stopping condition; and
generating a machine learning model that is configured to generate an output based on new data by:
training the machine learning model using the plurality of features;
applying the machine learning model to a validation dataset; and
iteratively retraining the machine learning model until applying the machine learning model to the validation dataset achieves a threshold accuracy measurement.
2 . The method of claim 1 , wherein generating the entity set by aggregating the plurality of data entities comprises:
identifying subsets of data entities from the plurality of data entities, each subset of data entities comprising two or more data entities that share a common variable;
generating an intermediate data entity by aggregating data entities in each subset of data entities; and
generating the entity set by aggregating the intermediate data entities.
3 . The method of claim 2 , wherein generating the intermediate data entity by aggregating the data entities in each of the subsets of data entities comprises:
determining a primary variable for each data entity in the subset of data entities; and
identifying a data entity in the subset of data entities as a parent entity and each of one or more other data entities in the subset of data entities as a child entity based on the respective primary variable determined for each data entity in the subset of data entities.
4 . The method of claim 2 , wherein identifying the subsets of data entities from the plurality of data entities comprises:
generating two new data entities from a given data entity of the plurality of data entities based on variables defining the given data entity, each of the two new data entities including a subset of the variables defining the given data entity; and
identifying the subsets of data entities from the plurality of data entities, excluding the given data entity, and the two new data entities.
5 . The method of claim 4 , wherein generating the two new entities from the given entity of the plurality of entities based on variables defining the given entity comprises:
identifying a first primary variable and a second primary variable from the variables defining the given entity;
classifying the variables defining the given entity into a first group of variables and a second group of variables, the first group of variables including the first primary variable and one or more other variables defining the given entity that are related to the first primary variable, the second group of variables including the second primary variable and one or more other variables defining the given entity that are related to the second primary variable;
generating one of the two new entities with the first group of variables and respective values of the first group of variables; and
generating the other one of the two new entities with the second group of variables and respective values of the second group of variables.
6 . The method of claim 1 , wherein synthesizing the plurality of features by applying the pool of primitives to the entity set is based on a temporal value and further comprises:
determining one or more cutoff times based on the temporal value;
extracting data from the entity set based on the one or more cutoff times; and
synthesizing the plurality of features from the extracted data.
7 . A system, comprising:
a computer processor for executing computer program instructions; and
a non-transitory computer-readable memory storing computer program instructions executable by the computer processor to perform operations comprising:
receiving a plurality of data entities from different data sources;
generating a pool of primitives based on the plurality of data entities, the pool of primitives including hundreds of primitives and each primitive in the pool of primitives including an algorithm that, when applied to one of more of the plurality of data entities, synthesizes a feature having an associated value;
generating an entity set by aggregating the plurality of data entities;
synthesizing a plurality of features by:
generating a pool of features by applying the primitives to the entity set;
updating the pool of features over a plurality of iterations, each of the plurality of iterations comprising:
determining a usefulness score for each feature in the pool of features using a portion of the entity set that is different from a portion of the entity set used in a different iteration; and
removing at least one feature from the pool of features based on the usefulness scores; and
outputting the updated pool of features as the plurality of features in response to a stopping condition; and
generating a machine learning model that is configured to generate an output based on new data by:
training the machine learning model using the plurality of features;
applying the machine learning model to a validation dataset; and
iteratively retraining the machine learning model until applying the machine learning model to the validation dataset achieves a threshold accuracy measurement.
8 . The system of claim 7 , wherein generating the entity set by aggregating the plurality of data entities comprises:
identifying subsets of data entities from the plurality of data entities, each subset of data entities comprising two or more data entities that share a common variable;
generating an intermediate data entity by aggregating data entities in each subset of data entities; and
generating the entity set by aggregating the intermediate data entities.
9 . The system of claim 8 , wherein generating the intermediate data entity by aggregating the data entities in each of the subsets of data entities comprises:
determining a primary variable for each data entity in the subset of data entities; and
identifying a data entity in the subset of data entities as a parent entity and each of one or more other data entities in the subset of data entities as a child entity based on the respective primary variable determined for each data entity in the subset of data entities.
10 . The system of claim 8 , wherein identifying the subsets of data entities from the plurality of data entities comprises:
generating two new data entities from a given data entity of the plurality of data entities based on variables defining the given data entity, each of the two new data entities including a subset of the variables defining the given data entity; and
identifying the subsets of data entities from the plurality of data entities, excluding the given data entity, and the two new data entities.
11 . The system of claim 10 , wherein generating the two new entities from the given entity of the plurality of entities based on variables defining the given entity comprises:
identifying a first primary variable and a second primary variable from the variables defining the given entity;
classifying the variables defining the given entity into a first group of variables and a second group of variables, the first group of variables including the first primary variable and one or more other variables defining the given entity that are related to the first primary variable, the second group of variables including the second primary variable and one or more other variables defining the given entity that are related to the second primary variable;
generating one of the two new entities with the first group of variables and respective values of the first group of variables; and
generating the other one of the two new entities with the second group of variables and respective values of the second group of variables.
12 . The system of claim 7 , wherein synthesizing the plurality of features by applying the pool of primitives to the entity set is based on a temporal value and further comprises:
determining one or more cutoff times based on the temporal value;
extracting data from the entity set based on the one or more cutoff times; and
synthesizing the plurality of features from the extracted data.
13 . A non-transitory computer-readable memory storing executable computer program instructions for processing data blocks in a data analytics system, the instructions executable to perform operations comprising:
receiving a plurality of data entities from different data sources;
generating a pool of primitives based on the plurality of data entities, the pool of primitives including hundreds of primitives and each primitive in the pool of primitives including an algorithm that, when applied to one of more of the plurality of data entities, synthesizes a feature having an associated value;
generating an entity set by aggregating the plurality of data entities;
synthesizing a plurality of features by:
generating a pool of features by applying the primitives to the entity set;
updating the pool of features over a plurality of iterations, each of the plurality of iterations comprising:
determining a usefulness score for each feature in the pool of features using a portion of the entity set that is different from a portion of the entity set used in a different iteration; and
removing at least one feature from the pool of features based on the usefulness scores; and
outputting the updated pool of features as the plurality of features in response to a stopping condition; and
generating a machine learning model that is configured to generate an output based on new data by:
training the machine learning model using the plurality of features;
applying the machine learning model to a validation dataset; and
iteratively retraining the machine learning model until applying the machine learning model to the validation dataset achieves a threshold accuracy measurement.
14 . The non-transitory computer-readable memory of claim 13 , wherein generating the entity set by aggregating the plurality of data entities comprises:
identifying subsets of data entities from the plurality of data entities, each subset of data entities comprising two or more data entities that share a common variable;
generating an intermediate data entity by aggregating data entities in each subset of data entities; and
generating the entity set by aggregating the intermediate data entities.
15 . The non-transitory computer-readable memory of claim 14 , wherein generating the intermediate data entity by aggregating the data entities in each of the subsets of data entities comprises:
determining a primary variable for each data entity in the subset of data entities; and
identifying a data entity in the subset of data entities as a parent entity and each of one or more other data entities in the subset of data entities as a child entity based on the respective primary variable determined for each data entity in the subset of data entities.
16 . The non-transitory computer-readable memory of claim 14 , wherein identifying the subsets of data entities from the plurality of data entities comprises:
generating two new data entities from a given data entity of the plurality of data entities based on variables defining the given data entity, each of the two new data entities including a subset of the variables defining the given data entity; and
identifying the subsets of data entities from the plurality of data entities, excluding the given data entity, and the two new data entities.
17 . The non-transitory computer-readable memory of claim 13 , wherein synthesizing the plurality of features by applying the pool of primitives to the entity set is based on a temporal value and further comprises:
determining one or more cutoff times based on the temporal value;
extracting data from the entity set based on the one or more cutoff times; and
synthesizing the plurality of features from the extracted data.
18 . The non-transitory computer-readable memory of claim 13 , wherein, during each of the plurality of iterations, determining the usefulness score for each feature in the pool of features is performed by:
constructing a random forest comprising decision trees with nodes representing different features in the pool of features; and
applying the portion of the entity set that is different from the portion of the entity set used in the different iteration set to the random forest.
19 . The method of claim 1 , wherein, during each of the plurality of iterations, determining the usefulness score for each feature in the pool of features is performed by:
constructing a random forest comprising decision trees with nodes representing different features in the pool of features; and
applying the portion of the entity set that is different from the portion of the entity set used in the different iteration set to the random forest.
20 . The system of claim 7 , wherein, during each of the plurality of iterations, determining the usefulness score for each feature in the pool of features is performed by:
constructing a random forest comprising decision trees with nodes representing different features in the pool of features; and
applying the portion of the entity set that is different from the portion of the entity set used in the different iteration set to the random forest.