Systems and methods for features engineering
Systems and methods for features engineering, in which internal and external signals are received and fused. The fusing is based on meta-data of each of the one or more internal signals and each of the one or more external signals. A set of features is generated based on one or more valid combinations that match a transformation input, the transformation forming part of library of transformations. Finally, a set of one or more features is selected from the plurality of features, based on a predictive strength of each feature. The set of selected features can be used to train and select a machine learning model.
1. A computer-implemented method comprising:
receiving, by a processor, internal signal data;
receiving, by the processor, external signal data;
fusing, by the processor, data from the internal signal data and the external signal data, the fusing based on meta-data of each of the internal signal data and each of the external signal data;
generating, by the processor, a plurality of features based on one or more valid combinations that match a transformation input, the transformation forming part of a library of transformations;
selecting, by the processor, one or more features from the plurality of features, based on a predictive strength of each feature, to provide a set of selected features;
training and/or validating, by the processor, one or more machine learning models using the set of selected features;
retraining, by the processor, the one or more machine learning models on an expanded engineered data set comprising data corresponding to the training and/or validation portions of the data as (i) part of a model selection process, or (ii) without the model selection process;
generating, by the processor, prediction data utilizing the retrained one or more machine learning models.
2. The computer-implemented method of claim 1 , wherein the external signal data is at least one of a weather signal and a financial signal.
3. The computer-implemented method of claim 1 , wherein:
at least one of the internal signal data and the external signal data includes a range; and
the at least one of the internal signal data and the external signal data is expanded to include one or more individual fields of the range.
4. The computer-implemented method of claim 1 , wherein training the one or more models comprises:
training, by the processor, one or more configurations of a model of the one or more models.
5. The computer-implemented method of claim 4 , wherein a configuration of the one or more configurations comprises a number of layers in a neural network of the model.
6. The computer-implemented method of claim 1 , wherein the method comprises:
training, by the processor, each machine learning model of a plurality of machine learning models using the set of selected features;
evaluating, by the processor, each trained machine learning model from the plurality of machine learning models for accuracy;
selecting, by the processor, a most-accurate machine learning model; and
executing, by the processor, the most accurate machine learning model on processed data to provide a forecast.
7. A system comprising:
a processor; and
a memory storing instructions that, when executed by the processor, configure the system to:
receive, by the processor, internal signal data;
receive, by the processor, external signal data;
fuse, by the processor, data from the internal signal data and the external signal data, the fusing based on meta-data of each of the internal signal data and each of the external signal data;
generate, by the processor, a plurality of features based on one or more valid combinations that match a transformation input, the transformation forming part of a library of transformations;
select, by the processor, one or more features from the plurality of features, based on a predictive strength of each feature, to provide a set of selected features;
train and/or validate, by the processor, one or more machine learning models using the set of selected features;
retrain, by the processor, the one or more machine learning models on an expanded engineered data set comprising data corresponding to the training and/or validation portions of the data as (i) part of a model selection process, or (ii) without the model selection process;
generate, by the processor, prediction data utilizing the retrained one or more machine learning models.
8. The system of claim 7 , wherein the external signal data is at least one of a weather signal and a financial signal.
9. The system of claim 7 , wherein:
at least one of the internal signal data and the external signal data includes a range; and
the at least one of the internal signal data and the external signal data is expanded to include one or more individual fields of the range.
10. The system of claim 7 , wherein when training the one or more machine learning models, the system is further configured to:
train, by the processor, one or more configurations of a machine learning model of the one or more machine learning models.
11. The system of claim 10 , wherein a configuration of the one or more configurations comprises a number of layers in a neural network of the model.
12. The system of claim 7 , wherein the system is further configured to:
train, by the processor, each machine learning model of a plurality of machine learning models using the set of selected features;
evaluate, by the processor, each trained machine learning model from the plurality of machine learning models for accuracy;
select, by the processor, a most-accurate machine learning model; and
execute, by the processor, the most accurate machine learning model on processed data to provide a forecast.
13. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to:
receive, by a processor, internal signal data;
receive, by the processor, external signal data;
fuse, by the processor, data from the internal signal data and the external signal data, the fusing based on meta-data of each of the internal signal data and each of the external signal data;
generate, by the processor, a plurality of features based on one or more valid combinations that match a transformation input, the transformation forming part of a library of transformations;
select, by the processor, one or more features from the plurality of features, based on a predictive strength of each feature, to provide a set of selected features;
train and/or validate, by the processor, one or more machine learning models using the set of selected features;
retrain, by the processor, the one or more machine learning models on an expanded engineered data set comprising data corresponding to the training and/or validation portions of the data as (i) part of a model selection process, or (ii) without the model selection process;
generate, by the processor, prediction data utilizing the retrained one or more machine learning models.
14. The computer-readable storage medium of claim 13 , wherein the external signal data is at least one of a weather signal and a financial signal.
15. The computer-readable storage medium of claim 13 , wherein:
at least one of the internal signal data and the external signal data includes a range; and
the at least one of the internal signal data and the external signal data is expanded to include one or more individual fields of the range.
16. The computer-readable storage medium of claim 13 , wherein when training the one or more machine learning models, the instructions that when executed by the computer, further cause the computer to:
train, by the processor, one or more configurations of a model of the one or more machine learning models.
17. The computer-readable storage medium of claim 16 , wherein a configuration of the one or more configurations comprises a number of layers in a neural network of the model.
18. The computer-readable storage medium of claim 13 , wherein the instructions that when executed by the computer, further cause the computer to:
train, by the processor, each machine learning model of a plurality of machine learning models using the set of selected features;
evaluate, by the processor, each trained machine learning model from the plurality of machine learning models for accuracy;
select, by the processor, a most-accurate machine learning model; and
execute, by the processor, the most accurate machine learning model on processed data to provide a forecast.