Interpretable feature discovery with grammar-based Bayesian optimization
A method, a system, and a computer program product for generating an interpretable set of features. One or more search parameters and one or more constraints on one or more search parameters for searching data received from one or more data sources are defined. The data received from one or more data sources is searched using the defined search parameters and constraints. One or more first features are extracted from the searched data. The first features are associated with one or more predictive score values. The searching is repeated in response to receiving a feedback data responsive to the extracted first features. One or more second features resulting from the repeated searching are generated.
1 . A computer-implemented method for dynamically generating predictive features using adaptive grammar rules, the method comprising:
searching, by one or more processors, a dataset comprising a plurality of data points from one or more data sources to determine predictive features for a predictive model;
defining, by the one or more processors, a set of initial grammar rules including constraints on allowable feature structures for the predictive features;
applying, by the one or more processors, the set of initial grammar rules to transform the dataset to generate a plurality of candidate predictive features in a first search space limited by the constraints included in the set of initial grammar rules;
evaluating, by the one or more processors, the plurality of candidate predictive features based on at least one objective function to determine interpretability and predictive performance requirements defined for the predictive model; and
automatically adjusting, by the one or more processors, the set of initial grammar rules based on the evaluation to produce an updated set of grammar rules,
wherein responsive to the updated set of grammar rules, a set of predictive features is generated that meets the interpretability and predictive performance requirements of the predictive model by defining a second search space that is smaller than the first search space thereby avoiding a search of redundant sources of information in the one or more data sources.
2 . The method of claim 1 , wherein the adjusting the grammar rules comprises modifying one or more constraints to allow for a different range of feature structures based on a target predictive score.
3 . The method of claim 1 , wherein the constraints comprise allowable feature types, feature complexity levels, aggregation types, filter criteria, and/or time window parameters.
4 . The method of claim 1 , wherein evaluating the candidate predictive features comprises a multi-objective optimization process that balances interpretability and predictive accuracy.
5 . The method of claim 1 , further comprising:
identifying, by the one or more processors, data patterns in the dataset that influence predictive efficacy of the predictive features; and
automatically adjusting the grammar rules to prioritize feature types with higher relevance level to the identified data patterns.
6 . The computer-implemented method of claim 1 , wherein the method further comprises:
analyzing an existing score of a predictive model to assess its current efficacy in forecasting a specified outcome variable; and
using an objective function to identify predictive features that provide marginal information not captured by the existing score.
7 . The computer-implemented method of claim 6 , wherein adjusting the set of grammar rules is triggered when the marginal information not captured by the existing score falls below a predefined threshold.
8 . A computer program product, for dynamically generating predictive features using adaptive grammar rules, comprising a non-transient machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
searching, by one or more processors, a dataset comprising a plurality of data points from one or more data sources to determine predictive features for a predictive model;
defining, by the one or more processors, a set of initial grammar rules including constraints on allowable feature structures for the predictive features;
applying, by the one or more processors, the set of initial grammar rules to transform the dataset to generate a plurality of candidate predictive features in a first search space limited by the constraints included in the set of initial grammar rules;
evaluating, by the one or more processors, the plurality of candidate predictive features based on at least one objective function to determine interpretability and predictive performance requirements defined for the predictive model; and
automatically adjusting, by the one or more processors, the set of initial grammar rules based on the evaluation to produce an updated set of grammar rules,
wherein responsive to the updated set of grammar rules, a set of predictive features is generated that meets the interpretability and predictive performance requirements of the predictive model by defining a second search space that is smaller than the first search space thereby avoiding a search of redundant sources of information in the one or more data sources.
9 . The computer program product of claim 8 , wherein the adjusting the grammar rules comprises modifying one or more constraints to allow for a different range of feature structures based on a target predictive score.
10 . The computer program product of claim 8 , wherein the constraints comprise allowable feature types, feature complexity levels, aggregation types, filter criteria, and/or time window parameters.
11 . The computer program product of claim 8 , wherein evaluating the candidate predictive features comprises a multi-objective optimization process that balances interpretability and predictive accuracy.
12 . The computer program product of claim 8 , wherein the operations further comprise:
identifying, by the one or more processors, data patterns in the dataset that influence the predictive efficacy of the predictive features; and
automatically adjusting the grammar rules to prioritize feature types with higher relevance level to the identified data patterns.
13 . The computer program product of claim 8 , wherein the operations further comprise:
analyzing an existing score of a predictive model to assess its current efficacy in forecasting a specified outcome variable; and
using an objective function to identify predictive features that provide marginal information not captured by the existing score.
14 . The computer program product of claim 13 , wherein adjusting the set of grammar rules is triggered when the marginal information not captured by the existing score falls below a predefined threshold.
15 . A system for dynamically generating predictive features using adaptive grammar rules comprising:
At least one programmable processor; and
a non-transient machine-readable medium storing instructions that, when executed by the processor, cause the at least one programmable processor to perform operations comprising:
searching, by one or more processors, a dataset comprising a plurality of data points from one or more data sources to determine predictive features for a predictive model;
defining, by the one or more processors, a set of initial grammar rules including constraints on allowable feature structures for the predictive features;
applying, by the one or more processors, the set of initial grammar rules to transform the dataset to generate a plurality of candidate predictive features in a first search space limited by the constraints included in the set of initial grammar rules;
evaluating, by the one or more processors, the plurality of candidate predictive features based on at least one objective function to determine interpretability and predictive performance requirements defined for the predictive model; and
automatically adjusting, by the one or more processors, the set of initial grammar rules based on the evaluation to produce an updated set of grammar rules,
wherein responsive to the updated set of grammar rules, a set of predictive features is generated that meets the interpretability and predictive performance requirements of the predictive model by defining a second search space that is smaller than the first search space thereby avoiding a search of redundant sources of information in the one or more data sources.
16 . The system of claim 15 , wherein the adjusting the grammar rules comprises modifying one or more constraints to allow for a different range of feature structures based on a target predictive score.
17 . The system of claim 15 , wherein the constraints comprise allowable feature types, feature complexity levels, aggregation types, filter criteria, and/or time window parameters.
18 . The system of claim 15 , wherein evaluating the candidate predictive features comprises a multi-objective optimization process that balances interpretability and predictive accuracy.
19 . The system of claim 15 , wherein the operations further comprise:
identifying, by the one or more processors, data patterns in the dataset that influence predictive efficacy of the predictive features; and
automatically adjusting the grammar rules to prioritize feature types with higher relevance level to the identified data patterns.
20 . The system of claim 15 , wherein the operations further comprise:
analyzing an existing score of a predictive model to assess its current efficacy in forecasting a specified outcome variable; and
using an objective function to identify predictive features that provide marginal information not captured by the existing score.