Categorical input machine learning models
There is a need for more effective and efficient predictive data analysis based at least in part on categorical input data. This need can be addressed by, for example, solutions for performing predictive data analysis that utilize at least one of categorical level merging, mutual-information-based feature filtering, feature-correlation-based feature filtering to generate training data feature value arrangements, as well as training and using categorical input machine learning models trained using the training data feature value arrangements.
1. A computer-implemented method comprising:
identifying, by one or more processors, an initial training set comprising a plurality of training data entries associated with a plurality of initial training data features, wherein: (i) an initial training data entry of the plurality of training data entries is associated with a respective training data feature value for each of a subset of initial training data features of the plurality of initial training data features and a training target value for a training target feature, and (ii) an initial training data feature of the plurality of initial training data features is associated with a plurality of initial categorical levels;
generating, by the one or more processors, an adjusted training set based at least in part on the initial training set, wherein generating the adjusted training set comprises:
generating a plurality of limited noise features based at least in part on the plurality of initial training data features, wherein generating the plurality of limited noise features comprises:
determining a limited noise subset of the plurality of initial training data features based at least in part on a per-feature mutual information measure for each initial training data feature of the plurality of initial training data features satisfying a condition defined by a mutual information noise floor, and
generating the adjusted training set based at least in part on the plurality of limited noise features;
training a categorical input machine learning model based at least in part on the adjusted training set; and
generating, using the trained categorical input machine learning model, one or more inferred predictions based at least in part on a prediction input data entry.
2. The computer-implemented method of claim 1 , wherein:
prior to generating the plurality of limited noise features based at least in part on the plurality of initial training data features, the plurality of initial training data features is transformed in accordance with a feature transformation routine, and
the feature transformation routine comprises:
generating a plurality of categorically refined features based at least in part on the plurality of initial training data features, wherein: (i) generating the plurality of categorically refined features comprises generating one or more adjusted categorical levels based at least in part on a plurality of initial categorical levels for each initial training data feature, and (ii) generating the one or more adjusted categorical levels for a respective training data feature of the plurality of initial training data features is performed based at least in part on a per-level predictive correlation measure for each initial categorical level of the plurality of initial categorical levels that is associated with the respective initial training data feature in relation to the training target feature, and
updating the plurality of initial training data features based at least in part on the plurality of categorically refined features.
3. The computer-implemented method of claim 2 , wherein generating the one or more adjusted categorical levels based at least in part on the plurality of initial categorical levels for the respective initial training data feature comprises:
for each initial categorical level of the plurality of initial categorical levels, determining the per-level predictive correlation measure;
determining a low-correlation subset of the plurality of initial categorical levels based at least in part on each per-level predictive correlation measure for each initial categorical level of the plurality of initial categorical levels;
assigning each initial categorical level in the low-correlation subset to a merged low-correlation categorical level;
determining a merged per-level predictive correlation measure for the merged low-correlation categorical level in relation to the training target feature;
determining whether the merged per-level predictive correlation measure satisfies a per-level predictive correlation threshold;
in response to determining that the merged per-level predictive correlation measure satisfies the per-level predictive correlation threshold, generating the one or more adjusted categorical levels to comprise each initial categorical level excluded from the low-correlation subset and the merged per-level predictive correlation measure; and
in response to determining that the merged per-level predictive correlation measure fails to satisfy the per-level predictive correlation threshold, generating the one or more adjusted categorical levels to comprise each initial categorical level excluded from the low-correlation subset.
4. The computer-implemented method of claim 1 , wherein generating the adjusted training set based at least in part on the plurality of limited noise features comprises:
generating one or more limited correlation features of the plurality of limited noise features, wherein generating the one or more limited correlation features comprises determining a limited correlation subset of the plurality of limited noise features based at least in part on a per-feature-pair symmetric correlation measure for each feature pair of one or more feature pairs comprising a first limited noise feature of the plurality of limited noise features and a second limited noise feature of the plurality of limited noise features; and
generating the adjusted training set based at least in part on the one or more limited correlation features.
5. The computer-implemented method of claim 4 , wherein generating the one or more limited correlation features comprises:
for each feature pair of the one or more feature pairs:
determining whether the per-feature-pair symmetric correlation measure for the feature pair satisfies a per-feature-pair symmetric correlation threshold, and
in response to determining that the per-feature-pair symmetric correlation measure for the feature pair fails to satisfy the per-feature-pair symmetric correlation threshold, excluding one of the first limited noise feature associated with the feature pair or the second limited noise feature associated with the feature pair from the one or more limited correlation features.
6. The computer-implemented method of claim 1 , wherein generating the plurality of limited noise features comprises:
for each initial training data feature of the plurality of initial training data features, determining the per-feature mutual information measure;
determining the mutual information noise floor for the plurality of initial training data features; and
determining the limited noise subset based at least in part on a selected subset of the plurality of initial training data features whose respective per-feature mutual information measures exceed the mutual information noise floor.
7. The computer-implemented method of claim 6 , wherein determining the mutual information noise floor comprises:
identifying a maximum spurious feature ratio for the plurality of initial training data features;
determining a non-spurious feature quantile value for a determined distribution of each per-feature mutual information measure of a respective initial training data feature of the plurality of initial training data features based at least in part on the maximum spurious feature ratio; and
determining the mutual information noise floor based at least in part on the non-spurious feature quantile value.
8. The computer-implemented method of claim 7 , wherein the determined distribution is a binomial distribution.
9. A computing system comprising one or more processors and memory including program code, the program code configured to, with the one or more processors, cause the computing system to:
identify an initial training set comprising a plurality of training data entries associated with a plurality of initial training data features, wherein: (i) an initial training data entry of the plurality of training data entries is associated with a respective training data feature value for each of a subset of initial training data features of the plurality of initial training data features and a training target value for a training target feature, and (ii) each initial training data feature of the plurality of initial training data features is associated with a plurality of initial categorical levels;
generate an adjusted training set based at least in part on the initial training set, wherein generating the adjusted training set comprises:
generating a plurality of limited noise features based at least in part on the plurality of initial training data features, wherein generating the plurality of limited noise features comprises:
determining a limited noise subset of the plurality of initial training data features based at least in part on a per-feature mutual information measure for each initial training data feature of the plurality of initial training data features satisfying a condition defined by a mutual information noise floor, and
generating the adjusted training set based at least in part on the plurality of limited noise features;
train a categorical input machine learning model based at least in part on the adjusted training set; and
generate, using the trained categorical input machine learning model, one or more inferred predictions based at least in part on a prediction input data entry.
10. The computing system of claim 9 , wherein:
prior to generating the plurality of limited noise features based at least in part on the plurality of initial training data features, the plurality of initial training data features is transformed in accordance with a feature transformation routine, and
the feature transformation routine comprises:
generating a plurality of categorically refined features based at least in part on the plurality of initial training data features, wherein: (i) generating the plurality of categorically refined features comprises generating one or more adjusted categorical levels based at least in part on the plurality of initial categorical levels for each initial training data feature, and (ii) generating the one or more adjusted categorical levels for a respective initial training data feature of the plurality of initial training data features is performed based at least in part on a per-level predictive correlation measure for each initial categorical level of the plurality of initial categorical levels that is associated with the respective initial training data feature in relation to the training target feature, and
updating the plurality of initial training data features based at least in part on the plurality of categorically refined features.
11. The computing system of claim 10 , wherein generating the one or more adjusted categorical levels based at least in part on the plurality of initial categorical levels for the respective initial training data feature of the plurality of initial training data features comprises:
for each initial categorical level of the plurality of initial categorical levels, determining the per-level predictive correlation measure;
determining a low-correlation subset of the plurality of initial categorical levels based at least in part on each per-level predictive correlation measure for each initial categorical level of a plurality of initial categorical levels;
assigning each initial categorical level in the low-correlation subset to a merged low-correlation categorical level;
determining a merged per-level predictive correlation measure for the merged low-correlation categorical level in relation to the training target feature;
determining whether the merged per-level predictive correlation measure satisfies a per-level predictive correlation threshold;
in response to determining that the merged per-level predictive correlation measure satisfies the per-level predictive correlation threshold, generating the one or more adjusted categorical levels to comprise each initial categorical level excluded from the low-correlation subset and the merged per-level predictive correlation measure; and
in response to determining that the merged per-level predictive correlation measure fails to satisfy the per-level predictive correlation threshold, generating the one or more adjusted categorical levels to comprise each initial categorical level excluded from the low-correlation subset.
12. The computing system of claim 9 , wherein generating the adjusted training set based at least in part on the plurality of limited noise features comprises:
generating one or more limited correlation features of the plurality of limited noise features, wherein generating the one or more limited correlation features comprises determining a limited correlation subset of the plurality of limited noise features based at least in part on a per-feature-pair symmetric correlation measure for each feature pair of one or more feature pairs comprising a first limited noise feature of the plurality of limited noise features and a second limited noise feature of the plurality of limited noise features; and
generating the adjusted training set based at least in part on the one or more limited correlation features.
13. The computing system of claim 12 , wherein generating the one or more limited correlation features comprises:
for each feature pair of the one or more feature pairs:
determining whether the per-feature-pair symmetric correlation measure for the feature pair satisfies a per-feature-pair symmetric correlation threshold, and
in response to determining that the per-feature-pair symmetric correlation measure for the feature pair fails to satisfy the per-feature-pair symmetric correlation threshold, excluding one of the first limited noise feature associated with the feature pair or the second limited noise feature associated with the feature pair from the one or more limited correlation features.
14. The computing system of claim 9 , wherein generating the plurality of limited noise features comprises:
for each initial training data feature of the plurality of initial training data features, determining the per-feature mutual information measure;
determining a mutual information noise floor for the plurality of initial training data features; and
determining the limited noise subset based at least in part on a selected subset of the plurality of initial training data features whose respective per-feature mutual information measures exceed the mutual information noise floor.
15. The computing system of claim 14 , wherein determining the mutual information noise floor comprises:
identifying a maximum spurious feature ratio for the plurality of initial training data features;
determining a non-spurious feature quantile value for a determined distribution of each per-feature mutual information measure of a respective initial training data feature of the plurality of initial training data features based at least in part on the maximum spurious feature ratio; and
determining the mutual information noise floor based at least in part on the non-spurious feature quantile value.
16. The computing system of claim 15 , wherein the determined distribution is a binomial distribution.
17. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions configured to:
identify an initial training set comprising a plurality of training data entries associated with a plurality of initial training data features, wherein: (i) an initial training data entry of the plurality of training data entries is associated with a respective training data feature value for each of a subset of initial training data features of the plurality of initial training data features and a training target value for a training target feature, and (ii) each initial training data feature of the plurality of initial training data features is associated with a plurality of initial categorical levels;
generate, an adjusted training set based at least in part on the initial training set, wherein generating the adjusted training set comprises:
generating a plurality of limited noise features based at least in part on the plurality of initial training data features, wherein generating the plurality of limited noise features comprises:
determining a limited noise subset of the plurality of initial training data features based at least in part on a per-feature mutual information measure for each initial training data feature of the plurality of initial training data features satisfying a condition defined by a mutual information noise floor, and
generating the adjusted training set based at least in part on the plurality of limited noise features;
train a categorical input machine learning model based at least in part on the adjusted training set; and
generate, using the trained categorical input machine learning model, one or more inferred predictions based at least in part on a prediction input data entry.
18. The computer program product of claim 17 , wherein generating the plurality of limited noise features comprises:
for each initial training data feature of the plurality of initial training data features, determining the per-feature mutual information measure;
determining the mutual information noise floor for the plurality of initial training data features; and
determining the limited noise subset based at least in part on a selected subset of the plurality of initial training data features whose respective per-feature mutual information measures exceed the mutual information noise floor.
19. The computer program product of claim 18 , wherein determining the mutual information noise floor comprises:
identifying a maximum spurious feature ratio for the plurality of initial training data features;
determining a non-spurious feature quantile value for a determined distribution of each per-feature mutual information measure of a respective initial training data feature of the plurality of initial training data features based at least in part on the maximum spurious feature ratio; and
determining the mutual information noise floor based at least in part on the non-spurious feature quantile value.
20. The computer program product of claim 19 , wherein the determined distribution is a binomial distribution.