Machine learning models with efficient feature learning
A method can be used to predict risk using machine learning models having efficient feature learning. A risk prediction model can be applied to time-series data associated with a target entity to generate a risk indicator. The risk prediction model can include a feature learning model for generating features from the time-series data. The risk prediction model can also include a risk classification model for generating the risk indicator. The feature learning model can include filters and can be trained. Parameters of the risk prediction model can be adjusted to minimize a loss function associated with risk indicators. An updated risk prediction model can be generated by removing a filter from an original set of filters based on influencing scores of the original filters. The risk indicator can be transmitted to a computing device for use in controlling access of the target entity to a computing environment.
1 . A method that includes one or more processing devices performing operations comprising:
applying a risk prediction model to time-series data associated with a target entity to generate a risk indicator for the target entity, wherein the risk prediction model comprises a feature learning model configured to generate features from the time-series data and a risk classification model configured to generate the risk indicator based on the features generated by the feature learning model, the feature learning model comprising a plurality of filters, and wherein the risk prediction model is trained via a training process comprising:
adjusting parameters of the risk prediction model to minimize a loss function defined based on risk indicators generated for training time-series data and training risk indicators corresponding to the training time-series data; and
generating an updated risk prediction model by removing a filter from an original set of filters for the feature learning model based on influencing scores determined for the original set of filters, the original set of filters organized as blocks of filters and comprising the plurality of filters in the trained feature learning model, the influencing scores for the original set of filters determined based on the adjusted parameters, wherein removing the filter from the original set of filters further comprises determining an influencing score for a block of filters of the blocks of filters by computing a sum of the L-2 norm of weights applied to subsets of the features that are generated via the block of filters; and
transmitting, to a remote computing device, a responsive message including at least the risk indicator for use in controlling access of the target entity to one or more interactive computing environments.
2 . The method of claim 1 , wherein the parameters of the risk prediction model comprise weights applied to the features by the risk classification model, and the loss function comprises a term defined based on an L-1 norm of the weights.
3 . The method of claim 1 , wherein removing a filter from the original set of filters comprises:
determining an influencing score for each block of the blocks of filters; and
removing a block of filters based on the influencing scores for the blocks of filters.
4 . The method of claim 3 , wherein removing a filter from the plurality of filters further comprises:
determining an influencing score for each filter in one remaining block of filters; and
removing a filter from the one block of filters based on the influencing score or a metric calculated from the influencing score being lower than a threshold.
5 . The method of claim 4 , wherein the influencing score for a filter is calculated by computing an L-2 norm of weights applied to a subset of the features that is generated via the filter.
6 . The method of claim 3 , wherein the blocks of filters comprise a first block of filters with a first window size and a second block of filters with a second window size.
7 . The method of claim 6 , wherein the first block of filters and the second block of filters have a same number of filters.
8 . The method of claim 1 , wherein the training process further comprises:
adjusting parameters of the updated risk prediction model to minimize an updated loss function defined based on the risk indicators generated for the training time-series data and the training risk indicators corresponding to the training time-series data.
9 . A system comprising:
a processor; and
a non-transitory computer-readable medium comprising instructions that are executable by the processor to cause the processor to perform operations comprising:
applying a risk prediction model to time-series data associated with a target entity to generate a risk indicator for the target entity, wherein the risk prediction model comprises a feature learning model configured to generate features from the time-series data and a risk classification model configured to generate the risk indicator based on the features generated by the feature learning model, the feature learning model comprising a plurality of filters, and wherein the risk prediction model is trainable via a training process comprising:
adjusting parameters of the risk prediction model to minimize a loss function defined based on risk indicators generated for training time-series data and training risk indicators corresponding to the training time-series data; and
generating an updated risk prediction model by removing a filter from an original set of filters for the feature learning model based on influencing scores determined for the original set of filters, the original set of filters organized as blocks of filters and comprising the plurality of filters in the trained feature learning model, the influencing scores for the original set of filters determined based on the adjusted parameters, wherein removing the filter from the original set of filters further comprises determining an influencing score for a block of filters of the blocks of filters by computing a sum of the L-2 norm of weights applied to subsets of the features that are generated via the block of filters; and
transmitting, to a remote computing device, a responsive message including at least the risk indicator for use in controlling access of the target entity to one or more interactive computing environments.
10 . The system of claim 9 , wherein the parameters of the risk prediction model comprise weights applied to the features by the risk classification model, and the loss function comprises a term defined based on an L-1 norm of the weights.
11 . The system of claim 9 , wherein removing a filter from the original set of filters comprises:
determining an influencing score for each block of the blocks of filters; and
removing a block of filters based on the influencing scores for the blocks of filters.
12 . The system of claim 11 , wherein removing a filter from the plurality of filters further comprises:
determining an influencing score for each filter in one remaining block of filters, the influencing score determined by computing an L-2 norm of weights applied to a subset of the features that is generated via a particular filter; and
removing the particular filter from the one block of filters based on the influencing score or a metric calculated from the influencing score being lower than a threshold.
13 . The system of claim 11 , wherein the blocks of filters comprise a first block of filters with a first window size and a second block of filters with a second window size, and wherein the first block of filters and the second block of filters have a same number of filters.
14 . The system of claim 9 , wherein the training process further comprises:
adjusting parameters of the updated risk prediction model to minimize an updated loss function defined based on the risk indicators generated for the training time-series data and the training risk indicators corresponding to the training time-series data.
15 . A non-transitory computer-readable medium comprising instructions that are executable by a processing device for causing the processing device to perform operations comprising:
applying a risk prediction model to time-series data associated with a target entity to generate a risk indicator for the target entity, wherein the risk prediction model comprises a feature learning model configured to generate features from the time-series data and a risk classification model configured to generate the risk indicator based on the features generated by the feature learning model, the feature learning model comprising a plurality of filters, and wherein the risk prediction model is trainable via a training process comprising:
adjusting parameters of the risk prediction model to minimize a loss function defined based on risk indicators generated for training time-series data and training risk indicators corresponding to the training time-series data; and
generating an updated risk prediction model by removing a filter from an original set of filters for the feature learning model based on influencing scores determined for the original set of filters, the original set of filters organized as blocks of filters and comprising the plurality of filters in the trained feature learning model, the influencing scores for the original set of filters determined based on the adjusted parameters, wherein removing the filter from the original set of filters further comprises determining an influencing score for a block of filters of the blocks of filters by computing a sum of the L-2 norm of weights applied to subsets of the features that are generated via the block of filters; and
transmitting, to a remote computing device, a responsive message including at least the risk indicator for use in controlling access of the target entity to one or more interactive computing environments.
16 . The non-transitory computer-readable medium of claim 15 , wherein the parameters of the risk prediction model comprise weights applied to the features by the risk classification model, and the loss function comprises a term defined based on an L-1 norm of the weights.
17 . The non-transitory computer-readable medium of claim 15 , wherein removing a filter from the original set of filters comprises:
determining an influencing score for each block of the blocks of filters; and
removing a block of filters based on the influencing scores for the blocks of filters.
18 . The non-transitory computer-readable medium of claim 15 , wherein the training process further comprises:
adjusting parameters of the updated risk prediction model to minimize an updated loss function defined based on the risk indicators generated for the training time-series data and the training risk indicators corresponding to the training time-series data.