Automated input-data monitoring to dynamically adapt machine-learning techniques
Systems and methods are disclosed for triggering an update to a machine-learning model upon detecting that a distribution of particular (e.g., recently collected) input data set is sufficiently different from a distribution training input data set used to train the model. The distributions may be determined to be sufficiently different when a classifier can identify to which distribution individual data elements belong (e.g., to at least a predetermined degree). An update to the machine-learning model can include morphing weights used by the model and/or retraining the model.
1. A computer-implemented method comprising:
receiving an input data set;
generating, for each input data element in the input data set and for each machine-learning model of multiple machine-learning models, a classification output that corresponds to a prediction as to whether the input data element is a sample from a distribution of a training data set used to train the machine-learning model;
selecting one of the machine-learning-models of the multiple machine-learning models based on the classification outputs indicating a distribution of the input data set is similar to the distribution of the training data set; and
processing the input data set or another input data set received subsequent to a receipt of the input data set using the selected machine-learning model.
2. The method of claim 1 , wherein the input data set includes accelerometer data or a search query.
3. The method of claim 1 , wherein an architecture of the one of the machine-learning models is different than an architecture of another model of the multiple machine-learning models.
4. The method of claim 1 , wherein a value of a hyperparameter of the one of the machine-learning models is different than a value of the hyperparameter of another model of the multiple machine-learning models.
5. The method of claim 1 , wherein learned parameters of the one of the machine-learning models are different than learned parameters of another model of the multiple machine-learning models.
6. The method of claim 1 , wherein the one of the machine-learning models corresponds to a use circumstance that is different than a use circumstance for another model of the multiple machine-learning models.
7. The method of claim 1 , wherein the one of the machine-learning models is associated with a user profile that is different than a user profile associated with another model of the multiple machine-learning models.
8. The method of claim 1 , wherein the one of the machine-learning models is associated with a type of movement that is different than a type of movement associated with another model of the multiple machine-learning models.
9. The method of claim 1 , wherein the prediction indicates that the input data element is the sample from the distribution of the training data set used to train the machine-learning model, and the method further comprises:
augmenting the training data set used to train the machine-learning model with the input data element; and
training the selected machine-learning model with the augmented training data set.
10. A system comprising:
one or more data processors; and
a non-transitory computer readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to perform a set of actions including:
receiving an input data set;
generating, for each input data element in the input data set and for each machine-learning model of multiple machine-learning models, a classification output that corresponds to a prediction as to whether the input data element is a sample from a distribution of a training data set used to train the machine-learning model;
selecting one of the machine-learning-models of the multiple machine-learning models based on the classification outputs indicating a distribution of the input data set is similar to the distribution of the training data set; and
processing the input data set or another input data set received subsequent to a receipt of the input data set using the selected machine-learning model.
11. The system of claim 10 , wherein the input data set includes accelerometer data or a search query.
12. The system of claim 10 , wherein an architecture of the one of the machine-learning models is different than an architecture of another model of the multiple machine-learning models.
13. The system of claim 10 , wherein a value of a hyperparameter of the one of the machine-learning models is different than a value of the hyperparameter of another model of the multiple machine-learning models.
14. The system of claim 10 , wherein learned parameters of the one of the machine-learning models are different than learned parameters of another model of the multiple machine-learning models.
15. The system of claim 10 , wherein the one of the machine-learning models corresponds to a use circumstance that is different than a use circumstance for another model of the multiple machine-learning models.
16. The system of claim 10 , wherein the one of the machine-learning models is associated with a user profile that is different than a user profile associated with another model of the multiple machine-learning models.
17. The system of claim 10 , wherein the one of the machine-learning models is associated with a type of movement that is different than a type of movement associated with another model of the multiple machine-learning models.
18. The system of claim 10 , wherein the prediction indicates that the input data element is the sample from the distribution of the training data set used to train the machine-learning model, and the instructions, when executed, cause the one or more data processors to perform the set of actions including:
augmenting the training data set used to train the machine-learning model with the input data element; and
training the selected machine-learning model with the augmented training data set.
19. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause one or more data processors to perform a set of actions including:
receiving an input data set;
generating, for each input data element in the input data set and for each of multiple machine-learning models, a classification output that corresponds to a prediction as to whether the input data element is a sample from a distribution of a training data set used to train the machine-learning model;
selecting one of the machine-learning-models of the multiple machine-learning models based on the classification outputs indicating a distribution of the input data set is similar to the distribution of the training data set; and
processing the input data set or another input data set received subsequent to a receipt of the input data set using the selected machine-learning model.
20. The computer-program product of claim 19 , wherein the machine-learning model is a first machine-learning model, and the classification output is generated using a second machine-learning model.