Landmark-based classification model updating
Systems and methods for updating a classification model of a neural network. The methods include selecting, as a set of landmarks, a limited number of data from a set of historical data used to train a classification model. Additionally, the methods generate new training data from recently collected data. Further, the methods update the classification model with the new training data and the set of landmarks to obtain an updated classification model having a loss function configured to capture similarities in the new training data and remember similarities in the historical data represented by the set of landmarks within a predefined tolerance.
1. A computer-implemented method for updating a classification model of a neural network, comprising:
selecting, as a set of landmarks, a limited number of data from a set of historical data used to train a classification model;
generating new training data from recently collected data; and
updating the classification model with the new training data and the set of landmarks to obtain an updated classification model having a loss function configured to capture similarities in the new training data and remember similarities in the historical data represented by the set of landmarks within a predefined tolerance, wherein a landmark component of the loss function (landmark loss ) for a data point (X) is determined as follows:
landmark loss ( X )= d ( f S ( X ), f T ( X )),
where d represents a Euclidean similarity distance d between a representation of the data point (X) in a historical mapping f s (X) and in a new mapping f T (X).
2. The method as in claim 1 , wherein the landmark component of the loss function includes Euclidean distances between representations of each landmark of the set of landmarks in the classification model and the updated classification model.
3. The method as in claim 1 , wherein selecting landmarks includes iteratively selecting landmarks.
4. The method as in claim 3 , wherein at each iterative step a probability of selecting a particular data segment as a landmark of the set of landmarks is proportional to its minimum squared distance to landmarks selected in previous iterative steps.
5. The method as in claim 1 , wherein the set of landmarks are divided into multiple subsets of landmarks, a different subset of landmarks being used for each epoch of the updating.
6. The method as in claim 1 , wherein the set of landmarks is used for each epoch of the updating.
7. The method as in claim 1 , wherein the classification model receives time series data from sensors deployed to monitor operations at a powerplant.
8. The method as in claim 1 , wherein the classification model receives time series data from one or more microphones coupled to a speech recognition system.
9. A neural network system comprising:
a non-transitory computer readable storage medium embodying computer readable instructions; and
a processor device configured to implement a classification model based on the computer readable instructions, the processor further configured to update the classification model by implementing:
a selection module configured to select, as a set of landmarks, a limited number of data from a set of historical data used to train the classification model; and
a model updating module configured to update the classification model with new training data and the set of landmarks to obtain an updated classification model having a loss function configured to capture similarities in the new training data and remember similarities in the historical data represented by the set of landmarks within a predefined tolerance, wherein a landmark component of the loss function (landmark loss ) for a data point (X) is determined as follows:
landmark loss ( X )= d ( f S ( X ), f T ( X )),
where d represents a Euclidean similarity distance d between a representation of the data point (X) in a historical mapping f x (X) and in a new f T (X).
10. The neural network system as in claim 9 , wherein the landmark component of the loss function includes Euclidean distances between representations of each landmark of the set of landmarks in the classification model and the updated classification model.
11. The neural network system as in claim 9 , wherein the selection module includes iteratively selecting landmarks.
12. The neural network system as in claim 11 , wherein at each iterative step a probability of selecting a particular data segment as a landmark of the set of landmarks is proportional to its minimum squared distance to landmarks selected in previous iterative steps.
13. The neural network system as in claim 9 , wherein the set of landmarks are divided into multiple subsets of landmarks, a different subset of landmarks being used for each epoch of the update.
14. The neural network system as in claim 9 , wherein the set of landmarks is used for each epoch of the update.
15. A non-transitory computer readable storage medium comprising a computer readable program for updating a classification model of a neural network, wherein the computer readable program when executed on a computer causes the computer to perform the method comprising:
selecting, as a set of landmarks, a limited number of data from a set of historical data used to train a classification model;
generating new training data from recently collected data; and
updating the classification model with the new training data to obtain an updated classification model having a loss function configured to capture similarities in the new training data and remember similarities in the historical data represented by the set of landmarks within a predefined tolerance, wherein a landmark component of the loss function (landmark loss ) for a data point (X) is determined as follows:
landmark loss ( X )= d ( f S ( X ), f T ( X )),
where d represents a :Euclidean similarity distance d between a representation of the data point (X) in a historical mapping f s (X) and in a new mapning f T (X).
16. The non-transitory computer readable storage medium as in claim 15 , wherein the landmark component of the loss function includes Euclidean distances between representations of each landmark of the set of landmarks in the classification model and the updated classification model.
17. The non-transitory computer readable storage medium as in claim 15 , wherein selecting landmarks includes iteratively selecting landmarks.
18. The non-transitory computer readable storage medium as in claim 17 , wherein at each iterative step a probability of selecting a particular data segment as a landmark of the set of landmarks is proportional to its minimum squared distance to landmarks selected in previous iterative steps.
19. The non-transitory computer readable storage medium as in claim 15 , wherein the set of landmarks are divided into multiple subsets of landmarks, a different subset of landmarks being used for each epoch of the updating.
20. The non-transitory computer readable storage medium as in claim 15 , wherein the set of landmarks is used for each epoch of the updating.