Reset systems and methods for continuous adaptation without forgetting
An autonomous system includes: a first semantic segmentation model trained based on a training dataset including images and labels for the images, the first semantic segmentation model configured to generate a first segmentation map based on an image from a camera; a second semantic segmentation model of the same type of semantic segmentation model as the first semantic segmentation model, the second semantic segmentation model configured to generate a second segmentation map based on the image from the camera; an adaptation module configured to selectively adjust one or more first parameters of the second semantic segmentation model; and a reset module configured to: determine a first total number of unique classifications included in the first segmentation map; determine a second total number of unique classifications included in the first segmentation map; and selectively reset the first parameters to previous parameters, respectively, based on the first and second total numbers.
1. An autonomous system, comprising:
a first semantic segmentation model trained based on a training dataset including images and labels for the images, the first semantic segmentation model configured to generate a first segmentation map based on an image from a camera of the autonomous system;
a second semantic segmentation model of the same type of semantic segmentation model as the first semantic segmentation model, the second semantic segmentation model configured to generate a second segmentation map based on the image from the camera of the autonomous system;
an adaptation module configured to selectively adjust one or more first parameters of the second semantic segmentation model; and
a reset module configured to:
determine a first total number of unique classifications included in the first segmentation map;
determine a second total number of unique classifications included in the second segmentation map; and
selectively reset the first parameters of the second semantic segmentation model to previous parameters, respectively, based on the first and second total numbers of unique classifications.
2. The autonomous system of claim 1 wherein the reset module is configured to reset the first parameters of the second semantic segmentation model to the previous parameters, respectively, when a difference between the first and second total numbers of unique classifications is greater than a predetermined value.
3. The autonomous system of claim 1 wherein the first parameters of the second semantic segmentation model are initialized to second parameters of the first semantic segmentation model, respectively.
4. The autonomous system of claim 1 wherein the previous parameters are second parameters of the first semantic segmentation model, respectively.
5. The autonomous system of claim 4 wherein the second parameters of the first semantic segmentation model are not changed after the training.
6. The autonomous system of claim 1 wherein the adaptation module is configured to selectively adjust one or more of the first parameters of the second semantic segmentation model based on at least one of: (a) the image and (b) the second segmentation map.
7. The autonomous system of claim 1 wherein the first and second semantic segmentation models each include a DeepLab model.
8. The autonomous system of claim 1 wherein the reset module is further configured to:
determine first weights of a portion of the first semantic segmentation model;
determine second weights of the portion of the second semantic segmentation model; and
selectively reset the first parameters of the second semantic segmentation model to the previous parameters, respectively, based on a comparison of one of the first weights with a respective one of the second weights.
9. The autonomous system of claim 8 wherein the reset module is configured to reset the first parameters of the second semantic segmentation model to the previous parameters, respectively, when a difference between (a) the one of the first weights and (b) the respective one of the second weights is greater than a predetermined value.
10. The autonomous system of claim 8 wherein the portion of the first and second semantic segmentation models is a final layer of a decoder of the first and second semantic segmentation models.
11. The autonomous system of claim 1 wherein the previous parameters include previous parameters, respectively, of the second semantic segmentation model.
12. The autonomous system of claim 1 further comprising a control module configured to selectively actuate an actuator of the autonomous system based on the second segmentation map.
13. An autonomous system, comprising:
a first semantic segmentation model trained based on a training dataset including images and labels for the images, the first semantic segmentation model configured to generate a first segmentation map based on an image from a camera of the autonomous system;
a second semantic segmentation model of the same type of semantic segmentation model as the first semantic segmentation model, the second semantic segmentation model configured to generate a second segmentation map based on the image from the camera of the autonomous system;
an adaptation module configured to selectively adjust one or more first parameters of the second semantic segmentation model; and
a reset module configured to:
determine first weights of a portion of the first semantic segmentation model;
determine second weights of the portion of the second semantic segmentation model; and
selectively reset the first parameters of the second semantic segmentation model to previous parameters, respectively, based on a comparison of one of the first weights with a respective one of the second weights.
14. The autonomous system of claim 13 wherein the reset module is configured to reset the first parameters of the second semantic segmentation model to the previous parameters, respectively, when a difference between (a) the one of the first weights and (b) the respective one of the second weights is greater than a predetermined value.
15. The autonomous system of claim 13 wherein the previous parameters include previous parameters, respectively, of the second semantic segmentation model.
16. The autonomous system of claim 13 wherein the first parameters of the second semantic segmentation model are initialized to second parameters of the first semantic segmentation model, respectively.
17. The autonomous system of claim 13 wherein the previous parameters are second parameters of the first semantic segmentation model, respectively.
18. The autonomous system of claim 13 wherein the adaptation module is configured to selectively adjust one or more of the first parameters of the second semantic segmentation model based on at least one of: (a) the image and (b) the second segmentation map.
19. The autonomous system of claim 13 further comprising a control module configured to selectively actuate an actuator of the autonomous system based on the second segmentation map.
20. An autonomous system, comprising:
a first model trained based on a training dataset including images and labels for the images, the first model configured to generate a first output based on an image from a camera of the autonomous system;
a second model of the same type of model as the first model, the second model configured to generate a second output based on the image from the camera of the autonomous system;
an adaptation module configured to selectively adjust one or more first parameters of the second model; and
a reset module configured to:
determine first weights of a portion of the first model;
determine second weights of the portion of the second model; and
selectively reset the first parameters of the second model to previous parameters, respectively, based on a comparison of one of the first weights with a respective one of the second weights.
21. An autonomous system, comprising:
a camera configured to record input images;
a sensor module configured to detect one or more environmental changes external to the autonomous system;
a segmentation module configured to receive the input images from the camera and to segment objects in the input images using a neural network model;
an adaptation module configured to selectively adjust one or more parameters of the neural network model utilized by the segmentation module from an input set of parameters to define an adapted set of parameters; the input set of parameters being defined by an initial set or parameters or a previously adapted set of parameters; the initial set of parameters of the neural network model being trained on a dataset including images and labels for the images;
memory configured to store a plurality of parameter checkpoints of the neural network model; the initial set of parameters and each adapted set of parameters stored in the memory defining a different parameter checkpoint; each parameter checkpoint being associated with a different set of parameters of a semantic segmentation model; and
a reset module configured to selectively reset the neural network model utilized to a parameter checkpoint stored in the memory when the reset module detects the parameters of the neural network model selectively adjusted by the adaptation module to be utilized by the segmentation module exceed a criteria dynamically determined by the reset module.
22. The autonomous system of claim 21 , wherein the criteria determined by the reset module is based on in whole or in part on one or more environmental changes detected by the sensor module.
23. The autonomous system of claim 22 , wherein the parameter checkpoint selected by the reset module to replace the neural network model selectively adjusted by the adaptation module is determined based in whole or in part on the one or more environmental changes detected by the environment sensor.
24. The autonomous system of claim 23 , wherein the sensor module includes one or more of a location sensor, a temperature sensor, a water sensor and a light sensor.