Training strategy search using reinforcement learning
In at least one embodiment, a reinforcement-learning-based searching approach is used to produce a training configuration for a machine-learning model. In at least one embodiment, 3D medical image segmentation is performed using learned image preprocessing parameters.
1. A system comprising: one or more processors to perform image segmentation based, at least in part, on one or more first neural networks including weight parameters based, at least in part, on one or more hyperparameter values generated by one or more second neural networks, wherein the one or more hyperparameter values are generated based at least in part on output from the one or more first neural networks.
2. The system of claim 1 , wherein the one or more hyperparameter values are data augmentation parameter values for inputs to the one or more first neural networks.
3. The system of claim 1 , wherein:
the one or more first neural networks take a medical image as input;
the one or more first neural networks produce a medical image segmentation of the medical image; and
the hyperparameter values identify one or more image-processing operations to be performed to the medical image.
4. The system of claim 3 , wherein the hyperparameter values include an image-sharpening parameter value, an image-smoothing parameter value, a Gaussian-noise parameter value, a contrast-adjustment parameter value, or a random-shift-of-intensity-range parameter value.
5. The system of claim 1 , wherein the one or more second neural networks are trained using a reward that is based at least in part on accuracy of the one or more first neural networks.
6. The system of claim 1 , wherein:
the one or more second neural networks is trained using a reward; and
the reward is based at least in part on a rate of convergence of training the one or more first neural networks.
7. The system of claim 1 , wherein the one or more second neural networks includes a recurrent neural network.
8. A processor comprising: one or more arithmetic logic units (ALUs) to help train one or more neural networks to perform image segmentation, at least in part, by using a first recurrent neural network to generate a set of hyperparameter values for a second neural network, wherein the first recurrent neural network is trained based on output from the second neural network and further wherein the set of hyperparameter values are generated based at least in part on the output.
9. The processor of claim 8 , wherein the one or more ALUs further:
update the first recurrent neural network based on accuracy of a value output from the second neural network;
update the set of hyperparameter values using the updated first recurrent neural network; and
train the second neural network using the updated set of hyperparameter values.
10. The processor of claim 9 , wherein:
the set of hyperparameter values include data augmentation parameter values; and
the data augmentation parameter values control pre-processing of image data provided to the second neural network.
11. The processor of claim 8 , wherein:
the set of hyperparameter values include data augmentation parameter values; and
the data augmentation parameter values control pre-processing of image data provided to the second neural network.
12. The processor of claim 11 , wherein:
the second neural network processes three dimensional medical image data; and
the second neural network produces a segmentation of the three dimensional image data.
13. The processor of claim 8 , wherein:
a parameter value in the set of hyperparameter values is learned, at least in part, by specifying an allowable range for the parameter value.
14. The processor of claim 8 , wherein:
the second neural network is trained using a reward; and
the reward is a validation accuracy of the second neural network.
15. A method comprising performing image segmentation based, at least in part, on one or more first neural networks including weight parameters based, at least in part, on one or more hyperparameter values generated by one or more second neural networks, wherein the one or more hyperparameter values are generated based at least in part on output from the one or more first neural networks.
16. The method of claim 15 , wherein the one or more first neural networks are trained using a reward that is a measure of a validation accuracy of the one or more second neural networks.
17. The method of claim 16 , further comprising:
comparing the validation accuracy to a threshold value;
retraining the one or more second neural networks as a result of the comparison;
generating an updated set of hyperparameter values using the one or more first neural networks; and
retraining the one or more first neural networks using the updated hyperparameter values.
18. The method of claim 15 , wherein the one or more first neural networks are trained using a reward that is a convergence rate of the one or more second neural networks.
19. The method of claim 15 , wherein:
each parameter value in the one or more hyperparameter values has an associated allowable range; and
individual parameter values in the one or more hyperparameter values are initialized to a random value in the associated allowable range.
20. The method of claim 15 , wherein:
the one or more first neural networks determines a set of training strategies for training the one or more second neural networks; and
the set of training strategies includes one or more of a data augmentation technique, a learning rate, or a data pre-processing technique.
21. The method of claim 20 , wherein the data augmentation technique is image sharpening, image smoothing, adding Gaussian noise, contrast adjustment, or random shift of intensity range.
22. A method comprising performing image segmentation based, at least in part, on determining a set of hyperparameter values for a first neural network by using a second neural network to adjust one or more weights of the second neural network based at least in part on an output of the first neural network, wherein the set of hyperparameter values are determined based at least in part on the output.
23. The method of claim 22 , wherein:
an image is processed to produce a processed image in accordance with the set of hyperparameter values; and
the processed image is provided to the first neural network.
24. The method of claim 23 , wherein the set of hyperparameter values specify that image sharpening, image smoothing, addition of Gaussian noise, contrast adjustment, or a random shift of intensity range is to be applied to the image.
25. The method of claim 22 , wherein the second network is trained to improve accuracy of the first neural network.
26. The method of claim 25 , wherein accuracy of the first neural network is based, at least in part, on a training loss of the first neural network.
27. The method of claim 25 , wherein accuracy of the first neural network is based, at least in part on a validation loss of the first neural network.
28. The method of claim 22 , wherein:
each parameter value in the set of hyperparameter values has an associated allowable range; and
each parameter value in the set of hyperparameter values is initialized to a value in the associated allowable range.
29. The system of claim 1 , wherein the one or more hyperparameter values identify one or more image-processing operations to be performed by the one or more first neural networks.