IP Library › Granted Patent US 12694532
Granted Patent B1
US 12694532 · App. 17/407,627 · Granted Jul 28, 2026

Training strategy search using reinforcement learning

Inventors: Dong Yang (North Bethesda, MD); Holger Reinhard Roth (Rockville, MD); Ziyue Xu (Reston, VA); Fausto Milletari (Munich, DE); Ling Zhang (Rockville, MD); Te-Chung Isaac Yang (San Ramon, CA); Daguang Xu (Potomac, MD)
Assignee: NVIDIA CORPORATION
G06T7/10G06N3/045G06N3/048G06T5/70G06T7/0012
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12694532
App. No.
17/407,627
Granted
Jul 28, 2026
Kind
B1
Abstract

In at least one embodiment, a reinforcement-learning-based searching approach is used to produce a training configuration for a machine-learning model. In at least one embodiment, 3D medical image segmentation is performed using learned image preprocessing parameters.

Claims (85)

1 . One or more processors, comprising:

circuitry to use one or more first neural networks to:

generate one or more hyperparameters based on applying one or more image augmentations to one or more images of one or more training data sets used to train one or more second neural networks;

receive feedback information corresponding to one or more first training iterations of the one or more second neural networks using the one or more hyperparameters;

generate one or more updated hyperparameters based, at least in part, on the feedback information, wherein the one or more updated hyperparameters are updated based on an augmentation type;

generate one or more updated training data sets based, at least in part, on the one or more updated hyperparameters;

provide the one or more updated training data sets to the one or more second neural networks to cause the one or more second neural networks to generate one or more new images; and

cause at least one neural network of the one or more first neural networks or at least one neural network of the one or more second neural networks to infer segmentation information for one or more objects depicted by the one or more new images.

2 . The one or more processors of claim 1 , wherein the circuitry is further to:

apply the one or more updated hyperparameters to the one or more second neural networks;

cause the one or more second neural networks to generate the one or more images based at least in part on the one or more updated hyperparameters; and

train the one or more second neural networks based at least in part on the updated training data sets.

3 . The one or more processors of claim 1 , wherein the circuitry is further to train the one or more second neural networks by at least calculating a validation accuracy of the one or more second neural networks.

4 . The one or more processors of claim 1 , wherein the one or more updated hyperparameters include a number of nodes.

5 . The one or more processors of claim 1 , wherein the one or more images are based on one or more medical images.

6 . The one or more processors of claim 1 , wherein the one or more first neural networks include a recurrent neural network (RNN).

7 . A system, comprising:

one or more processors to use one or more first neural networks to:

generate one or more hyperparameters based on applying one or more image augmentations to one or more images of one or more update training data sets used to train one or more second neural networks;

receive feedback information corresponding to one or more first training iterations of the one or more second neural networks using the one or more hyperparameters;

generate one or more updated hyperparameters based, at least in part, on the feedback information, wherein the one or more updated hyperparameters are updated based on an augmentation type;

generate one or more updated training data sets based, at least in part, on the one or more updated hyperparameters;

provide the one or more updated training data sets to the one or more second neural networks to cause the one or more second neural networks to generate one or more new images; and

cause at least one neural network of the one or more first neural networks or at least one neural network of the one or more second neural networks to infer segmentation information for one or more objects depicted by the one or more new images.

8 . The system of claim 7 , wherein the one or more processors are further to:

use the one or more second neural networks to generate the one or more images based at least in part on the one or more updated hyperparameters;

calculate an accuracy of the one or more second neural networks based on the one or more images and the updated training data sets; and

calculate whether the accuracy exceeds a threshold value.

9 . The system of claim 8 , wherein the one or more processors are further to, as a result of calculating that the accuracy of the one or more second neural networks does not exceed the threshold value, cause the one or more first neural networks to generate one or more updated hyperparameters.

10 . The system of claim 7 , wherein the one or more second neural networks include a deep neural network (DNN).

11 . The system of claim 7 , wherein the one or more updated hyperparameters include a number of layers.

12 . The system of claim 7 , wherein the one or more images include one or more three-dimensional (3D) images.

13 . A non-transitory machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to use one or more first neural networks to:

generate one or more hyperparameters based on applying one or more image augmentations to one or more images of one or more training data sets used to train one or more second neural networks;

receive feedback information corresponding to one or more first training iterations of the one or more second neural networks using the one or more hyperparameters;

generate one or more updated hyperparameters based, at least in part, on the feedback information, wherein the one or more updated hyperparameters are updated based on an augmentation type;

generate one or more updated training data sets based, at least in part, on the one or more updated hyperparameters;

provide the one or more updated training data sets to the one or more second neural networks to cause the one or more second neural networks to generate one or more new images; and

cause at least one neural network of the one or more first neural networks or at least one neural network of the one or more second neural networks to infer segmentation information for one or more objects depicted by the one or more new images.

14 . The non-transitory machine-readable medium of claim 13 , wherein the set of instructions further include instructions, which if performed by the one or more processors, cause the one or more processors to:

generate the one or more images using the one or more second neural networks and the one or more updated hyperparameters; and

train the one or more second neural networks by at least calculating an accuracy value based at least in part on the updated training data sets.

15 . The non-transitory machine-readable medium of claim 13 , wherein the set of instructions further include instructions, which if performed by the one or more processors, cause the one or more processors to train the one or more second neural networks based at least in part on a convergence rate associated with the one or more second neural networks.

16 . The non-transitory machine-readable medium of claim 13 , wherein the set of instructions further include instructions, which if performed by the one or more processors, cause the one or more processors to use the one or more first neural networks to generate the one or more updated hyperparameters based on one or more defined ranges of values.

17 . The non-transitory machine-readable medium of claim 13 , wherein the one or more first neural networks include a reinforcement learning (RL) network.

18 . The non-transitory machine-readable medium of claim 13 , wherein the one or more updated hyperparameters include structural parameters of the one or more second neural networks.

19 . A processor, comprising:

circuitry to train one or more first neural networks to:

generate one or more hyperparameters based on applying one or more image augmentations to one or more images of one or more update training data sets used to train one or more second neural networks;

receive feedback information corresponding to one or more first training iterations of the one or more second neural networks using the one or more hyperparameters;

generate one or more updated hyperparameters based, at least in part, on the feedback information, wherein the one or more updated hyperparameters are updated based on an augmentation type;

generate one or more updated training data sets based, at least in part, on the one or more updated hyperparameters;

provide the one or more updated training data sets to the one or more second neural networks to cause the one or more second neural networks to generate one or more new images; and

cause at least one neural network of the one or more first neural networks or at least one neural network of the one or more second neural networks to infer segmentation information for one or more objects depicted by the one or more new images.

20 . The processor of claim 19 , wherein the circuitry is further to:

calculate one or more accuracy values based at least in part on the one or more images and the updated training data sets; and

update one or more weights of the one or more first neural networks based on the one or more accuracy values.

21 . The processor of claim 20 , wherein the circuitry is further to:

use the one or more first neural networks to generate one or more updated hyperparameters using one or more updated weights; and

cause the one or more second neural networks to generate one or more images based on the one or more updated hyperparameters.

22 . The processor of claim 19 , wherein the one or more images include one or more segmented three-dimensional (3D) medical images.

23 . The processor of claim 19 , wherein the one or more updated hyperparameters indicate one or more image processing operations.

24 . The processor of claim 19 , wherein the one or more second neural networks include a fully convolutional neural network (FCN).

25 . A method, comprising:

training one or more first neural networks to:

generate one or more hyperparameters based on applying one or more image augmentations to one or more images of one or more update training data sets used to train one or more second neural networks;

receive feedback information corresponding to one or more first training iterations of the one or more second neural networks using the one or more hyperparameters;

generate one or more updated hyperparameters based, at least in part, on the feedback information, wherein the one or more updated hyperparameters are updated based on an augmentation type;

generate one or more updated training data sets based, at least in part, on the one or more updated hyperparameters;

provide the one or more updated training data sets to the one or more second neural networks to cause the one or more second neural networks to generate one or more new images; and

cause at least one neural network of the one or more first neural networks or at least one neural network of the one or more second neural networks to infer segmentation information for one or more objects depicted by the one or more new images.

26 . The method of claim 25 , further comprising:

obtaining one or more training data sets of the one or more second neural networks;

causing the one or more second neural networks to process the one or more training data sets using the one or more updated hyperparameters to generate the one or more images; and

training the one or more first neural networks using one or more back-propagation processes based on a reward calculated from the one or more second neural networks.

27 . The method of claim 26 , wherein the reward is a validation accuracy calculated based on the updated training data sets and the one or more images.

28 . The method of claim 25 , further comprising training the one or more first neural networks to generate the one or more updated hyperparameters through a continuous search space.

29 . The method of claim 25 , wherein the one or more first neural networks include state information.

30 . The method of claim 25 , wherein the one or more updated hyperparameters include at least one of an image sharpening parameter, an image smoothing parameter, a Gaussian noise parameter, a contrast adjustment parameter, or a random shift of intensity range parameter.

31 . The one or more processors of claim 1 , wherein the circuitry is further to use at least one neural network of the one or more first neural networks or one or more second neural networks to generate inferences for the generated one or more new images.

32 . The system of claim 7 , wherein the one or more processors are further to use at least one neural network of the one or more first neural networks or one or more second neural networks to generate inferences for the generated one or more new images.

33 . The non-transitory machine-readable medium of claim 13 , wherein the set of instructions further include instructions, which if performed by the one or more processors, cause the one or more processors to:

use at least one neural network of the one or more first neural networks or one or more second neural networks to generate inferences for the generated one or more new images.

34 . The processor of claim 19 , wherein the circuitry is further to use at least one neural network of the one or more first neural networks or one or more second neural networks to generate inferences for the generated one or more new images.

35 . The method of claim 25 , further comprising using at least one neural network of the one or more first neural networks or one or more second neural networks to generate inferences for the generated one or more new images.