Granted Patent B2

US 12,423,583 · App. 16/889,670 · Granted Sep 23, 2025

Inventors: Xiaosong Wang (Rockville, MD); Ziyue Xu (Reston, VA); Dong Yang (Pocatello, ID); Lickkong Tam (Santa Clara, CA); Daguang Xu (Potomac, MD)

Assignee: NVIDIA Corporation

G06N3/084G06F18/214G06F18/24G06F40/20G06N3/047

Abstract

Claims (99)

1. One or more processors comprising circuitry to:

use one or more neural networks to:

obtain data as input to train the one or more neural networks;

generate a set of weights for the one or more neural networks by processing a portion of the data;

compute, using the set of weights, a set of features representing characteristics corresponding to a label set of the portion;

compute weights for one or more labels from the label set using the set of features; and

select at least one label from the one or more labels based on the weights to update the one or more neural networks; and

cause the one or more labels to be compared with one or more objects within one or more images identified by the one or more neural networks.

2. The one or more processors of claim 1 , wherein:

the data is a batch of data comprising a portion of a set of labels and a plurality of training images;

the set of weights is generated by processing the batch;

the circuitry is to further compute a weighted average, using the weights for each label of the portion of the set of labels; and

the one or more labels are selected, based on the weighted average for the portion of the set of labels, to update the one or more neural networks.

3. The one or more processors of claim 2 , wherein the weights are computed through one or more fully-connected layers and one or more activation functions.

4. The one or more processors of claim 1 , wherein the one or more labels are generated by different algorithm-based labelers.

5. The one or more processors of claim 1 , where the circuitry is further to:

perform a comparison of an output of the one or more neural networks being trained with labels with the one or more labels; and

select a label based at least in part on the comparison.

6. The one or more processors of claim 5 , wherein the output is compared through one or more binary cross entropy (BCE) loss functions.

7. The one or more processors of claim 1 , wherein the one or more neural networks uses the selected one or more labels and a training image to perform one or more image classification tasks.

8. The one or more processors claim 1 , wherein the one or more neural networks select the one or more labels from a plurality of labels.

9. A system, comprising one or more processors to:

use one or more neural networks to:

receive a portion of training data comprising a set of labels available for an image;

generate a set of weights for the one or more neural networks by processing the portion;

compute, using the set of weights, features representing characteristics corresponding to the set of labels;

use the features to compute weighted averages for the set of labels; and

select one or more labels from the set of labels based on the weighted averages; and

cause the selected one or more labels to be compared with one or more objects within one or more images identified by the one or more neural networks.

10. The system of claim 9 , wherein:

the image is from a plurality of images associated with the one or more labels; and

using the one or more neural networks further comprises the one or more processors to:

provide the plurality of images to the one or more neural networks; and

select the one or more labels for the image of the plurality of images to train the one or more neural networks.

11. The system of claim 9 , wherein the set of weights for the one or more neural networks are generated through one or more back-propagation processes.

12. The system of claim 9 , wherein one or more differentiable binarization processes are performed on the weighted averages.

13. The system of claim 9 , wherein the one or more labels are generated by different natural language processing (NLP) algorithms executed by different computing resources.

14. A non-transitory computer-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to:

use one or more neural networks to:

receive data as input to train the one or more neural networks;

generate a set of model weights for the one or more neural networks by

processing a portion of the data, wherein the portion of the data comprises labels available for a training image;

compute, based on the set of model weights, a set of features representing characteristics corresponding to a label set of the portion;

use the set of features to compute weights for one or more labels from the label set;

select at least one label from the one or more labels from the label set based on the weights to update parameters of the one or more neural networks; and

cause the selected one or more labels to be compared with one or more objects within one or more images identified by the one or more neural networks.

15. The non-transitory computer-readable medium of claim 14 , wherein using the one or more neural networks further causes the one or more processors to:

perform a comparison of with an output of the one or more neural networks with the label set; and

select the one or more labels based in part on the comparison.

16. The non-transitory computer-readable medium of claim 15 , wherein the comparison is performed using one or more loss functions.

17. The non-transitory computer-readable medium of claim 14 , wherein the labels available for the training image are generated by one or more algorithm-based labelers.

18. The non-transitory computer-readable medium of claim 14 , wherein the selected one or more labels and the training image train the one or more neural networks to perform one or more multi-label classification tasks.

19. The non-transitory computer-readable medium of claim 14 , wherein the set of model weights are generated through one or more back-propagation processes.

20. One or more processors processor, comprising circuitry to use a neural network to infer information, wherein the neural network is trained by:

selecting one or more labels at least by:

obtaining data as input to train the neural network;

generating a set of weights for the neural network by processing a portion of the data;

computing, using the set of weights, a set of features representing characteristics corresponding to a label set of the portion;

computing weights for one or more labels from the label set using the set of features; and

selecting at least one label from the one or more labels based on the weights; and

comparing the selected one or more labels with one or more objects within one or more images identified by the neural network while training the neural network.

21. The one or more processors of claim 20 , wherein:

the data is a batch from training data, received as input to the neural network, comprising labels available for a training image;

the weights are model weights for the neural network generated by processing the batch;

the set of features are computed using the model weights, the set of features representing characteristics of the labels available for the training image;

the circuitry is further to use the features to compute weighted averages for labels available for the training image;

the one or more labels are selected from the labels available for the training image based on the weighted averages; and

the circuitry is further to update parameters of the neural network using the selected one or more labels to perform image segmentation tasks.

22. The one or more processors of claim 21 , wherein the training data comprises a plurality of training images, wherein each of the plurality of training images is accompanied with a text report.

23. The one or more processors of claim 22 , wherein the circuitry is further to:

generate text embedding based on the text report; and

use the text embedding to identify the labels available for the training image.

24. The one or more processors of claim 20 , wherein the neural network is trained to select the one or more labels based, at least in part, on weighted averages generated for the one or more labels.

25. The one or more processors of claim 20 , wherein the neural network is trained by selecting the one or more labels to be used while simultaneously training the neural network to classify training images and associated labels.

26. A system, comprising one or more processors to use a neural network to infer information, wherein the neural network is trained by:

selecting one or more labels by at least:

receiving data as input to the neural network;

generating a set of weights for the neural network by processing a subset of the data, wherein the subset of the data comprises labels available for a training image;

computing, based on the set of weights, a set of features representing characteristics corresponding to a label set of the subset;

using the set of features to compute weights for one or more labels from the subset;

updating a component of the neural network using a portion of the labels from the label set based on the computed weights for the one or more labels; and

selecting the one or more labels based at least in part on the component; and

causing the selected one or more labels to be compared with one or more objects within one or more images identified by the neural network while training the neural network.

27. The system of claim 26 , wherein the one or more processors are to further train the neural network to select the one or more labels while simultaneously training the neural network to classify training images.

28. The system of claim 26 , wherein the labels available for the training image are generated by one or more natural-language processing (NLP) algorithms performed by different processors.

29. The system of claim 26 , wherein the portion of the labels and the training image are used by the neural network to perform one or more image classification tasks.

30. A non-transitory computer-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to use a neural network to infer information, wherein the neural network is trained by at least:

receiving data as input to the neural network;

generating a set of weights for the neural network by processing a subset of the data, wherein the subset of the data comprises labels available for a training image;

computing, based on the set of weights, a set of features representing characteristics corresponding to a label set of the subset;

using the set of features to compute weights for one or more labels from the subset;

selecting at least one label from the one or more labels based at least in part on the weights; and

causing the selected one or more labels to be compared with one or more objects within one or more images identified by the neural network while training the neural network.

31. The non-transitory computer-readable medium of claim 30 , wherein training the neural network further causes the one or more processors to:

use training data, received as input to the neural network, comprising a plurality of labels available for a plurality of training images to determine labels for a training image; and

use the labels for the training image to update parameters of the neural network.

32. The non-transitory computer-readable medium of claim 31 , wherein parameters of the neural network is updated based on results from performing a comparison of the labels for the training image with an output of the neural network.

33. The non-transitory computer-readable medium of claim 31 , wherein the training data comprising a plurality of labels for a plurality of images is extracted from at least one text report accompanying each training image of the plurality of images.

34. The non-transitory computer-readable medium of claim 33 , wherein the at least one text report is generated by one or more natural-language processing (NLP) algorithms performed by different computing resources.

Assignments (1)

Continuity (1)

References Cited (36)