Generating labels for synthetic images using one or more neural networks
Apparatuses, systems, and techniques to determine pixel-level labels of a synthetic image. In at least one embodiment, the synthetic image is generated by one or more generative networks and the pixel-level labels are generated using a combination of data output by a plurality of layers of the generative networks.
1 . One or more processors, comprising: circuitry to:
cause one or more neural networks to generate one or more feature maps during a generation of one or more images;
generate a representation of the one or more images by adding one or more channels of the one or more generated images with one or more channels of the one or more feature maps obtained from one or more layers of the one or more neural networks; and
input the representation of the one or more images to the one or more neural networks to cause the one or more neural networks to generate labels of one or more objects within the one or more images.
2 . The one or more processors of claim 1 , wherein each feature map of the one or more feature maps corresponds to a stage of generation of the one or more images depicting the one or more objects by the one or more neural networks, the one or more feature maps providing information to be used by the one or more neural networks to generate the labels of the one or more objects.
3 . The one or more processors of claim 1 , wherein the one or more feature maps are generated by the one or more neural networks that comprise one or more generative adversarial networks (GANs), wherein the one or more images comprise one or more synthetic images generated by a GAN of the one or more GANs.
4 . The one or more processors of claim 3 , wherein the one or more feature maps are generated by a plurality of layers of the GAN.
5 . The one or more processors of claim 3 , wherein the circuitry is further to:
use the GAN to generate the one or more images wherein the labels are usable to train an additional machine learning model to perform pixel-level segmentation of images.
6 . The one or more processors of claim 1 , wherein the circuitry is further to:
extract the one or more feature maps from intermediate layers of a generative network of the one or more neural networks, the one or more feature maps generated by the generative network during generation of the one or more images;
for each feature map of the one or more feature maps, resize the respective feature map to a certain resolution; and
concatenate data from the one or more feature maps and at least one of the one or more images to generate a combined feature map, the combined feature map having the one or more channels of the one or more feature maps.
7 . The one or more processors of claim 6 , wherein the circuitry is further to:
input the combined feature map to the one or more neural networks to perform pixel-level classification of pixels of the combined feature map;
determine, for each pixel in the combined feature map and using the one or more channels of the one or more feature maps, a classification associated with the respective pixel, wherein the classification is one of a plurality of classifications associated with the combined feature map; and
generate a mask for the at least one of the one or more images, wherein each entry in the mask is associated with a specific pixel in the at least one of the one or more images and indicates for the specific pixel an association between the specific pixel and a classification of the plurality of classifications.
8 . The one or more processors of claim 1 , wherein the one or more images are generated based, at least in part, on the one or more feature maps.
9 . One or more processors, comprising: circuitry to cause one or more neural networks to perform pixel-level labeling of synthetic images generated by a generative network, the circuitry to:
cause the one or more neural networks to generate one or more feature maps during a generation of one or more images;
generate a representation of the one or more images by adding one or more channels of the one or more generated images with one or more channels of the one or more feature maps obtained from one or more layers of the one or more neural networks; and
input the representation of the one or more images to the one or more neural networks to cause the one or more neural networks to generate labels of one or more objects within the one or more images.
10 . The one or more processors of claim 9 , wherein the synthetic images comprises 50 or fewer images.
11 . The one or more processors of claim 9 , wherein each of the one or more feature maps is output by a different layer of the one or more layers.
12 . The one or more processors of claim 9 , wherein the circuitry is further to:
use the generative network to generate each of the synthetic images, wherein the feature maps are produced in generation of the respective synthetic image.
13 . The one or more processors of claim 9 , wherein the one or more neural networks are to perform operations comprising:
Extracting a plurality of intermediate feature maps from the one or more layers of the generative network, the plurality of intermediate feature maps are generated by the generative network during generation of the synthetic images;
for each intermediate feature map of the plurality of intermediate feature maps, resizing the respective intermediate feature map to a same certain resolution; and
concatenating data from the plurality of intermediate feature maps and the synthetic image to generate a combined feature map, the combined feature map having the one or more channels of one or more feature maps.
14 . The one or more processors of claim 13 , wherein the one or more neural networks are to perform further operations comprising:
inputting the combined feature map to a trained pixel-level classifier to perform pixel-level classification of pixels of the combined feature map;
determining, for each pixel in the combined feature map and using the one or more channels of the one or more feature maps, a classification associated with the respective pixel, wherein the classification is one of a plurality of classifications associated with the combined feature map; and
generating a mask for the synthetic image, wherein each entry in the mask is associated with a specific pixel in the synthetic image and indicates for the specific pixel an association between the specific pixel and a classification of the plurality of classifications.
15 . A method comprising:
causing one or more neural networks to generate one or more feature maps during a generation of one or more images;
generating a representation of the one or more images by adding one or more channels of the one or more generated images with one or more channels of the one or more feature maps obtained from one or more layers of the one or more neural networks; and
inputting the representation of the one or more images to the one or more neural networks to cause the one or more neural networks to generate labels of one or more objects within the one or more images.
16 . The method of claim 15 , wherein the one or more images comprise one or more synthetic images generated by a generative network from the one or more neural networks, wherein each synthetic image of the one or more synthetic images is generated using a combination of data output by a plurality of layers of the generative network that include the one or more feature maps.
17 . The method of claim 16 , wherein the one or more feature maps are an output by a different layer of the plurality of layers.
18 . The method of claim 17 , further comprising:
using the generative network to generate the one or more synthetic images,
wherein for each synthetic image of the one or more synthetic images the data output by the plurality of layers of the generative network is produced in generation of the synthetic image, and wherein the labels are usable to train an additional machine learning model to perform pixel-level segmentation of images.
19 . The method of claim 15 , further comprising:
extracting the one or more feature maps from intermediate layers of a generative network, the one or more feature maps generated by the generative network during generation of a synthetic image;
for each feature map of the one or more feature maps, resizing the respective feature map to a certain resolution; and
concatenating data from the one or more feature maps and the synthetic image to generate a combined feature map, the combined feature map having the one or more channels of the one or more feature maps.
20 . The method of claim 19 , further comprising: inputting the combined feature map to the one or more neural networks to perform pixel-level classification of pixels of the combined feature map;
determining, for each pixel in the combined feature map and using the one or more channels of the one or more feature maps, a classification associated with the respective pixel, wherein the classification is one of a plurality of classifications associated with the combined feature map; and
generating a mask for the synthetic image, wherein each entry in the mask is associated with a specific pixel in the synthetic image and indicates for the specific pixel an association between the specific pixel and a classification of the plurality of classifications.
21 . A system comprising:
one or more processors to:
cause one or more neural networks to generate one or more feature maps during a generation of one or more images;
generate a representation of the one or more images by adding one or more channels of the one or more generated images with one or more channels of the one or more feature maps obtained from one or more layers of the one or more neural networks; and
input the representation of the one or more images to the one or more neural networks to cause the one or more neural networks to generate labels of one or more objects within the one or more images; and
one or more memories to store parameters associated with the one or more neural networks.
22 . The system of claim 21 , wherein the labels of the one or more objects comprise pixel-level labels of the one or more objects within the one or more images.
23 . The system of claim 21 , wherein the one or more feature maps are generated by one or more generative networks that comprise one or more generative adversarial networks (GANs), wherein the one or more images comprise one or more synthetic images generated by a GAN of the one or more GANs.
24 . The system of claim 23 , wherein a plurality of layers of the GAN outputs the one or more feature maps associated with the respective synthetic image of the one or more synthetic images, wherein each of the one or more feature maps is output by a different layer of the plurality of layers.
25 . The system of claim 23 , wherein the one or more processors are further to:
use the GAN to generate the one or more synthetic images wherein the labels are usable to train an additional machine learning model to perform pixel-level segmentation of images.
26 . The system of claim 21 , wherein the one or more processors are further to:
extract the one or more feature maps from intermediate layers of a generative network, the one or more feature maps generated by the generative network during generation of the one or more images;
for each feature map of the one or more feature maps, resize the respective feature map to a certain resolution; and
concatenate data from the one or more feature maps and a synthetic image generated by the generative network to generate a combined feature map, the combined feature map having the one or more channels of the one or more feature maps.
27 . The system of claim 26 , wherein the one or more processors are further to:
input the combined feature map to the one or more neural networks to perform pixel-level classification of pixels of the combined feature map;
determine, for each pixel in the combined feature map and using the one or more channels of the one or more feature maps, a classification associated with the respective pixel, wherein the classification is one of a plurality of classifications associated with the combined feature map; and
generate a mask for the one or more images, wherein each entry in the mask is associated with a specific pixel in the one or more images and indicates for the specific pixel an association between the specific pixel and a classification of the plurality of classifications.
28 . A method comprising:
causing one or more neural networks to generate one or more feature maps during a generation of one or more automobile images;
generating a representation of the one or more automobile images by adding one or more channels of the one or more generated automobile images with one or more channels of the one or more feature maps obtained from one or more layers of the one or more neural networks; and
inputting the representation of the one or more automobile images to the one or more neural networks to cause the one or more neural networks to generate labels of one or more parts of an automobile within the one or more automobile images.
29 . The method of claim 28 , wherein the one or more neural networks comprise one or more generative adversarial networks (GANs), and wherein generating the labels further comprises:
extracting the one or more feature maps from intermediate layers of a GAN of the one or more GANs, the one or more feature maps generated by the GAN during generation of the automobile image;
for each feature map of the one or more feature maps, resizing the respective feature map to a certain resolution; and
concatenating data from the one or more feature maps and the automobile image to generate a combined automobile feature map, the combined automobile feature map having the one or more channels of the one or more feature maps.
30 . The method of claim 29 further comprises:
inputting the combined automobile feature map to the one or more neural networks including a trained pixel-level classifier to perform pixel-level classification of pixels of the combined automobile feature map;
determining, for each pixel in the combined automobile feature map and using the one or more channels of the one or more feature maps, a classification associated with the respective pixel, wherein the classification is one of a plurality of classifications corresponding to automobile parts of the combined automobile feature map; and
generating a mask for the automobile image, wherein each entry in the mask is associated with a specific pixel in the automobile image and indicates for the specific pixel an association between the specific pixel and a classification of the plurality of classifications.
31 . The method of claim 30 further comprises:
using the labels to train an additional machine learning model to perform pixel-level segmentation of the one or more automobile images.
32 . A system, comprising:
one or more processors to cause one or more neural networks to perform pixel-level labeling of synthetic images generated by a generative network, the one or more processors to:
cause the one or more neural networks to generate one or more feature maps during a generation of one or more synthetic images;
generate a representation of the one or more synthetic images by adding one or more channels of the one or more synthetic images with one or more channels of the one or more feature maps generated by a plurality of layers of the generative network; and
input the representation of the one or more synthetic images to the one or more neural networks to cause the one or more neural networks to generate labels of one or more objects within the one or more synthetic images; and
one or more memories to store parameters associated with the one or more neural networks.