Controllable image generation using one or more neural networks
Apparatuses, systems, and techniques are presented to generate one or more images. In at least one embodiment, one or more neural networks are used to generate one or more images of one or more objects including one or more portions of one or more other objects.
1 . One or more processors, comprising:
circuitry to:
obtain geometric data and texture information for one or more first objects at least partially depicted in one or more first images, and specified poses of one or more second objects at least partially depicted in one or more second images; and
use the geometric data and texture information for the one or more first objects and the specified poses of the one or more second objects as input to one or more neural networks to generate one or more second images to depict the one or more first objects in the same specified poses as the one or more second objects.
2 . The one or more processors of claim 1 , wherein the circuitry is further to use the one or more neural networks to generate the one or more second images including a representation of another object in the specified pose depicting the one or more first objects in the same specified poses as the one or more second objects.
3 . The one or more processors of claim 1 , wherein the circuitry is further to identify the one or more first objects of the one or more first images to be posed across different images.
4 . The one or more processors of claim 1 , wherein the one or more neural networks include a generative model to synthesize the one or more second images, wherein the generative model is trained to synthesize image data for multiple different objects.
5 . The one or more processors of claim 1 , wherein the circuitry is further to train a generative model using at least one video including a representation of different objects.
6 . The one or more processors of claim 1 , wherein the circuitry is further to use the one or more neural networks to generate the one or more second images from one or more viewpoints, wherein the one or more viewpoints include viewpoints not represented in a single video.
7 . A system comprising:
one or more processors to:
obtain geometric data and texture information for one or more first objects at least partially depicted in one or more first images, and specified poses of one or more second objects at least partially depicted in one or more second images; and
use the geometric data and texture information for the one or more first objects and the specified poses of the one or more second objects as input to one or more neural networks to generate one or more second images to depict the one or more first objects in the same specified poses as the one or more second objects.
8 . The system of claim 7 , wherein the one or more processors are further to use the one or more neural networks to generate one or more second images including a representation of another object in the specified pose depicting the one or more first objects in the same specified poses as the one or more second objects.
9 . The system of claim 7 , wherein the one or more processors are further to identify the one or more first objects of the one or more first images and the one or more poses of the one or more second objects.
10 . The system of claim 7 , wherein the one or more neural networks include a generative model to synthesize the one or more second images, wherein the generative model is trained to synthesize image data for multiple different objects.
11 . The system of claim 7 , wherein the one or more processors are further to train a generative model using at least one video including a representation of one or more different objects.
12 . The system of claim 7 , wherein the one or more processors are further to use the one or more neural networks to generate the one or more second images from one or more viewpoints, wherein the one or more viewpoints include viewpoints not represented in a single video.
13 . A method comprising:
obtaining geometric data and texture information for one or more first objects at least partially depicted in one or more first images, and specified poses of one or more second objects at least partially depicted in one or more second images; and
using the geometric data and texture information for the one or more first objects and the specified poses of the one or more second objects as input to one or more neural networks to generate one or more second images to depict the one or more first objects in the same specified poses as the one or more second objects.
14 . The method of claim 13 , further comprising:
using the one or more neural networks to generate one or more second images including a representation of another object in the specified pose depicting the one or more first objects in the same specified poses as the one or more second objects.
15 . The method of claim 13 , further comprising:
identifying the one or more first objects of the one or more first images to be posed across different images.
16 . The method of claim 13 , wherein the one or more neural networks include a generative model to synthesize the one or more second images, wherein the generative model is trained to synthesize image data for multiple different objects.
17 . The method of claim 13 , further comprising:
training a generative model using at least one video including a representation of different objects.
18 . The method of claim 13 , further comprising:
using the one or more neural networks to generate the one or more second images from one or more viewpoints, wherein the one or more viewpoints include different viewpoints not represented in a single video.
19 . An image synthesis system, comprising:
one or more processors to obtain geometric data and texture information for one or more first objects at least partially depicted in one or more first images, and specified poses of one or more second objects at least partially depicted in one or more second images; and
use the geometric data and texture information for the one or more first objects and the specified poses of the one or more second objects as input to one or more neural networks to generate one or more second images to depict the one or more first objects in the same specified poses as the one or more second objects; and
memory for storing network parameters for the one or more neural networks.
20 . The image synthesis system of claim 19 , wherein the one or more processors are further to use the one or more neural networks to generate one or more second images including a representation of another object in the same specified pose depicting the one or more first objects in the specified poses of the one or more second objects.
21 . The image synthesis system of claim 19 , wherein the one or more processors are further to identify the one or more first objects of the one or more first images to be posed across different images.
22 . The image synthesis system of claim 19 , wherein the one or more neural networks include a generative model to synthesize the one or more second images, wherein the generative model is trained to synthesize image data for multiple different objects.
23 . The image synthesis system of claim 19 , wherein the one or more processors are further to train a generative model using at least one video including a representation of different objects.
24 . The image synthesis system of claim 19 , wherein the one or more processors are further to use the one or more neural networks to generate the one or more second images from one or more viewpoints.