Denoising diffusion generative adversarial networks
Apparatuses, systems, and techniques are presented to train and utilize one or more neural networks. A denoising diffusion generative adversarial network (denoising diffusion GAN) reduces a number of denoising steps during a reverse process. The denoising diffusion GAN does not assume a Gaussian distribution for large steps of the denoising process and applies a multi-model model to permit denoising with fewer steps. Systems and methods further minimize a divergence between a diffused real data distribution and a diffused generator distribution over several timesteps. Accordingly, various embodiments may enable faster sample generation, in which the samples are generated from noise using the denoising diffusion GAN.
1. A method comprising:
receiving a sample input;
generating, from the sample input, a series of noisy inputs corresponding to a denoising distribution having a plurality of steps;
selecting a first noisy input for a first step of the plurality of steps;
selecting, based at least in part on the first noisy input, a second noisy input for a second step of the plurality of steps, the first step preceding the second step;
generating, from the second noisy input, a generated output;
generating, from the generated output, a second generated output based, at least in part, on a reverse process to add noise to the generated output;
comparing, within a discriminator, the first noisy input, the second noisy input, and the second generated output; and
determining a loss based, at least in part, on the comparing.
2. The method of claim 1 , wherein generating the generated output includes processing the second noisy input using a trained generator of a generative adversarial network (GAN).
3. The method of claim 1 , wherein the second step is directly after the first step.
4. The method of claim 1 , further comprising:
adding a latent variable for generating the generated output.
5. The method of claim 1 , wherein a number of denoising steps to generate the generated output is a fixed constant.
6. The method of claim 1 , further comprising:
retraining a generator based, at least in part, on the loss.
7. The method of claim 1 , wherein the reverse process includes, at least in part, posterior sampling.
8. A processor, comprising:
one or more processing units to:
generate, using forward diffusion, a series of noisy inputs corresponding to a plurality of steps of the forward diffusion;
select, from the series of noisy inputs, a first noisy input;
select, from the series of noisy inputs, a second noisy input, corresponding to a final noisy input in the series;
generate a predicted noisy input based, at least in part, on the second noisy input and a latent variable, wherein the predicted noisy input includes noise added to a predicted input; and
determine a loss for training a generator based, at least in part, on a comparison between the first noisy input, the second noisy input, and the predicted noisy input.
9. The processor of claim 8 , wherein the predicted noisy input is modeled based, at least in part, on a multi-modal denoising distribution.
10. The processor of claim 8 , wherein the one or more processing units are further to:
receive an input;
provide the input to a trained generator; and
provide an output corresponding to the input.
11. The processor of claim 8 , wherein the generator forms at least a portion of a conditional generative adversarial network.
12. The processor of claim 8 , wherein the loss is based, at least in part, on an adversarial training process.
13. The processor of claim 8 , wherein the one or more processing units are further to:
update one or more parameters of the generator based, at least in part, on the loss.
14. The processor of claim 8 , wherein the series of noisy inputs form a true denoising distribution without a Gaussian assumption.
15. The processor of claim 14 , wherein the true denoising distribution computes a reverse diffusion process with fewer steps than the forward diffusion.
16. A system, comprising:
one or more processors comprising processing circuitry to generate, using a processing sequence including a conditional generative adversarial network and a diffusion process, an output image from an input using a smaller number of steps during a reverse denoising process of the diffusion process than during a forward denoising process of the diffusion process.
17. The system of claim 16 , wherein the system comprises at least one of:
a system for performing simulation operations;
a system for performing simulation operations to test or validate autonomous machine applications;
an infotainment system of a machine;
an entertainment system of a machine;
a system for generating synthetic data;
a system for collaborative content creation of multi-dimensional assets;
a system for performing digital twin simulation;
a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content;
a system for rendering graphical output;
a system for performing deep learning operations;
a system implemented using an edge device;
a system incorporating one or more Virtual Machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
18. The system of claim 16 , wherein the diffusion process includes generating a true denoising distribution that is multi-modal and non-Gaussian for steps larger than a threshold.
19. The system of claim 16 , wherein the one or more processors comprising processing circuitry are further to train a generator of the conditional generative adversarial network using a time-dependent discriminator.
20. The system of claim 16 , wherein the input it as least one of an input image or an input text sequence.