Hybrid sampling for diffusion models

1 . A method comprising:

obtaining a noisy image;

generating a first denoised image based on the noisy image during a first reverse diffusion phase using a diffusion neural network, wherein the first denoised image is generated based on a first sampler that uses a first sampling density during at least a portion of the first reverse diffusion phase; and

generating a second denoised image based on the first denoised image during a second reverse diffusion phase using the diffusion neural network, wherein the second denoised image is generated based on a second sampler that uses a second sampling density different from the first sampling density during at least a portion of the second reverse diffusion phase.

2 . The method of claim 1 , wherein:

the first sampler uses a first sampling density during at least a portion of the first reverse diffusion phase, the second sampler uses a second sampling density during at least a portion of the second reverse diffusion phase, and wherein the first sampling density is greater than the second sampling density.

3 . The method of claim 1 , wherein:

the first sampler is based on a denoising diffusion implicit model (DDIM) and the second sampler is based on a diffusion probabilistic model (DPM).

4 . The method of claim 1 , wherein:

the first sampler is based on a lower order differential equation solver and the second sampler is based on a higher order differential equation solver.

5 . The method of claim 1 , further comprising:

injecting noise during each iteration of the first reverse diffusion phase.

6 . The method of claim 1 , further comprising:

refraining from injecting noise during each iteration of the second reverse diffusion phase.

7 . The method of claim 1 , further comprising:

generating a third denoised image based on the second denoised image during a third reverse diffusion phase using the diffusion neural network, wherein the third denoised image is generated based on a third sampler.

8 . The method of claim 7 , further comprising:

refraining from injecting noise during the third reverse diffusion phase.

9 . The method of claim 7 , wherein:

the third sampler is based on a DDIM.

10 . The method of claim 7 , wherein:

the third sampler is based on a lower order differential equation solver and the second sampler is based on a higher order differential equation solver.

11 . The method of claim 7 , wherein:

the first sampler uses a first sampling density during at least a portion of the first reverse diffusion phase, the third sampler uses a third sampling density during at least a portion of the third reverse diffusion phase, and wherein the first sampling density is greater than the third sampling density.

12 . A method comprising:

obtaining a noisy image;

generating a first denoised image based on the noisy image during a first reverse diffusion phase using a diffusion neural network;

injecting noise into the first denoised image during the first reverse diffusion phase; and

generating a second denoised image based on the first denoised image and the noise injection from the first reverse diffusion phase, wherein the second denoised image is generated during a second reverse diffusion phase using the diffusion neural network without injecting noise during the second reverse diffusion phase.

13 . The method of claim 12 , further comprising:

identifying a diffusion time step during the first reverse diffusion phase; and

removing noise from the noisy image based on the diffusion time step using a first sampler to obtain the first denoised image, wherein the first sampler operates based on the noise injection.

14 . The method of claim 13 , further comprising:

identifying an additional diffusion time step during the first reverse diffusion phase; and

removing noise at the additional diffusion time step using the first sampler.

15 . The method of claim 12 , further comprising:

identifying a diffusion time step during the second reverse diffusion phase; and

removing noise from the noisy image based on the diffusion time step using a second sampler to obtain the second denoised image, wherein the second sampler operates without noise injection.

16 . The method of claim 15 , further comprising:

identifying an additional diffusion time step during the second reverse diffusion phase; and

removing noise at the additional diffusion time step using the second sampler.

17 . An apparatus comprising:

at least one processor;

at least one memory including instructions executable by the processor;

the apparatus further comprising a diffusion neural network comprising parameters stored in the at least one memory, wherein the diffusion neural network is configured to compute noise of a noisy image using a reverse diffusion process;

a first sampler configured to remove noise based on an output of the diffusion neural network using a first sampling density during a first reverse diffusion phase of the reverse diffusion process; and

a second sampler configured to remove noise based on an output of the diffusion neural network using a second sampling density different from the first sampling density during a second reverse diffusion phase of the reverse diffusion process.

18 . The apparatus of claim 17 , wherein:

the first sampler injects noise during the first reverse diffusion phase, and the second sampler does not inject noise during the second reverse diffusion phase.

19 . The apparatus of claim 17 , further comprising:

a third sampler configured to remove noise based on an output of the diffusion neural network during a third reverse diffusion phase.

20 . The apparatus of claim 17 , wherein:

the diffusion neural network comprises a U-net architecture.