Generation of image corresponding to input text using dynamic value clipping

1. A computer system for generating an output image corresponding to an input text, the computing system comprising:

a processor and memory of a computing device, the processor being configured to execute a program using portions of the memory to:

receive the input text from a user; and

for a predetermined number of iterations:

input an initial image into a diffusion process to generate a processed image;

back-propagate the processed image through a text-image match gradient calculator to calculate a gradient against the input text; and

update the initial image with an image generated by applying the calculated gradient to the processed image;

wherein:

pixel values of the processed image during a first portion of the predetermined number of iterations are value clamped to a first range; and

pixel values of the processed image during a second portion of the predetermined number of iterations are value clamped to a second range that is a subset of the first range.

2. The computer system of claim 1 , wherein the first portion of the predetermined number of iterations is at least seventy percent of the predetermined number of iterations.

3. The computer system of claim 2 , wherein the second portion of the predetermined number of iterations is at least twenty percent of the predetermined number of iterations.

4. The computer system of claim 1 , wherein the first range is approximately [−3,3] unit Gaussian normalized to a maximum value of a pixel.

5. The computer system of claim 4 , wherein the second range is approximately [−1.4,1.4] unit Gaussian.

6. The computer system of claim 1 , wherein pixel values of the processed image during a third portion of the predetermined number of iterations are value clamped to a third range that is a subset of the second range.

7. The computer system of claim 6 , wherein the third range is approximately [−1,1] unit Gaussian normalized to a maximum value of a pixel.

8. The computer system of claim 1 , wherein the diffusion process is a denoising diffusion implicit model.

9. The computer system of claim 1 , wherein the diffusion process includes a gradient estimator model.

10. The computer system of claim 1 , wherein the predetermined number of iterations is between 70 and 100 iterations.

11. A method for generating an output image corresponding to an input text, the method comprising steps to:

receive the input text from a user; and

for a predetermined number of iterations:

input an initial image into a diffusion process to generate a processed image;

back-propagate the processed image through a text-image match gradient calculator to calculate a gradient against the input text; and

update the initial image with an image generated by applying the calculated gradient to the processed image;

wherein:

pixel values of the processed image during a first portion of the predetermined number of iterations are value clamped to a first range; and

pixel values of the processed image during a second portion of the predetermined number of iterations are value clamped to a second range that is a subset of the first range.

12. The method of claim 11 , wherein the first portion of the predetermined number of iterations is at least seventy percent of the predetermined number of iterations.

13. The method of claim 12 , wherein the second portion of the predetermined number of iterations is at least twenty percent of the predetermined number of iterations.

14. The method of claim 11 , wherein the first range is approximately [−3,3] unit Gaussian normalized to a maximum value of a pixel.

15. The method of claim 14 , wherein the second range is approximately [−1.4,1.4] unit Gaussian.

16. The method of claim 11 , wherein pixel values of the processed image during a third portion of the predetermined number of iterations are value clamped to a third range that is a subset of the second range.

17. The method of claim 16 , wherein the third range is approximately [−1,1] unit Gaussian normalized to a maximum value of a pixel.

18. The method of claim 11 , wherein the diffusion process is a denoising diffusion implicit model.

19. The method of claim 11 , wherein the diffusion process includes a gradient estimator model.

20. A computer system for generating an output image corresponding to an input text, the computing system comprising:

a processor and memory of a computing device, the processor being configured to execute a program using portions of the memory to:

receive the input text from a user;

process an initial image through a first diffusion stage to generate a final first stage image, wherein the first diffusion stage includes processing the initial image, for a first predetermined number of iterations, using a diffusion model, a gradient estimator model smaller than the diffusion model, and a text-image match gradient calculator; and

process the final first stage image through a second diffusion stage to generate a final second stage image, wherein the second diffusion stage includes using the final first stage image as a second stage image to, for a second predetermined number of iterations, perform steps to:

input the second stage image through the diffusion model to generate a second stage processed image;

back-propagate the second stage processed image through the text-image match gradient calculator to calculate a second stage gradient against the input text; and

update the second stage image by applying the second stage gradient to the second stage processed image;

wherein:

pixel values of the processed initial image during a first portion of the first predetermined number of iterations are value clamped to a first range; and

pixel values of the processed initial image during a second portion of the first predetermined number of iterations are value clamped to a second range that is a subset of the first range.