Vector generative fill

1 . A method comprising:

obtaining an outline image and an input prompt, wherein the outline image includes an outline of a region and the input prompt indicates content for the region;

generating a mask based on the outline image, wherein the mask indicates the region within the outline;

generating a preliminary image based on the mask, wherein the preliminary image includes noise in the region indicated by the outline;

generating, using an image generation model, a synthetic image based on the input prompt, the mask, and the preliminary image, wherein the synthetic image depicts the content in the region indicated by the outline; and

generating a vectorized image based on the synthetic image.

2 . The method of claim 1 , wherein generating the mask comprises:

labeling each pixel of the mask based on whether the pixel is located within the outline.

3 . The method of claim 1 , wherein generating the preliminary image comprises:

adding noise to the region within the outline.

4 . The method of claim 1 , wherein generating the synthetic image comprises:

adding additional noise to the preliminary image to obtain a noise input; and

denoising the noise input based on the preliminary image to generate the synthetic image.

5 . The method of claim 1 , wherein generating the synthetic image comprises:

obtaining an outline strength parameter; and

performing a number of diffusion steps based on the outline strength parameter.

6 . The method of claim 1 , wherein generating the synthetic image comprises:

encoding the input prompt to obtain a text embedding, wherein the synthetic image is generated based on the text embedding.

7 . The method of claim 1 , wherein generating the synthetic image comprises:

obtaining a reference style, wherein the synthetic image is generated based on the reference style.

8 . The method of claim 1 , wherein:

the synthetic image comprises a vectorizable image.

9 . The method of claim 1 , further comprising:

removing a background region of the vectorized image based on the mask.

10 . A non-transitory computer readable medium storing code for image processing, the code comprising instructions that, when executed by at least one processor, causes the at least one processor to perform operations comprising:

obtaining an outline image and an input prompt, wherein the outline image includes an outline of a region and the input prompt indicates content for the region;

generating a mask based on the outline image, wherein the mask indicates the region within the outline;

generating a preliminary image based on the mask, wherein the preliminary image includes noise in the region indicated by the outline;

generating, using an image generation model, a synthetic image based on the input prompt, the mask, and the preliminary image, wherein the synthetic image depicts the content in the region indicated by the outline; and

generating a vectorized image based on the synthetic image.

11 . The non-transitory computer readable medium of claim 10 , the code further comprising instructions that, when executed by the at least one processor, causes the at least one processor to perform operations comprising:

labeling each pixel of the mask based on whether the pixel is located within an outline of the outline image.

12 . The non-transitory computer readable medium of claim 10 , wherein generating the preliminary image comprises:

adding noise to a region within an outline of the outline image.

13 . The non-transitory computer readable medium of claim 10 , the code further comprising instructions that, when executed by the at least one processor, causes the at least one processor to perform operations comprising:

obtaining an outline strength parameter; and

performing a number of diffusion steps based on the outline strength parameter.

14 . The non-transitory computer readable medium of claim 10 , the code further comprising instructions that, when executed by the at least one processor, causes the at least one processor to perform operations comprising:

encoding the input prompt to obtain a text embedding.

15 . The non-transitory computer readable medium of claim 10 , the code further comprising instructions that, when executed by the at least one processor, causes the at least one processor to perform operations comprising:

denoising the noise input based on the preliminary image to generate the synthetic image.

16 . The non-transitory computer readable medium of claim 10 , the code further comprising instructions that, when executed by the at least one processor, causes the at least one processor to perform operations comprising:

removing a background region of the vectorized image.

17 . A system comprising:

a memory component; and

a processing device coupled to the memory component, the processing device configured to perform operations comprising:

obtaining an outline image and an input prompt, wherein the outline image includes an outline of a region and the input prompt indicates content for the region;

generating a mask based on the outline image, wherein the mask indicates the region within the outline;

generating a preliminary image based on the mask, wherein the preliminary image includes noise in the region indicated by the outline;

generating, using an image generation model, a synthetic image based on the input prompt, the mask, and the preliminary image, wherein the synthetic image depicts the content in the region indicated by the outline; and

generating a vectorized image based on the synthetic image.

18 . The system of claim 17 , further comprising:

a text encoder configured to encode the input prompt to obtain a text embedding, wherein the synthetic image is generated based on the text embedding.

19 . The system of claim 17 , further comprising:

a vectorization component configured to generate the vectorized image based on the synthetic image.

20 . The system of claim 17 , wherein:

the image generation model includes a diffusion model.