Granted Patent B2

US 12,626,420 · App. 18/511,692 · Granted May 12, 2026

Inventors: Kambiz Azarian Yazdi (San Diego, CA); Fatih Murat Porikli (San Diego, CA); Qiqi Hou (San Diego, CA); Debasmit Das (San Diego, CA)

Assignee: QUALCOMM Incorporated

G06T11/00G06F40/284G06T5/70G06T2200/24

Abstract

Claims (78)

1 . An apparatus configured to generate an output image based on a text prompt, comprising:

one or more memories configured to store a latent image representation; and

one or more processors, coupled to the one or more memories, configured to:

obtain the text prompt;

encode the text prompt into a plurality of conditioning tokens;

for each of a plurality of patches of the latent image representation:

calculate a respective plurality of cross-attention weights corresponding to the plurality of conditioning tokens based on the patch as a query and the plurality of conditioning tokens as a key; and

modify a maximum value cross-attention weight among the respective plurality of cross-attention weights to generate a modified respective plurality of cross-attention weights;

perform an iteration of denoising using the modified respective plurality of cross-attention weights for each of the plurality of patches to obtain a modified latent image representation; and

generate the output image based on the modified latent image representation.

2 . The apparatus of claim 1 , wherein to modify the maximum value cross-attention weight among the respective plurality of cross-attention weights comprises to reduce the maximum value cross-attention weight.

3 . The apparatus of claim 1 , wherein to modify the maximum value cross-attention weight among the respective plurality of cross-attention weights comprises to multiply the maximum value cross-attention weight by a negative scalar value.

4 . The apparatus of claim 3 , wherein the one or more processors are configured to receive the negative scalar value as a user-specified parameter.

5 . The apparatus of claim 1 , wherein to modify the maximum value cross-attention weight among the respective plurality of cross-attention weights comprises to set the maximum value cross-attention weight to zero.

6 . The apparatus of claim 1 , wherein the one or more processors are configured to perform one or more initial denoising iterations using classifier-free guidance prior to performing the iteration of denoising using the modified respective plurality of cross-attention weights for each of the plurality of patches.

7 . The apparatus of claim 1 , further comprising a display, coupled to the one or more processors, configured to display a user interface configured to receive input indicative of an emphasis strength associated with one or more words of the text prompt, wherein the emphasis strength controls an amount to modify the maximum value cross-attention weight among the respective plurality of cross-attention weights.

8 . The apparatus of claim 7 , wherein the user interface includes one or more interface elements configured to receive the input, the one or more interface elements including at least one of a slider, a numerical input, or a keyword highlight.

9 . The apparatus of claim 1 , wherein the one or more processors are configured to:

for each of a plurality of patches of the modified latent image representation:

calculate a respective second plurality of cross-attention weights corresponding to the plurality of conditioning tokens based on the patch as another query and the plurality of conditioning tokens as another key; and

modify another maximum value cross-attention weight among the respective second plurality of cross-attention weights to generate another modified respective plurality of second cross-attention weights; and

perform a second iteration of denoising using the modified respective plurality of second cross-attention weights for each of the plurality of patches of the modified latent image representation to obtain a second modified latent image representation, wherein to generate the output image based on the modified latent image representation comprises to decode the second modified latent image representation using a decoder to generate the output image.

10 . The apparatus of claim 1 , further comprising a display, coupled to the one or more processors, configured to display the output image.

11 . The apparatus of claim 1 , further comprising:

a modem, coupled to the one or more processors, configured to modulate one or more carrier wave signals with data indicative of the output image.

12 . The apparatus of claim 11 , further comprising:

one or more antennas, coupled to the modem, configured to transmit the one or more carrier wave signals to a device.

13 . The apparatus of claim 1 , wherein to perform the iteration of denoising, the one or more processors are configured to:

generate a first output based on the latent image representation and the respective plurality of cross-attention weights for each of the plurality of patches of the latent image representation;

generate a second output based on the latent image representation and the modified respective plurality of cross-attention weights for each of the plurality of patches of the latent image representation; and

subtract the second output from the first output to obtain the modified latent image representation.

14 . The apparatus of claim 1 , wherein to perform the iteration of denoising, the one or more processors are configured to:

generate the modified latent image representation based on the latent image representation and the modified respective plurality of cross-attention weights for each of the plurality of patches of the latent image representation.

15 . An apparatus configured to generate an output image based on a text prompt, comprising:

one or more memories configured to store the output image; and

one or more processors, coupled to the one or more memories, configured to:

receive the text prompt;

provide a user interface comprising one or more input elements associated with one or more words of the text prompt;

receive input corresponding to at least one of the one or more input elements, the input indicating a semantic importance for each of at least one of the one or more words associated with the at least one of the one or more input elements; and

generate the output image based on the text prompt and the input.

16 . The apparatus of claim 15 , further comprising a display, coupled to the one or more processors, configured to:

display the user interface; and

display the output image.

17 . The apparatus of claim 15 , wherein to generate the output image, the one or more processors are configured to generate the output image using the text prompt and the input as inputs to a generative artificial intelligence (AI) model.

18 . The apparatus of claim 15 , wherein a first input element of the one or more input elements is a slider element configured to increase or decrease the importance of a first word of the one or more words.

19 . The apparatus of claim 15 , wherein a first input element of the one or more input elements is a dial element configured to increase or decrease the importance of a first word of the one or more words.

20 . The apparatus of claim 15 , wherein the one or more processors are configured to modify an appearance of the at least one of the one or more words based on the indicated semantic importance.

21 . The apparatus of claim 20 , wherein to modify the appearance of the at least one of the one or more words comprises to highlight the at least one of the one or more words.

22 . The apparatus of claim 15 , wherein the output image emphasizes one or more objects associated with the at least one of the one or more words indicated as having higher semantic importance as compared to other words of the one or more words.

23 . The apparatus of claim 15 , further comprising a display, coupled to the one or more processors, configured to:

prior to the one or more processors receiving the input, display a first image associated with the text prompt; and

after the one or more processors receiving the input, display the output image.

24 . The apparatus of claim 23 , wherein the display is configured to:

display a transition from the first image to the output image.

25 . The apparatus of claim 23 , wherein the output image emphasizes one or more objects associated with the at least one of the one or more words as compared to the one or more objects in the first image.

26 . The apparatus of claim 23 , wherein:

the one or more processors are configured to generate the first image using the text prompt as input to a generative artificial intelligence (AI) model; and

to generate the output image, the one or more processors are configured to generate the output image using the text prompt and the input as inputs to the generative AI model.

27 . The apparatus of claim 15 , wherein the one or more processors are configured to:

encode the text prompt into a plurality of conditioning tokens;

for each of one or more patches of a latent image representation:

calculate a respective plurality of cross-attention weights corresponding to the plurality of conditioning tokens based on the patch as a query and the plurality of conditioning tokens as a key; and

modify a maximum value cross-attention weight among the respective plurality of cross-attention weights to generate a modified respective plurality of cross-attention weights; and

perform an iteration of denoising using the modified respective plurality of cross-attention weights for each of the one or more patches to generate the output image.

28 . The apparatus of claim 27 , wherein at least one cross-attention weight among the respective plurality of cross-attention weights is modified based on the indicated semantic importance.

29 . A method comprising:

obtaining a text prompt;

encoding the text prompt into a plurality of conditioning tokens;

for each of a plurality of patches of a latent image representation:

calculating a respective plurality of cross-attention weights corresponding to the plurality of conditioning tokens based on the patch as a query and the plurality of conditioning tokens as a key; and

modifying a maximum value cross-attention weight among the respective plurality of cross-attention weights to generate a modified respective plurality of cross-attention weights;

performing an iteration of denoising using the modified respective plurality of cross-attention weights for each of the plurality of patches to obtain a modified latent image representation; and

generating an output image based on the modified latent representation.

30 . A method comprising:

receiving a text prompt;

providing a user interface comprising one or more input elements associated with one or more words of the text prompt;

receiving input corresponding to at least one of the one or more input elements, the input indicating a semantic importance for each of at least one of the one or more words associated with the at least one of the one or more input elements; and

generating an output image based on the text prompt and the input.

Assignments (1)

Continuity (1)

References Cited (32)