A method is provided that includes: obtaining an editing instruction input by a user in a current round of a dialogue and history dialogue information in at least one history round of the dialogue, wherein the history dialogue information comprises a history dialogue text and at least one history image; determining a source image to be edited from the at least one history image based on the editing instruction and the history dialogue information; and editing the source image to generate a target image based on the editing instruction.
1 . A method, comprising:
obtaining an editing instruction input by a user in a current round of a dialogue and history dialogue information in at least one history round of the dialogue, wherein the history dialogue information comprises a history dialogue text and at least one history image;
determining a source image to be edited from the at least one history image based on the editing instruction and the history dialogue information; and
editing the source image to generate a target image based on the editing instruction, wherein editing the source image comprises:
obtaining a source description text of the source image;
determining a target description text of the target image based on the source description text and the editing instruction; and
generating the target image based on the target description text, wherein the source description text is used to control the process for generating the target image, and wherein generating the target image comprises:
denoising, based on the source description text, a first initial image by performing a plurality of first iterations using a text-to-image diffusion model, and recording random variable values sampled in each first iteration of the plurality of first iterations; and
denoising, based on the target description text, a second initial image by performing a plurality of second iterations using the text-to-image diffusion model to generate the target image, wherein the plurality of second iterations correspond to the plurality of first iterations, respectively, and wherein each second iteration of the plurality of second iterations reuses the random variable values sampled in a corresponding first iteration.
2 . The method according to claim 1 , wherein determining the source image to be edited from the at least one history image comprises:
obtaining a set prompt template, wherein the prompt template comprises a guide text for guiding a language model to determine the source image to be edited from the at least one history image and slots to be filled;
filling the editing instruction and the history dialogue information into the slots to obtain input information; and
inputting the input information into the language model to obtain the source image output by the language model.
3 . The method according to claim 1 , wherein determining the target description text of the target image comprises:
rewriting, based on the editing instruction, the source description text using a language model to obtain the target description text.
4 . The method according to claim 1 , wherein the text-to-image diffusion model comprises a text encoder and a noise generation network, and wherein each second iteration of the plurality of second iterations comprises:
inputting the target description text into the text encoder to generate a target text vector corresponding to the target description text;
inputting the target text vector and the random variable values sampled in the first iteration of the corresponding round into the noise generation network to obtain a predicted noise of a current image; and
removing the predicted noise from the current image to obtain a result image of the second iteration,
wherein the current image in a first second iteration is the second initial image, and the current image in a second second iteration or each subsequent second iteration is the result image generated in a previous second iteration.
5 . The method according to claim 1 , wherein the second initial image is generated based on the source image.
6 . The method according to claim 5 , wherein the second initial image is obtained by adding noise to the source image.
7 . An electronic device, comprising: a processor; and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, and the instructions, when executed by the processor, cause the processor to perform operations comprising:
obtaining an editing instruction input by a user in a current round of a dialogue and history dialogue information in at least one history round of the dialogue, wherein the history dialogue information comprises a history dialogue text and at least one history image;
determining a source image to be edited from the at least one history image based on the editing instruction and the history dialogue information; and
editing the source image to generate a target image based on the editing instruction, wherein editing the source image comprises:
obtaining a source description text of the source image;
determining a target description text of the target image based on the source description text and the editing instruction; and
generating the target image based on the target description text, wherein the source description text is used to control the process for generating the target image, and wherein generating the target image comprises:
denoising, based on the source description text, a first initial image by performing a plurality of first iterations using a text-to-image diffusion model, and recording random variable values sampled in each first iteration of the plurality of first iterations; and
denoising, based on the target description text, a second initial image by performing a plurality of second iterations using the text-to-image diffusion model to generate the target image, wherein the plurality of second iterations correspond to the plurality of first iterations, respectively, and wherein each second iteration of the plurality of second iterations reuses the random variable values sampled in a corresponding first iteration.
8 . The electronic device according to claim 7 , wherein determining the source image to be edited from the at least one history image comprises:
obtaining a set prompt template, wherein the prompt template comprises a guide text for guiding a language model to determine the source image to be edited from the at least one history image and slots to be filled;
filling the editing instruction and the history dialogue information into the slots to obtain input information; and
inputting the input information into the language model to obtain the source image output by the language model.
9 . The electronic device according to claim 7 , wherein determining the target description text of the target image comprises:
rewriting, based on the editing instruction, the source description text using a language model to obtain the target description text.
10 . The electronic device according to claim 7 , wherein the text-to-image diffusion model comprises a text encoder and a noise generation network, and wherein each second iteration of the plurality of second iterations comprises:
inputting the target description text into the text encoder to generate a target text vector corresponding to the target description text;
inputting the target text vector and the random variable values sampled in the first iteration of the corresponding round into the noise generation network to obtain a predicted noise of a current image; and
removing the predicted noise from the current image to obtain a result image of the second iteration,
wherein the current image in a first second iteration is the second initial image, and the current image in a second second iteration or each subsequent second iteration is the result image generated in a previous second iteration.
11 . The electronic device according to claim 7 , wherein the second initial image is generated based on the source image.
12 . The electronic device according to claim 11 , wherein the second initial image is obtained by adding noise to the source image.
13 . A non-transitory computer readable storage medium storing computer instructions, wherein the computer instructions are configured to enable a computer to perform operations comprising:
obtaining an editing instruction input by a user in a current round of a dialogue and history dialogue information in at least one history round of the dialogue, wherein the history dialogue information comprises a history dialogue text and at least one history image;
determining a source image to be edited from the at least one history image based on the editing instruction and the history dialogue information; and
editing the source image to generate a target image based on the editing instruction, wherein editing the source image comprises:
obtaining a source description text of the source image;
determining a target description text of the target image based on the source description text and the editing instruction; and
generating the target image based on the target description text, wherein the source description text is used to control the process for generating the target image, and wherein generating the target image comprises:
denoising, based on the source description text, a first initial image by performing a plurality of first iterations using a text-to-image diffusion model, and recording random variable values sampled in each first iteration of the plurality of first iterations; and
denoising, based on the target description text, a second initial image by performing a plurality of second iterations using the text-to-image diffusion model to generate the target image, wherein the plurality of second iterations correspond to the plurality of first iterations, respectively, and wherein each second iteration of the plurality of second iterations reuses the random variable values sampled in a corresponding first iteration.
14 . The storage medium according to claim 13 , wherein determining the source image to be edited from the at least one history image comprises:
obtaining a set prompt template, wherein the prompt template comprises a guide text for guiding a language model to determine the source image to be edited from the at least one history image and slots to be filled;
filling the editing instruction and the history dialogue information into the slots to obtain input information; and
inputting the input information into the language model to obtain the source image output by the language model.
15 . The storage medium according to claim 13 , wherein determining the target description text of the target image comprises:
rewriting, based on the editing instruction, the source description text using a language model to obtain the target description text.