Image processing method, apparatus, and device, and storage medium

1 . A method for processing an image, comprising:

extracting facial feature information of a first character appearance image and structural feature information of a second character appearance image;

inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtain a third character appearance image;

training a second character appearance fusion model based on the first character appearance image, the second character appearance image and the third character appearance image; and

inputting a source character appearance image and a target character appearance image into the trained second character appearance fusion model to obtain a new character appearance image.

2 . The method of claim 1 , wherein the first character appearance fusion model includes a feature injection module and a first decoder, and inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtain a third character appearance image comprises:

inputting the facial feature information and the structural feature information into the feature injection module to obtain fused feature information; and

inputting the fused feature information into the first decoder to output the third character appearance image.

3 . The method of claim 2 , wherein the first character appearance fusion model further includes a first encoder, the first encoder is hop-connected with the first decoder; and inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtaining a third character appearance image comprises:

inputting the second character appearance image into the first encoder to obtain first encoding information;

inputting the first encoding information, the facial feature information, and the structural feature information into the feature injection module to obtain fused feature information; and

inputting the fused feature information, the first encoding information, and the structural feature information into the first decoder to obtain the third character appearance image.

4 . The method of claim 3 , wherein the feature injection module comprises a fully-connected layer, a convolutional layer and an injection layer; and inputting the first encoding information, the facial feature information, and the structural feature information into the feature injection module to obtain fused feature information comprises:

inputting the facial feature information into the fully-connected layer to obtain intermediate facial feature information;

inputting the structural feature information into the convolutional layer to obtain intermediate structural feature information; and

inputting the intermediate facial feature information, the intermediate structural feature information, and the first encoding information into the injection layer for fusion to output the fused feature information.

5 . The method of claim 1 , wherein the second character appearance fusion model comprises a second encoder and a second decoder; and inputting a source character appearance image and a target character appearance image into the trained second character appearance fusion model to obtain a new character appearance image comprises:

inputting the source character appearance image and the target character appearance image into the second encoder to obtain second encoding information; and

inputting the second encoding information into the second decoder to obtain the new character appearance image.

6 . The method of claim 1 , wherein training the first character appearance fusion model comprises:

extracting first facial feature sample information of a first character appearance sample and first structural feature sample information of a second character appearance sample;

inputting the first facial feature sample information and the first structural feature sample information into the first character appearance fusion model to output a third character appearance sample;

extracting second facial feature sample information and second structural feature sample information of the third character appearance sample;

determining a first loss function of the second facial feature sample information and the first facial feature sample information, and a second loss function of the second structural feature sample information and the first structural feature sample information; and

training the first character appearance fusion model based on the first loss function and the second loss function.

7 . The method of claim 1 , wherein training a second character appearance fusion model based on the first character appearance image, the second character appearance image, and the third character appearance image comprises:

inputting the first character appearance image and the second character appearance image into the second character appearance fusion model to obtain a fused character appearance image;

determining a third loss function of the fused character appearance image and the third character appearance image; and

training the second character appearance fusion model based on the third loss function.

8 . The method of claim 1 , including a computer program product which, when executed by a computer, causes the computer to implement the method, comprising:

extracting facial feature information of a first character appearance image and structural feature information of a second character appearance image;

inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtain a third character appearance image;

training a second character appearance fusion model based on the first character appearance image, the second character appearance image and the third character appearance image; and

inputting a source character appearance image and a target character appearance image into the trained second character appearance fusion model to obtain a new character appearance image.

9 . An electronic device, comprising:

at least one processor; and

a memory configured to store at least one program which, when executed by the at least one processor, causes the at least one processor to implement the method, comprising:

extracting facial feature information of a first character appearance image and structural feature information of a second character appearance image;

inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtain a third character appearance image;

training a second character appearance fusion model based on the first character appearance image, the second character appearance image and the third character appearance image; and

inputting a source character appearance image and a target character appearance image into the trained second character appearance fusion model to obtain a new character appearance image.

10 . The electronic device of claim 9 , wherein the first character appearance fusion model includes a feature injection module and a first decoder, and inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtain a third character appearance image comprises:

inputting the facial feature information and the structural feature information into the feature injection module to obtain fused feature information; and

inputting the fused feature information into the first decoder to output the third character appearance image.

11 . The electronic device of claim 10 , wherein the first character appearance fusion model further includes a first encoder, the first encoder is hop-connected with the first decoder; and inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtaining a third character appearance image comprises:

inputting the second character appearance image into the first encoder to obtain first encoding information;

inputting the first encoding information, the facial feature information, and the structural feature information into the feature injection module to obtain fused feature information; and

inputting the fused feature information, the first encoding information, and the structural feature information into the first decoder to obtain the third character appearance image.

12 . The electronic device of claim 11 , wherein the feature injection module comprises a fully-connected layer, a convolutional layer and an injection layer; and inputting the first encoding information, the facial feature information, and the structural feature information into the feature injection module to obtain fused feature information comprises:

inputting the facial feature information into the fully-connected layer to obtain intermediate facial feature information;

inputting the structural feature information into the convolutional layer to obtain intermediate structural feature information; and

inputting the intermediate facial feature information, the intermediate structural feature information, and the first encoding information into the injection layer for fusion to output the fused feature information.

13 . The electronic device of claim 9 , wherein the second character appearance fusion model comprises a second encoder and a second decoder; and inputting a source character appearance image and a target character appearance image into the trained second character appearance fusion model to obtain a new character appearance image comprises:

inputting the source character appearance image and the target character appearance image into the second encoder to obtain second encoding information; and

inputting the second encoding information into the second decoder to obtain the new character appearance image.

14 . The electronic device of claim 9 , wherein training the first character appearance fusion model comprises:

extracting first facial feature sample information of a first character appearance sample and first structural feature sample information of a second character appearance sample;

inputting the first facial feature sample information and the first structural feature sample information into the first character appearance fusion model to output a third character appearance sample;

extracting second facial feature sample information and second structural feature sample information of the third character appearance sample;

determining a first loss function of the second facial feature sample information and the first facial feature sample information, and a second loss function of the second structural feature sample information and the first structural feature sample information; and

training the first character appearance fusion model based on the first loss function and the second loss function.

15 . The electronic device of claim 14 , wherein training a second character appearance fusion model based on the first character appearance image, the second character appearance image, and the third character appearance image comprises:

inputting the first character appearance image and the second character appearance image into the second character appearance fusion model to obtain a fused character appearance image;

determining a third loss function of the fused character appearance image and the third character appearance image; and

training the second character appearance fusion model based on the third loss function.

16 . A non-transitory computer-readable storage medium, storing a computer program which, when executed by a processor, causes the processor to implement the method, comprising:

extracting facial feature information of a first character appearance image and structural feature information of a second character appearance image;

inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtain a third character appearance image;

training a second character appearance fusion model based on the first character appearance image, the second character appearance image and the third character appearance image; and

inputting a source character appearance image and a target character appearance image into the trained second character appearance fusion model to obtain a new character appearance image.

17 . The non-transitory computer-readable storage medium of claim 16 , wherein the first character appearance fusion model includes a feature injection module and a first decoder, and inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtain a third character appearance image comprises:

inputting the facial feature information and the structural feature information into the feature injection module to obtain fused feature information; and

inputting the fused feature information into the first decoder to output the third character appearance image.

18 . The non-transitory computer-readable storage medium of claim 17 , wherein the first character appearance fusion model further includes a first encoder, the first encoder is hop-connected with the first decoder; and inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtaining a third character appearance image comprises:

inputting the second character appearance image into the first encoder to obtain first encoding information;

inputting the first encoding information, the facial feature information, and the structural feature information into the feature injection module to obtain fused feature information; and

inputting the fused feature information, the first encoding information, and the structural feature information into the first decoder to obtain the third character appearance image.

19 . The non-transitory computer-readable storage medium of claim 18 , wherein the feature injection module comprises a fully-connected layer, a convolutional layer and an injection layer; and inputting the first encoding information, the facial feature information, and the structural feature information into the feature injection module to obtain fused feature information comprises:

inputting the facial feature information into the fully-connected layer to obtain intermediate facial feature information;

inputting the structural feature information into the convolutional layer to obtain intermediate structural feature information; and

inputting the intermediate facial feature information, the intermediate structural feature information, and the first encoding information into the injection layer for fusion to output the fused feature information.

20 . The non-transitory computer-readable storage medium of claim 16 , wherein the second character appearance fusion model comprises a second encoder and a second decoder; and inputting a source character appearance image and a target character appearance image into the trained second character appearance fusion model to obtain a new character appearance image comprises:

inputting the source character appearance image and the target character appearance image into the second encoder to obtain second encoding information; and

inputting the second encoding information into the second decoder to obtain the new character appearance image.