Granted Patent B2

US 12,633,011 · App. 18/289,855 · Granted May 19, 2026

Inventors: Siyu Zhou (Beijing, CN); Qian He (Beijing, CN)

Assignee: BEIJING ZITIAO NETWORK TECHNOLOGY CO., LTD.

G06T11/60G06T5/50G06T7/12G06T2207/20221G06T2207/30201

Abstract

Claims (80)

1 . A method for image processing, comprising:

obtaining, via an interface of an electronic device, a user image; and

performing, by the electronic device, a beautifier operation on the user image with an image beautifier model, to obtain a beautified user image;

wherein the image beautifier model is trained with a training sample pair determined based on a first sample image and a second sample image; the second sample image is an image obtained by performing an overall beautifier and adjustment of an image element on the first sample image based on a target image using an image processing model; the image element is an image style feature and/or a target part of the target image, wherein the image processing model comprises a first sub-model and a second sub-model;

wherein the performing an overall beautifier and adjustment of an image element to the first sample image based on a target image using an image processing model comprises:

performing overall beautifier on the first sample image using the first sub-model, to obtain a to-be-edited image;

obtaining a first segmented image based on the target image;

obtaining a second segmented image based on the to-be-edited image; and

obtaining the second sample image by inputting the first segmented image, the second segmented image, the target image, and the to-be-edited image into the second sub-model.

2 . The method of claim 1 , wherein the performing an overall beautifier and adjustment of an image element to the first sample image based on a target image using an image processing model comprises:

obtaining the first sample image and the target image; and

generating the second sample image corresponding to the first sample image using the image processing model, wherein the first sub-model is used for image overall beautifier, the second sub-model is used for adjustment of an image element based on the target image.

3 . The method of claim 2 , wherein a training process of the image beautifier model comprises:

determining the first sample image and the second sample image as the training sample pair; and

training the image beautifier model based on the training sample pair to cause the trained image beautifier model to perform the beautifier operation on the user image, wherein the beautifier operation comprises adjustment of an image style feature and/or a target part of a target image.

4 . The method of claim 2 , wherein the generating the second sample image corresponding to the first sample image using the image processing model comprises:

adjusting an image element of the to-be-edited image based on the target image using the second sub-model, to obtain the second sample image.

5 . The method of claim 4 , wherein the obtaining a second sample image by adjusting an image element of the to-be-edited image based on the target image with the second sub-model comprises:

inputting the target image and the to-be-edited image into the second sub-model; and

obtaining a second sample image by transforming an image style of the to-be-edited image based on the target image using the second sub-model, wherein the second sample image comprises both a style feature of the to-be-edited image and an image style feature of the target image.

6 . The method of claim 4 , wherein the adjusting an image element of the to-be-edited image based on the target image using the second sub-model to obtain the second sample image comprises:

performing face segmentation on the target image to obtain an initial segmented image;

adding different identifications to the initial segmented image based on segmented parts on the initial segmented image, to obtain a plurality of first segmented images with different identifications, wherein an identification is used to represent a part on a first segmented image that is a beautifier target;

performing face segmentation on the to-be-edited image, to obtain second segmented images; and

processing, using the second sub-model, a target part of a second segmented image and a first part represented by an identification in a first segmented image, to obtain a second sample image, wherein the target part and the first part are the same part of a face.

7 . The method of claim 6 , wherein the method further comprises:

determining a first part on a first segmented image as a beautifier target based on an identification of the first segmented image; and

determining a second part on a second segmented image which is the same as the first part and determining the second part as a target part of the to-be-edited image.

8 . The method of claim 2 , wherein the obtaining the first sample image and the target image comprises:

obtaining a first image set and a second image set, wherein the first image set comprises unbeautified images, the second image set comprises beautified images, and the number of images in the second image set is less than that in the first image set;

obtaining any unbeautified image from the first image set as a first sample image; and

obtaining any beautified image from the second image set as a target image, wherein the target image is used as a beautifier target of the first sample image.

9 . The method of claim 8 , wherein the method further comprises:

generating a generated image corresponding to an unbeautified image in the first image set using the first sub-model and determining a third image set from respective generated images;

randomly combining images in the third image set and beautified images in the second image set, to obtain a plurality of different image combinations, wherein within an image combination, an image belonging to the third image set is a third sample image, and an image belonging to the second image set is a fourth sample image; and

repeating the following training operations with the different image combinations until a preset condition is satisfied:

generating a fifth sample image by inputting an image combination into a to-be-trained second sub-model;

determining a loss function between the fifth sample image and the fourth sample image;

converging the to-be-trained second sub-model based on the loss function, and determining that the preset condition is satisfied and the training stops until the loss function converges to a preset value; and

using the second sub-model obtained after the training stops as the trained second sub-model.

10 . The method of claim 1 , wherein the image beautifier model is applied in a mobile terminal.

11 . An electronic device, comprising:

a processor; and

a memory for storing instructions executable by the processor which when executed by the processor, causing the processor to perform acts comprising:

obtaining, via an interface of the electronic device, a user image; and

performing, by the electronic device, a beautifier operation on the user image with an image beautifier model, to obtain a beautified user image;

wherein the image beautifier model is trained with a training sample pair determined based on a first sample image and a second sample image; the second sample image is an image obtained by performing an overall beautifier and adjustment of an image element on the first sample image based on a target image using an image processing model; the image element is an image style feature and/or a target part of the target image, wherein the image processing model comprises a first sub-model and a second sub-model;

wherein the performing an overall beautifier and adjustment of an image element to the first sample image based on a target image using an image processing model comprises:

performing overall beautifier on the first sample image using the first sub-model, to obtain a to-be-edited image;

obtaining a first segmented image based on the target image;

obtaining a second segmented image based on the to-be-edited image; and

obtaining the second sample image by inputting the first segmented image, the second segmented image, the target image, and the to-be-edited image into the second sub-model.

12 . The electronic device of claim 11 , wherein the performing an overall beautifier and adjustment of an image element to the first sample image based on a target image using an image processing model comprises:

obtaining the first sample image and the target image; and

generating the second sample image corresponding to the first sample image using the image processing model, wherein the first sub-model is used for image overall beautifier, the second sub-model is used for adjustment of an image element based on the target image.

13 . The electronic device of claim 12 , wherein a training process of the image beautifier model comprises:

determining the first sample image and the second sample image as the training sample pair; and

training the image beautifier model based on the training sample pair to cause the trained image beautifier model to perform the beautifier operation on the user image, wherein the beautifier operation comprises adjustment of an image style feature and/or a target part of a target image.

14 . The electronic device of claim 12 , wherein the generating the second sample image corresponding to the first sample image using the image processing model comprises:

adjusting an image element of the to-be-edited image based on the target image using the second sub-model, to obtain the second sample image.

15 . The electronic device of claim 14 , wherein the obtaining a second sample image by adjusting an image element of the to-be-edited image based on the target image with the second sub-model comprises:

inputting the target image and the to-be-edited image into the second sub-model; and

obtaining a second sample image by transforming an image style of the to-be-edited image based on the target image using the second sub-model, wherein the second sample image comprises both a style feature of the to-be-edited image and an image style feature of the target image.

16 . The electronic device of claim 14 , wherein the adjusting an image element of the to-be-edited image based on the target image using the second sub-model to obtain the second sample image comprises:

performing face segmentation on the target image to obtain an initial segmented image;

adding different identifications to the initial segmented image based on segmented parts on the initial segmented image, to obtain a plurality of first segmented images with different identifications, wherein an identification is used to represent a part on a first segmented image that is a beautifier target;

performing face segmentation on the to-be-edited image, to obtain second segmented images; and

processing, using the second sub-model, a target part of a second segmented image and a first part represented by an identification in a first segmented image, to obtain a second sample image, wherein the target part and the first part are the same part of a face.

17 . The electronic device of claim 16 , wherein the acts further comprises:

determining a first part on a first segmented image as a beautifier target based on an identification of the first segmented image; and

determining a second part on a second segmented image which is the same as the first part and determining the second part as a target part of the to-be-edited image.

18 . A non-transitory computer readable storage medium, characterized in that the storage medium stores a computer program which, when executed by a processor, performs acts comprising:

obtaining, via an interface of an electronic device, a user image; and

performing, by the electronic device, a beautifier operation on the user image with an image beautifier model, to obtain a beautified user image;

wherein the image beautifier model is trained with a training sample pair determined based on a first sample image and a second sample image; the second sample image is an image obtained by performing an overall beautifier and adjustment of an image element on the first sample image based on a target image using an image processing model; the image element is an image style feature and/or a target part of the target image, wherein the image processing model comprises a first sub-model and a second sub-model;

wherein the performing an overall beautifier and adjustment of an image element to the first sample image based on a target image using an image processing model comprises:

performing overall beautifier on the first sample image using the first sub-model, to obtain a to-be-edited image;

obtaining a first segmented image based on the target image;

obtaining a second segmented image based on the to-be-edited image; and

obtaining the second sample image by inputting the first segmented image, the second segmented image, the target image, and the to-be-edited image into the second sub-model.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (15)