Granted Patent B2

US 12,620,260 · App. 18/396,971 · Granted May 5, 2026

Inventors: Chun Wang (Chongqing, CN); Dingheng Zeng (Chongqing, CN); Xunyi Zhou (Chongqing, CN); Ning Jiang (Chongqing, CN)

Assignee: MASHANG CONSUMER FINANCE CO., LTD.

G06V40/168G06T17/00G06V10/54G06V40/174

Abstract

Claims (114)

1 . An information processing method, comprising:

obtaining a target video comprising a plurality of frames, each of the plurality of frames comprising a face image corresponding to a same object;

obtaining a first target image feature corresponding to the face image of each frame;

determining a first target identity coefficient and a first target texture coefficient;

determining a first target identity feature based on the first target identity coefficient;

determining a first target texture feature based on the first target texture coefficient;

splicing the first target image feature, the first target identity feature, and the first target texture feature, and obtaining a first target feature; and

determining a first target expression coefficient based on the first target feature.

2 . The information processing method according to claim 1 , wherein determining the first target identity coefficient and the first target texture coefficient comprises:

obtaining a first identity coefficient and a first texture coefficient of the face image corresponding to the first target image feature in a previous frame of the target video;

obtaining a second identity coefficient and a second texture coefficient corresponding to the first target image feature;

performing a weighted summation on the first identity coefficient and the second identity coefficient, and obtaining the first target identity coefficient corresponding to the first target image feature;

performing the weighted summation on the first texture coefficient and the second texture coefficient, and obtaining the first target texture coefficient corresponding to the first target image feature.

3 . The information processing method according to claim 2 , wherein after determining the first target expression coefficient, the method further comprises:

using the first target identity coefficient to replace the second identity coefficient corresponding to the first target image feature in the face image of a current frame in the target video;

using the first target texture coefficient to replace the second texture coefficient corresponding to the first target image feature in the face image of the current frame in the target video.

4 . The information processing method according to claim 3 , wherein obtaining the target video comprises:

acquiring an initial video;

extracting the face image of each frame in the initial video;

determining the same object by analyzing the face image of each frame, and determining one or more video segments from the initial video, each of the one or more video segments comprising at least two frames and the same object comprised in each of the at least two frames;

determining one of the one or more video segments with a number of frames greater than a preset threshold as the target video.

5 . The information processing method according to claim 4 , wherein determining one of the one or more video segments with the number of frames greater than the preset threshold as the target video comprises:

determining the one of the one or more video segments with the number of frames greater than the preset threshold as a first target video segment;

obtaining a second target video segment by performing a style transformation on the first target video segment; and

determining each of the first target video segment and the second target video segment as the target video.

6 . The information processing method according to claim 5 , wherein after determining the first target identity coefficient and the first target texture coefficient, further comprises:

generating a first target loss function, comprising:

inputting the first target identity coefficient into a second preset backbone model, and outputting a first identity feature;

inputting the first target texture coefficient into a third preset backbone model, and outputting a first texture feature;

splicing the first target image feature, the first identity feature, and the first texture feature, and obtaining a first feature;

inputting the first feature into a preset head network model, and outputting a first predicted expression coefficient;

generating a first predicted face three-dimensional model according to a label identity coefficient, a label texture coefficient, the first predicted expression coefficient, a label posture coefficient, and a label lighting coefficient;

obtaining a first difference between a first face estimated value corresponding to the first predicted face three-dimensional model and an un-occluded area in the face image;

obtaining a second difference between first predicted face three-dimensional key points corresponding to the first predicted face three-dimensional model and face three-dimensional key points in the face image; and

establishing the first target loss function based on the first difference and the second difference.

7 . The information processing method according to claim 6 , further comprising:

performing an optimization on a first network parameters of the second preset backbone model, the third preset backbone model, and the preset head network model according to the first target loss function;

returning to repeatedly generate the first target loss function, iteratively optimizing the first network parameters of the second preset backbone model, the third preset backbone model, and the preset head network model through the first target loss function that is generated, until the first target loss function converges, and obtaining a second target preset backbone model, a third target preset backbone model, and a target preset head network model that have been trained.

8 . A computer device comprising:

a storage device;

at least one processor; and

the storage device storing one or more programs, which when executed by the at least one processor, cause the at least one processor to:

obtain a target video comprising a plurality of frames, each of the plurality of frames comprising a face image corresponding to a same object;

obtain a first target image feature corresponding to the face image of each frame;

determine a first target identity coefficient and a first target texture coefficient;

determine a first target identity feature based on the first target identity coefficient;

determine a first target texture feature based on the first target texture coefficient;

splice the first target image feature, the first target identity feature, and the first target texture feature, and obtain a first target feature; and

determine a first target expression coefficient based on the first target feature.

9 . The computer device according to claim 8 , wherein the at least one processor determines the first target identity coefficient and the first target texture coefficient by:

obtaining a first identity coefficient and a first texture coefficient of the face image corresponding to the first target image feature in a previous frame of the target video;

obtaining a second identity coefficient and a second texture coefficient corresponding to the first target image feature;

performing a weighted summation on the first identity coefficient and the second identity coefficient, and obtaining the first target identity coefficient corresponding to the first target image feature;

performing the weighted summation on the first texture coefficient and the second texture coefficient, and obtaining the first target texture coefficient corresponding to the first target image feature.

10 . The computer device according to claim 9 , wherein after determining the first target expression coefficient, the at least one processor is further caused to:

use the first target identity coefficient to replace the second identity coefficient corresponding to the first target image feature in the face image of a current frame in the target video;

use the first target texture coefficient to replace the second texture coefficient corresponding to the first target image feature in the face image of the current frame in the target video.

11 . The computer device according to claim 10 , wherein the at least one processor obtains the target video by:

acquiring an initial video;

extracting the face image of each frame in the initial video;

determining the same object by analyzing the face image of each frame, and determining one or more video segments from the initial video, each of the one or more video segments comprising at least two frames and the same object comprised in each of the at least two frames;

determining one of the one or more video segments with a number of frames greater than a preset threshold as the target video.

12 . The computer device according to claim 11 , wherein the at least one processor determines one of the one or more video segments with the number of frames greater than the preset threshold as the target video by:

determining the one of the one or more video segments with the number of frames greater than the preset threshold as a first target video segment;

obtaining a second target video segment by performing a style transformation on the first target video segment; and

determining each of the first target video segment and the second target video segment as the target video.

13 . The computer device according to claim 12 , wherein after determining the first target identity coefficient and the first target texture coefficient, the at least one processor is further caused to:

generate a first target loss function, comprising:

input the first target identity coefficient into a second preset backbone model, and output a first identity feature;

input the first target texture coefficient into a third preset backbone model, and output a first texture feature;

splice the first target image feature, the first identity feature, and the first texture feature, and obtain a first feature;

input the first feature into a preset head network model, and output a first predicted expression coefficient;

generate a first predicted face three-dimensional model according to a label identity coefficient, a label texture coefficient, the first predicted expression coefficient, a label posture coefficient, and a label lighting coefficient;

obtain a first difference between a first face estimated value corresponding to the first predicted face three-dimensional model and an un-occluded area in the face image;

obtain a second difference between first predicted face three-dimensional key points corresponding to the first predicted face three-dimensional model and face three-dimensional key points in the face image; and

establish the first target loss function based on the first difference and the second difference.

14 . The computer device according to claim 13 , the at least one processor is further caused to:

perform an optimization on a first network parameters of the second preset backbone model, the third preset backbone model, and the preset head network model according to the first target loss function;

return to repeatedly generate the first target loss function, iteratively optimizing the first network parameters of the second preset backbone model, the third preset backbone model, and the preset head network model through the first target loss function that is generated, until the first target loss function converges, and obtaining a second target preset backbone model, a third target preset backbone model, and a target preset head network model that have been trained.

15 . A non-transitory storage medium having instructions stored thereon, when the instructions are executed by a processor of a computer device, the processor is caused to perform an information processing method, wherein the method comprises:

obtaining a target video comprising a plurality of frames, each of the plurality of frames comprising a face image corresponding to a same object;

obtaining a first target image feature corresponding to the face image of each frame;

determining a first target identity coefficient and a first target texture coefficient;

determining a first target identity feature based on the first target identity coefficient;

determining a first target texture feature based on the first target texture coefficient;

splicing the first target image feature, the first target identity feature, and the first target texture feature, and obtaining a first target feature; and

determining a first target expression coefficient based on the first target feature.

16 . The non-transitory storage medium according to claim 15 , wherein determining the first target identity coefficient and the first target texture coefficient comprises:

obtaining a first identity coefficient and a first texture coefficient of the face image corresponding to the first target image feature in a previous frame of the target video;

obtaining a second identity coefficient and a second texture coefficient corresponding to the first target image feature;

performing a weighted summation on the first identity coefficient and the second identity coefficient, and obtaining the first target identity coefficient corresponding to the first target image feature;

performing the weighted summation on the first texture coefficient and the second texture coefficient, and obtaining the first target texture coefficient corresponding to the first target image feature.

17 . The non-transitory storage medium according to claim 16 , wherein after determining the first target expression coefficient, the method further comprises:

using the first target identity coefficient to replace the second identity coefficient corresponding to the first target image feature in the face image of a current frame in the target video;

using the first target texture coefficient to replace the second texture coefficient corresponding to the first target image feature in the face image of the current frame in the target video.

18 . The non-transitory storage medium according to claim 17 , wherein obtaining the target video comprises:

acquiring an initial video;

extracting the face image of each frame in the initial video;

determining the same object by analyzing the face image of each frame, and determining one or more video segments from the initial video, each of the one or more video segments comprising at least two frames and the same object comprised in each of the at least two frames;

determining one of the one or more video segments with a number of frames greater than a preset threshold as the target video.

19 . The non-transitory storage medium according to claim 18 , wherein determining one of the one or more video segments with the number of frames greater than the preset threshold as the target video comprises:

determining the one of the one or more video segments with the number of frames greater than the preset threshold as a first target video segment;

obtaining a second target video segment by performing a style transformation on the first target video segment; and

determining each of the first target video segment and the second target video segment as the target video.

20 . The non-transitory storage medium according to claim 19 , wherein after determining the first target identity coefficient and the first target texture coefficient, the method further comprises:

generating a first target loss function, comprising:

inputting the first target identity coefficient into a second preset backbone model, and outputting a first identity feature;

inputting the first target texture coefficient into a third preset backbone model, and outputting a first texture feature;

splicing the first target image feature, the first identity feature, and the first texture feature, and obtaining a first feature;

inputting the first feature into a preset head network model, and outputting a first predicted expression coefficient;

generating a first predicted face three-dimensional model according to a label identity coefficient, a label texture coefficient, the first predicted expression coefficient, a label posture coefficient, and a label lighting coefficient;

obtaining a first difference between a first face estimated value corresponding to the first predicted face three-dimensional model and an un-occluded area in the face image;

obtaining a second difference between first predicted face three-dimensional key points corresponding to the first predicted face three-dimensional model and face three-dimensional key points in the face image; and

establishing the first target loss function based on the first difference and the second difference.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (45)