Granted Patent B2

US 12,646,256 · App. 18/396,966 · Granted Jun 2, 2026

Inventors: Chun Wang (Chongqing, CN); Dingheng Zeng (Chongqing, CN); Xunyi Zhou (Chongqing, CN); Ning Jiang (Chongqing, CN)

Assignee: MASHANG CONSUMER FINANCE CO., LTD.

G06T17/10G06T15/04G06T15/506G06V10/54G06V10/778G06V10/82G06V20/41G06V20/46G06V20/49G06V20/70G06V40/171G06V40/172G06V40/174

Abstract

Claims (124)

1 . An information processing method, comprising:

obtaining a target video comprising a plurality of frames, each frame of the plurality of frames comprising a face image of a same object; and

for each frame of the frames,

obtaining a first target image feature by inputting the face image comprised in the each frame into a first target preset backbone model comprising a neural network model; and

obtaining a first target expression coefficient by inputting the first target image feature into a first target preset network model comprising a neural network model, wherein the first target preset network model is obtained by performing expression coefficient prediction optimization training on a first preset network model according to a shared identity coefficient and a shared texture coefficient for the same object involved in the plurality of frames,

wherein the obtaining of the target video comprises:

obtaining an initial video;

extracting the face image of each frame in the initial video;

determining the same object by analyzing the face image of each frame, and determining one or more video segments from the initial video, each of the one or more video segments includes at least two frames, and the same object appears in each of the at least two frames; and

determining one of the one or more video segments with a number of frames greater than a preset threshold as the target video.

2 . The information processing method according to claim 1 , further comprising: after obtaining the first target image feature,

obtaining a first target posture coefficient by inputting the first target image feature into a second target preset network model comprising a neural network model; and obtaining a first target lighting coefficient by inputting the first target image feature into a third target preset network model comprising a neural network model, wherein the second target preset network model is obtained by performing posture coefficient prediction optimization training on a second preset network model according to the shared identity coefficient and the shared texture coefficient, and the third target preset network model is obtained by performing lighting coefficient prediction optimization training on a third preset network model according to the shared identity coefficient and the shared texture coefficient; and

obtaining a first target identity coefficient by inputting the first target image feature into a fourth target preset network model comprising a neural network model; and obtaining a first target texture coefficient by inputting the first target image feature into a fifth target preset network model comprising a neural network model, wherein the fourth target preset network model is obtained by performing identity coefficient prediction optimization training on a fourth preset network model after determining the first target expression coefficient, the first target posture coefficient and the first target lighting coefficient, and the fifth target preset network model is obtained by performing texture coefficient prediction optimization training on a fifth preset network model after determining the first target expression coefficient, the first target posture coefficient and the first target lighting coefficient.

3 . The information processing method according to claim 2 , further comprising:

obtaining a second target identity coefficient and a second target texture coefficient of the face image comprised in another one of the frames different from the each frame;

obtaining a first target identity feature by inputting the second target identity coefficient into a second target preset backbone model comprising a neural network model;

obtaining a first target texture feature by inputting the second target texture coefficient into a third target preset backbone model comprising a neural network model;

splicing the first target image feature, the first target identity feature, and the first target texture feature together to obtain a first target feature; and

obtaining a second target expression coefficient by inputting the first target feature into a target preset head network model comprising a neural network model, wherein the target preset head network model is obtained by performing prediction optimization training on a preset head network model with a prior identity coefficient and a prior texture coefficient of the face image contained in each one of the frames different from the each frame.

4 . The information processing method according to claim 3 , further comprising: after obtaining the second target identity coefficient and the second target texture coefficient of the face image another one of the frames different from the each frame,

generating a third target loss function; and

iteratively optimizing third network parameters of a second preset backbone model, a third preset backbone model, and a preset head network model according to the third target loss function, until the third target loss function converges, to obtain the second target preset backbone model, the third target preset backbone model and the target preset head network model that have been trained;

wherein, the generating of the third target loss function comprises:

inputting the second target identity coefficient into the second preset backbone model, to obtain a first identity feature;

inputting the second target texture coefficient into the third preset backbone model, to obtain the first texture feature;

splicing the first target image feature, the first identity feature, and the first texture feature together to obtain a first feature;

inputting the first feature into the preset head network model, to obtain a second predicted expression coefficient;

generating a third predicted face three-dimensional model according to the first target identity coefficient, the first target texture coefficient, the second predicted expression coefficient, the first target posture coefficient, and the first target lighting coefficient;

obtaining a fifth difference between a second face estimated value corresponding to the third predicted face three-dimensional model and an un-occluded area in the face image;

obtaining a sixth difference between second predicted face three-dimensional key points corresponding to the third predicted face three-dimensional model and face three-dimensional key points; and

establishing the third target loss function based on the fifth difference and the sixth difference.

5 . The information processing method according to claim 4 , wherein iteratively optimizing the third network parameters of the second preset backbone model, the third preset backbone model, and the preset head network model according to the third target loss function, until the third target loss function converges, to obtain the second target preset backbone model, the third target preset backbone model and the target preset head network model that have been trained, comprises:

optimizing the third network parameters of the second preset backbone model, the third preset backbone model, and the preset head network model according to the third target loss function; and

returning to repeatedly performing the generation of the third target loss function, iteratively optimizing the third network parameters of the second preset backbone model, the third preset backbone model, and the preset head network model by using the third target loss function that is generated, until the third target loss function converges, to obtain the second target preset backbone model, the third target preset backbone model, and the target preset head network model that have been trained.

6 . The information processing method according to claim 5 , further comprising: after obtaining the first target posture coefficient and obtaining the first target lighting coefficient,

determining label geometry information and label texture information based on the shared identity coefficient and the shared texture coefficient that have been trained;

iteratively optimizing a fourth preset network model based on the label geometry information to obtain the fourth target preset network model; and

iteratively optimizing a fifth preset network model based on the label texture information, to obtain the fifth target preset network model that have been trained.

7 . The information processing method according to claim 6 , wherein iteratively optimizing the fourth preset network model based on the label geometry information to obtain the fourth target preset network model; and iteratively optimizing the fifth preset network model based on the label texture information, to obtain the fifth target preset network model that have been trained, comprises:

generating a second target loss function; and

iteratively optimizing second network parameters of the fourth preset network model and the fifth preset network model according to the second target loss function until the second target loss function converges, to obtain the fourth target preset network model and the fifth target preset network model that have been trained;

wherein the generating of the second target loss function comprises:

inputting the first target image feature into the fourth preset network model to obtain a first predicted identity coefficient; and inputting the first target image feature into the fifth preset network model to obtain a first predicted texture coefficient;

establishing a third difference based on a difference between the first predicted identity coefficient and a label identity coefficient and a difference between the first predicted texture coefficient and a label texture coefficient;

determining predicted geometry information and predicted texture information of second predicted face three-dimensional model according to the first predicted identity coefficient, the first predicted texture coefficient, the first target expression coefficient, the first target posture coefficient, and the first target lighting coefficient;

establishing a fourth difference based on a difference between the predicted geometry information and the label geometry information and a difference between the predicted texture information and the label texture information; and

establishing the second target loss function based on the third difference and the fourth difference.

8 . The information processing method according to claim 7 , wherein iteratively optimizing the second network parameters of the fourth preset network model and the fifth preset network model according to the second target loss function until the second target loss function converges, to obtain the fourth target preset network model and the fifth target preset network model that have been trained, comprises:

optimizing the second network parameters of the fourth preset network model and the fifth preset network model according to the second target loss function;

returning to repeatedly performing the generation of the second target loss function, iteratively optimizing the second network parameters of the fourth preset network model and the fifth preset network model by using the second target loss function that is generated, until the second target loss function converges, to obtain the fourth target preset network model and the fifth target preset network model that have been trained.

9 . The information processing method according to claim 8 , further comprising:

obtaining a shared identity coefficient set of shared identity coefficients respectively corresponding to respective face images of different objects and a shared texture coefficient set of shared texture coefficients respectively corresponding to the respective face images of the different objects;

obtaining one of the shared identity coefficients corresponding to the face image of the same object in the target video as the shared identity coefficient, and

obtaining one of the shared texture coefficients corresponding to the face image of the same object in the target video as the shared texture coefficient; and

performing the expression coefficient prediction optimization training on the first preset network model according to the shared identity coefficient and the shared texture coefficient, to obtain a first target preset network model.

10 . The information processing method according to claim 9 , further comprising:

generating a first target loss function; and

iteratively optimizing first network parameters of a first preset backbone model, the first preset network model, the second preset network model, the third preset network model, the shared identity coefficient, and the shared texture coefficient according to the first target loss function, until the first target loss function converges, to obtain the first target preset backbone model, the first target preset network model, the second target preset network model, the third target preset network model, the shared identity coefficient, and the shared texture coefficient that have been trained;

wherein, the generating of the first target loss function comprises:

inputting the face image of each frame into the first preset backbone model, to obtain the first image feature;

inputting the first image feature into the first preset network model to obtain a first predicted expression coefficient, inputting the first image feature into the second preset network model to obtain a first predicted posture coefficient, and inputting the first image feature into the third preset network model to obtain a first predicted lighting coefficient;

generating a first predicted face three-dimensional model according to the shared identity coefficient, the shared texture coefficient, the first predicted expression coefficient, the first predicted posture coefficient, and the first predicted lighting coefficient;

obtaining a first difference between a first face estimated value corresponding to the first predicted face three-dimensional model and an un-occluded area in the face image;

obtaining a second difference between first predicted face three-dimensional key points corresponding to the first predicted face three-dimensional model and face three-dimensional key points;

establishing a constraint relationship between the shared identity coefficient, the shared texture coefficient, and the first predicted expression coefficient through a regularization loss function; and

generating the first target loss function according to the first difference, the second difference, and the constraint relationship.

11 . The information processing method according to claim 10 , wherein iteratively optimizing the first network parameters of the first preset backbone model, the first preset network model, the second preset network model, the third preset network model, the shared identity coefficient, and the shared texture coefficient according to the first target loss function, until the first target loss function converges, to obtain the first target preset backbone model, the first target preset network model, the second target preset network model, the third target preset network model, the shared identity coefficient, and the shared texture coefficient that have been trained, comprises:

optimizing the first network parameter of the first preset backbone model, the first preset network model, the second preset network model, the third preset network model, the shared identity coefficient, and the shared texture coefficient according to the first target loss function; and

returning to repeatedly performing the generation of the first target loss function, iteratively optimizing the first network parameters of the first preset backbone model, the first preset network model, the second preset network model, the third preset network model, the shared identity coefficient, and the shared texture coefficient by using the first target loss function that is generated, until the first target loss function converges, to obtain the first target preset backbone model, the first target preset network model, the second target preset network model, the third target preset network model, the shared identity coefficient, and the shared texture coefficient that have been trained.

12 . The information processing method according to claim 1 , wherein determining the one of the one or more video segments with the number of frames greater than the preset threshold as the target video, comprises:

determining the one of the one or more video segments with the number of frames greater than the preset threshold as a first target video segment;

obtaining a second target video segment by performing a style transformation on the first target video segment; and

determining each of the first target video segment and the second target video segment as the target video.

13 . A computer device comprising:

a storage device;

at least one processor; and

the storage device storing one or more programs, which when executed by the at least one processor, cause the at least one processor to:

obtain a target video comprising a plurality of frames, each frame of the plurality of frames comprising a face image of a same object; and

for each frame of the frames,

obtain a first target image feature by inputting the face image comprised in the each frame into a first target preset backbone model comprising a neural network model; and

obtain a first target expression coefficient by inputting the first target image feature into a first target preset network model comprising a neural network model, wherein the first target preset network model is obtained by performing expression coefficient prediction optimization training on a first preset network model according to a shared identity coefficient and a shared texture coefficient for the same object involved in the plurality of frames,

wherein the obtaining of the target video comprises:

obtaining an initial video;

extracting the face image of each frame in the initial video;

determining the same object by analyzing the face image of each frame, and determining one or more video segments from the initial video, each of the one or more video segments includes at least two frames, and the same object appears in each of the at least two frames; and

determining one of the one or more video segments with a number of frames greater than a preset threshold as the target video.

14 . The computer device according to claim 13 , wherein after obtaining the first target image feature, the at least one processor is further configured to:

input the first target image feature into a second target preset network model comprising a neural network model, to obtain a first target posture coefficient; and input the first target image feature into a third target preset network model comprising a neural network model, to obtain a first target lighting coefficient, wherein the second target preset network model is obtained by performing posture coefficient prediction optimization training on a second preset network model according to the shared identity coefficient and the shared texture coefficient, and the third target preset network model is obtained by performing lighting coefficient prediction optimization training on a third preset network model according to the shared identity coefficient and the shared texture coefficient; and

input the first target image feature into a fourth target preset network model comprising a neural network model, to obtain a first target identity coefficient; and input the first target image feature into a fifth target preset network model comprising a neural network model, to obtain a first target texture coefficient, wherein the fourth target preset network model is obtained by performing identity coefficient prediction optimization training on a fourth preset network model after determining the first target expression coefficient, the first target posture coefficient and the first target lighting coefficient, and the fifth target preset network model is obtained by performing texture coefficient prediction optimization training on a fifth preset network model after determining the first target expression coefficient, the first target posture coefficient and the first target lighting coefficient.

15 . The computer device according to claim 14 , wherein the at least one processor is further configured to:

obtain a second target identity coefficient and a second target texture coefficient of the face image comprised in another one of the frames different from the each frame;

input the second target identity coefficient into a second target preset backbone model comprising a neural network model, to obtain a first target identity feature;

input the second target texture coefficient into a third target preset backbone model comprising a neural network model, to obtain a first target texture feature;

splice the first target image feature, the first target identity feature, and the first target texture feature together to obtain a first target feature; and

input the first target feature into a target preset head network model comprising a neural network model, to obtain a second target expression coefficient, wherein the target preset head network model is obtained by performing prediction optimization training on a preset head network model with a prior identity coefficient and a prior texture coefficient of the face image contained in each one of the frames different from the each frame.

16 . The computer device according to claim 15 , wherein after obtaining the second target identity coefficient and the second target texture coefficient of the face image another one of the frames different from the each frame, the at least one processor is further configured to:

generate a third target loss function;

iteratively optimize third network parameters of a second preset backbone model, a third preset backbone model, and a preset head network model according to the third target loss function, until the third target loss function converges, to obtain the second target preset backbone model, the third target preset backbone model and the target preset head network model that have been trained;

wherein the at least one processor generates the third target loss function by:

inputting the second target identity coefficient into the second preset backbone model, to obtain a first identity feature;

inputting the second target texture coefficient into the third preset backbone model, to obtain the first texture feature;

splicing the first target image feature, the first identity feature, and the first texture feature together to obtain a first feature;

inputting the first feature into the preset head network model, to obtain a second predicted expression coefficient;

generating a third predicted face three-dimensional model according to the first target identity coefficient, the first target texture coefficient, the second predicted expression coefficient, the first target posture coefficient, and the first target lighting coefficient;

obtaining a fifth difference between a second face estimated value corresponding to the third predicted face three-dimensional model and an un-occluded area in the face image;

obtaining a sixth difference between second predicted face three-dimensional key points corresponding to the third predicted face three-dimensional model and face three-dimensional key points; and

establishing the third target loss function based on the fifth difference and the sixth difference.

17 . The computer device according to claim 16 , wherein the at least one processor iteratively optimizes the third network parameters of the second preset backbone model, the third preset backbone model, and the preset head network model according to the third target loss function, until the third target loss function converges, to obtain the second target preset backbone model, the third target preset backbone model and the target preset head network model that have been trained, by:

optimizing the third network parameters of the second preset backbone model, the third preset backbone model, and the preset head network model according to the third target loss function; and

returning to repeatedly performing the generation of the third target loss function, iteratively optimizing the third network parameters of the second preset backbone model, the third preset backbone model, and the preset head network model by using the third target loss function that is generated, until the third target loss function converges, to obtain the second target preset backbone model, the third target preset backbone model, and the target preset head network model that have been trained.

18 . The computer device according to claim 17 , wherein after the first target posture coefficient and the first target lighting coefficient are obtained, the at least one processor is further configured to:

determine label geometry information and label texture information based on the shared identity coefficient and the shared texture coefficient that have been trained;

iteratively optimize a fourth preset network model based on the label geometry information to obtain the fourth target preset network model; and

iteratively optimize a fifth preset network model based on the label texture information, to obtain the fifth target preset network model that have been trained.

19 . A non-transitory storage medium having instructions stored thereon, when the instructions are executed by a processor of a computer device, the processor is caused to perform an information processing method, wherein the method comprises:

obtaining a target video comprising a plurality of frames, each frame of the plurality of frames comprising a face image of a same object; and

for each frame of the frames,

obtain a first target image feature by inputting the face image comprised in the each frame into a first target preset backbone model comprising a neural network model; and

obtaining a first target expression coefficient by inputting the first target image feature into a first target preset network model comprising a neural network model, wherein the first target preset network model is obtained by performing expression coefficient prediction optimization training on a first preset network model according to a shared identity coefficient and a shared texture coefficient for the same object involved in the plurality of frames,

wherein the obtaining of the target video comprises:

obtaining an initial video;

extracting the face image of each frame in the initial video;

determining the same object by analyzing the face image of each frame, and determining one or more video segments from the initial video, each of the one or more video segments includes at least two frames, and the same object appears in each of the at least two frames; and

determining one of the one or more video segments with a number of frames greater than a preset threshold as the target video.

Assignments (1)

Priority Claims (3)

Continuity (2)

References Cited (49)