Granted Patent B2

US 12,548,274 · App. 18/581,670 · Granted Feb 10, 2026

Inventors: Zijia Wang (London, GB); Zhisong Liu (Shenzhen, CN); Zhen Jia (Shanghai, CN)

Assignee: Dell Products L.P.

G06T19/20G06T7/40G06T7/50G06T7/73G06T17/00G06T2207/20081G06T2207/20084G06T2207/30201G06T2219/2024G06V10/774G06V10/776G06V40/174

Abstract

Claims (72)

1 . A method comprising:

acquiring an input image containing a two-dimensional (2D) object;

determining a shape feature, a texture feature, and a posture feature of the 2D object;

generating a rendered image based on the shape feature, the texture feature, the posture feature, and the input image;

generating a 3D object reconstruction model based on the input image and the rendered image, wherein the 3D object reconstruction model is configured to generate a 3D object from the 2D object utilizing a weighted combination of multiple distinct loss functions including at least one loss function that is generated based on (i) estimated expression coefficients of the 3D object reconstruction model and (ii) an output of an expression classifier of the 3D object reconstruction model; and

tuning the 3D object reconstruction model according to a parameter tuning model for stylization.

2 . The method according to claim 1 , wherein determining a shape feature, a texture feature, and a posture feature of the 2D object comprises:

determining an average object shape feature, and an identity feature and an expression feature of the 2D object; and

determining the shape feature based on the average object shape feature, the identity feature, and the expression feature as determined.

3 . The method according to claim 1 , wherein determining a shape feature, a texture feature, and a posture feature of the 2D object comprises:

determining an average object texture feature and a constant feature of the 2D object; and

determining the texture feature based on the average object texture feature and the constant feature of the 2D object as determined.

4 . The method according to claim 1 , wherein determining a shape feature, a texture feature, and a posture feature of the 2D object comprises:

determining a rotation feature, a scaling feature, and a translation feature of the 2D object; and

determining the posture feature based on the rotation feature, the scaling feature, and the translation feature as determined.

5 . The method according to claim 1 , wherein generating the 3D object reconstruction model comprises:

determining a reconstruction loss for measuring a pixel intensity difference, an expression loss for measuring an emotional expression, and a regularization loss for measuring object plausibility based on the shape feature, the texture feature, and the posture feature; and

generating the 3D object reconstruction model based on the reconstruction loss, the expression loss, and the regularization loss.

6 . The method according to claim 5 , wherein determining a reconstruction loss for measuring a pixel intensity difference, an expression loss for measuring an emotional expression, and a regularization loss for measuring object plausibility comprises:

determining input features of the input image and rendered features of the rendered image; and

generating the reconstruction loss based on the input image, the rendered image, the input features, and the rendered features.

7 . The method according to claim 5 , wherein determining a reconstruction loss for measuring a pixel intensity difference, an expression loss for measuring an emotional expression, and a regularization loss for measuring object plausibility comprises:

generating an emotional expression coefficient based on the shape feature, the texture feature, and the posture feature;

extracting a real expression label of the 2D object; and

generating the expression loss based on the emotional expression coefficient and the real expression label.

8 . The method according to claim 5 , wherein determining a reconstruction loss for measuring a pixel intensity difference, an expression loss for measuring an emotional expression, and a regularization loss for measuring object plausibility comprises:

determining an average shape feature, an average texture feature, and an average posture feature based on average features of the object; and

generating the regularization loss based on the average shape feature, the average texture feature, the average posture feature, the shape feature, the texture feature, and the posture feature.

9 . The method according to claim 1 , wherein tuning the 3D object reconstruction model comprises:

taking the input image, the rendered image, and a style label of the parameter tuning model as observed variables; and

tuning the 3D object reconstruction model based on a lower bound of log marginal likelihood of the observed variables.

10 . The method according to claim 1 , further comprising:

reconstructing the 2D object into a 3D object based on the tuned 3D object reconstruction model.

11 . An electronic device, comprising:

at least one processor; and

memory coupled to the at least one processor and having instructions stored therein, wherein the instructions, when executed by the at least one processor, cause the electronic device to perform actions comprising:

acquiring an input image containing a two-dimensional (2D) object;

determining a shape feature, a texture feature, and a posture feature of the 2D object;

generating a rendered image based on the shape feature, the texture feature, the posture feature, and the input image;

generating a 3D object reconstruction model based on the input image and the rendered image, wherein the 3D object reconstruction model is configured to generate a 3D object from the 2D object utilizing a weighted combination of multiple distinct loss functions including at least one loss function that is generated based on (i) estimated expression coefficients of the 3D object reconstruction model and (ii) an output of an expression classifier of the 3D object reconstruction model; and

tuning the 3D object reconstruction model according to a parameter tuning model for stylization.

12 . The electronic device according to claim 11 , wherein determining a shape feature, a texture feature, and a posture feature of the 2D object comprises:

determining an average object shape feature, and an identity feature and an expression feature of the 2D object; and

determining the shape feature based on the average object shape feature, the identity feature, and the expression feature as determined.

13 . The electronic device according to claim 11 , wherein determining a shape feature, a texture feature, and a posture feature of the 2D object comprises:

determining an average object texture feature and a constant feature of the 2D object; and

determining the texture feature based on the average object texture feature and the constant feature of the 2D object as determined.

14 . The electronic device according to claim 11 , wherein determining a shape feature, a texture feature, and a posture feature of the 2D object comprises:

determining a rotation feature, a scaling feature, and a translation feature of the 2D object; and

determining the posture feature based on the rotation feature, the scaling feature, and the translation feature as determined.

15 . The electronic device according to claim 11 , wherein generating the 3D object reconstruction model comprises:

determining a reconstruction loss for measuring a pixel intensity difference, an expression loss for measuring an emotional expression, and a regularization loss for measuring object plausibility based on the shape feature, the texture feature, and the posture feature; and

generating the 3D object reconstruction model based on the reconstruction loss, the expression loss, and the regularization loss.

16 . The electronic device according to claim 15 , wherein determining a reconstruction loss for measuring a pixel intensity difference, an expression loss for measuring an emotional expression, and a regularization loss for measuring object plausibility comprises:

determining input features of the input image and rendered features of the rendered image; and

generating the reconstruction loss based on the input image, the rendered image, the input features, and the rendered features.

17 . The electronic device according to claim 15 , wherein determining a reconstruction loss for measuring a pixel intensity difference, an expression loss for measuring an emotional expression, and a regularization loss for measuring object plausibility comprises:

generating an emotional expression coefficient based on the shape feature, the texture feature, and the posture feature;

extracting a real expression label of the 2D object; and

generating the expression loss based on the emotional expression coefficient and the real expression label.

18 . The electronic device according to claim 15 , wherein determining a reconstruction loss for measuring a pixel intensity difference, an expression loss for measuring an emotional expression, and a regularization loss for measuring object plausibility comprises:

determining an average shape feature, an average texture feature, and an average posture feature based on average features of the object; and

generating the regularization loss based on the average shape feature, the average texture feature, the average posture feature, the shape feature, the texture feature, and the posture feature.

19 . The electronic device according to claim 11 , wherein tuning the 3D object reconstruction model comprises:

taking the input image, the rendered image, and a style label of the parameter tuning model as observed variables; and

tuning the 3D object reconstruction model based on a lower bound of log marginal likelihood of the observed variables.

20 . A computer program product comprising a non-transitory computer-readable medium having machine-executable instructions stored therein, wherein the machine-executable instructions, when executed by a machine, cause the machine to perform actions comprising:

acquiring an input image containing a two-dimensional (2D) object;

determining a shape feature, a texture feature, and a posture feature of the 2D object;

generating a rendered image based on the shape feature, the texture feature, the posture feature, and the input image;

generating a 3D object reconstruction model based on the input image and the rendered image, wherein the 3D object reconstruction model is configured to generate a 3D object from the 2D object utilizing a weighted combination of multiple distinct loss functions including at least one loss function that is generated based on (i) estimated expression coefficients of the 3D object reconstruction model and (ii) an output of an expression classifier of the 3D object reconstruction model; and

tuning the 3D object reconstruction model according to a parameter tuning model for stylization.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (29)