Granted Patent B2

US 12,387,409 · App. 18/048,822 · Granted Aug 12, 2025

Inventor: Hau Nghiep Phan (Montreal, CA)

Assignee: Electronic Arts Inc.

G06T13/40G06T17/20

Abstract

Claims (50)

1. A computer-implemented method comprising:

accessing a first facial rig having a rig topology, a first set of range of motion data of facial characteristics of the first facial rig, and a first set of calculated positional coefficients for the first set of range of motion data, wherein the first set of calculated positional coefficients define positions of the facial characteristics within the first facial rig, wherein the first facial rig has a first rig identity;

receiving input scan data comprising an input facial model and a plurality of facial expressions of facial characteristics of the input model, wherein the input facial model uses the topology of the first facial rig, wherein the input facial model has a facial model identity;

calculating positional coefficients of the input facial model for each of the plurality of facial expressions;

calculating frequency separation for a defined range of the motion data of the first facial rig resulting in first frequency data and second frequency data, wherein the first frequency data is higher than the second frequency data;

calculating frequency separation on each of the plurality of facial expressions of the input model resulting in third frequency data and fourth frequency data, wherein the third frequency data is higher than the fourth frequency data;

matching portions of the second frequency data to corresponding portions of the fourth frequency data, resulting in matched frequency data;

identifying third frequency data corresponding to the matched first low frequency data;

generating a mesh based on the third frequency data;

generating a second set of calculated positional coefficients associated with the input facial model identity; and

outputting a second facial rig having the rig topology, wherein the second facial rig is based at least in part on the mesh and the second set of calculated positional coefficients, wherein the second facial rig has a second rig identity that corresponds to the facial model identity of the input scan data, wherein the second set of calculated positional coefficients define positions of the facial characteristics within the second facial rig.

2. The computer-implemented method of claim 1 , wherein the input facial model is a three dimensional model.

3. The computer-implemented method of claim 2 , wherein the three dimensional model is based on two dimensional input data of a person.

4. The computer-implemented method of claim 1 , wherein the input facial model is composed of a plurality of blendshapes.

5. The computer-implemented method of claim 4 further comprising dividing the blendshapes into regions and calculating coefficients for each region in each of the plurality of expressions.

6. The computer-implemented method of claim 1 , wherein the coefficients correspond to displacement of the input model relative to a neutral state for each region of the input model.

7. The computer-implemented method of claim 1 , wherein the plurality of facial expressions of facial characteristics of the input model are a sequence of animations.

8. The computer-implemented method of claim 1 further comprising decomposing the input scan data into direction and magnitude.

9. The computer-implemented method of claim 8 further comprising combining the direction and magnitude to generate the second facial rig.

10. The computer-implemented method of claim 1 further comprising performing principle component analysis on input scan prior to calculating frequency separation.

11. The computer-implemented method of claim 1 further comprising calculating a final mesh deformation of the plurality of expressions using matrix multiplication based at least in part on the coefficients of the input facial model.

12. The computer-implemented method of claim 1 , wherein the frequency separation is calculated using a Laplacian filter.

13. A system comprising one or more processors and non-transitory computer storage media storing instructions that when executed by the one or more processors, cause the one or more processors to perform operations comprising:

accessing a first facial rig having a rig topology, a first set of range of motion data of facial characteristics of the first facial rig, and a first set of calculated positional coefficients for the first set of range of motion data, wherein the first set of calculated positional coefficients define positions of the facial characteristics within the first facial rig, wherein the first facial rig has a first rig identity;

receiving input scan data comprising an input facial model and a plurality of facial expressions of facial characteristics of the input model, wherein the input facial model uses the topology of the first facial rig, wherein the input facial model has a facial model identity;

calculating positional coefficients of the input facial model for each of the plurality of facial expressions;

calculating frequency separation for a defined range of the motion data of the first facial rig resulting in first frequency data and second frequency data, wherein the first frequency data is higher than the second frequency data;

calculating frequency separation on each of the plurality of facial expressions of the input model resulting in third frequency data and fourth frequency data, wherein the third frequency data is higher than the fourth frequency data;

matching portions of the second frequency data to corresponding portions of the fourth frequency data, resulting in matched frequency data;

identifying third frequency data corresponding to the matched frequency data;

generating a mesh based on the third frequency data;

generating a second set of calculated positional coefficients associated with the input facial model identity; and

outputting a second facial rig having the rig topology, wherein the second facial rig is based at least in part on the mesh and the second set of calculated positional coefficients, wherein the second facial rig has a second rig identity that corresponds to the facial model identity of the input scan data, wherein the second set of calculated positional coefficients define positions of the facial characteristics within the second facial rig.

14. The system of claim 13 , wherein the input facial model is composed of a plurality of blendshapes.

15. The system of claim 14 further comprising dividing the blendshapes into regions and calculating coefficients for each region in each of the plurality of expressions.

16. The system of claim 13 , wherein the coefficients correspond to displacement of the input model relative to a neutral state for each region of the input model.

17. The system of claim 13 further comprising decomposing input scan data into direction and magnitude.

18. The system of claim 17 further comprising combining the direction and magnitude to generate the second facial rig.

19. The system of claim 13 further comprising calculating a final mesh deformation of the plurality of expressions using matrix multiplication based at least in part on the coefficients of the input facial model.

20. A non-transitory computer storage medium storing instructions that when executed by one or more processors, cause the one or more processors to perform operations comprising:

accessing a first facial rig having a rig topology, a first set of range of motion data of facial characteristics of the first facial rig, and a first set of calculated positional coefficients for the first set of range of motion data, wherein the first set of calculated positional coefficients define positions of the facial characteristics within the first facial rig, wherein the first facial rig has a first rig identity;

receiving input scan data comprising an input facial model and a plurality of facial expressions of facial characteristics of the input model, wherein the input facial model uses the topology of the first facial rig, wherein the input facial model has a facial model identity;

calculating positional coefficients of the input facial model for each of the plurality of facial expressions;

calculating frequency separation for a defined range of the motion data of the first facial rig resulting in first frequency data and second frequency data, wherein the first frequency data is higher than the second frequency data;

calculating frequency separation on each of the plurality of facial expressions of the input model resulting in third frequency data and fourth frequency data, wherein the third frequency data is higher than the fourth frequency data;

matching portions of the second frequency data to corresponding portions of the fourth frequency data, resulting in matched frequency data;

identifying third frequency data corresponding to the matched frequency data;

generating a mesh based on the third frequency data;

generating a second set of calculated positional coefficients associated with the input facial model identity; and

outputting a second facial rig having the rig topology, wherein the second facial rig is based at least in part on the mesh and the second set of calculated positional coefficients, wherein the second facial rig has a second rig identity that corresponds to the facial model identity of the input scan data, wherein the second set of calculated positional coefficients define positions of the facial characteristics within the second facial rig.

Assignments (1)

Continuity (2)

References Cited (297)