Techniques for sculpting digital faces based on anatomical modeling
Techniques are disclosed for creating digital faces. In some examples, an anatomical face model is generated from a data set including captured facial geometries of different individuals and associated bone geometries. A model generator segments each of the captured facial geometries into patches, compresses the segmented geometry associated with each patch to determine local deformation subspaces of the anatomical face model, and determines corresponding compressed anatomical subspaces of the anatomical face model. A sculpting application determines, based on sculpting input from a user, constraints for an optimization to determine parameter values associated with the anatomical face model. The parameter values can be used, along with the anatomical face model, to generate facial geometry that reflects the sculpting input.
1. A computer-implemented method for generating a digital face, the method comprising:
segmenting a plurality of facial geometries associated with a plurality of different individuals into a first plurality of patch geometries corresponding to a plurality of patches of skin;
generating, based on the segmented plurality of facial geometries, a plurality of local deformation subspaces defining how the plurality of patches deform, wherein each local deformation subspace included in the plurality of local deformation subspaces is generated based on an aggregation of a second plurality of patch geometries corresponding to a subset of the first plurality of patch geometries for a patch included in the plurality of patches;
generating, based on a plurality of bone geometries associated with the plurality of different individuals, a plurality of anatomical subspaces defining at least one of how the plurality of patches slide on underlying bones, in which directions the plurality of patches slide, or skin thicknesses associated with the plurality of patches, wherein each anatomical subspace included in the plurality of anatomical subspaces is generated based on an aggregation of a portion of the plurality of bone geometries associated with a patch included in the plurality of patches;
performing one or more optimization operations based on (i) a model that includes the plurality of local deformation subspaces and the plurality of anatomical subspaces and (ii) one or more constraints to determine parameter values associated with the model; and
generating a three-dimensional (3D) facial geometry based on the parameter values and the model.
2. The computer-implemented method of claim 1 , further comprising:
receiving, via one or more user-specified sculpting operations, one or more characteristics associated with the 3D facial geometry; and
determining the one or more constraints based on the one or more characteristics.
3. The computer-implemented method of claim 2 , wherein the one or more characteristics comprise positions of one or more points included in the 3D facial geometry.
4. The computer-implemented method of claim 2 , wherein the one or more characteristics comprise at least one of age, sex, body mass index, or ethnicity, and determining the one or more constraints comprises computing vertex displacements based on the one or more characteristics and a plurality of differential shape vectors that map changes in the one or more characteristics to changes in positions of vertices in the 3D facial geometry.
5. The computer-implemented method of claim 2 , wherein the one or more characteristics comprise at least one characteristic of a skull or jaw bone associated with the plurality of bone geometries.
6. The computer-implemented method of claim 1 , wherein generating the plurality of local deformation subspaces comprises compressing the second plurality of patch geometries within each local deformation subspace included in the plurality of local deformation subspaces to reduce a dimensionality associated with the second plurality of patch geometries.
7. The computer-implemented method of claim 1 , wherein performing the one or more optimization operations comprises performing one or more operations to minimize an energy equation that includes the one or more constraints.
8. The computer-implemented method of claim 1 , wherein performing the one or more optimization operations comprises performing one or more operations to minimize an energy equation that includes at least one term specifying 3D positional constraints over a set of constrained vertices.
9. The computer-implemented method of claim 1 , further comprising updating the model based on the parameter values.
10. The computer-implemented method of claim 1 , further comprising determining the plurality of bone geometries by deforming and fitting a template skull inside the plurality of facial geometries.
11. One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processing units, cause the one or more processing units to generate a digital face, by performing the steps of:
segmenting a plurality of facial geometries associated with a plurality of different individuals into a first plurality of patch geometries corresponding to a plurality of patches of skin;
generating, based on the segmented plurality of facial geometries, a plurality of local deformation subspaces defining how the plurality of patches deform, wherein each local deformation subspace included in the plurality of local deformation subspaces is generated based on an aggregation of a second plurality of patch geometries corresponding to a subset of the first plurality of patch geometries for a patch included in the plurality of patches;
generating, based on a plurality of bone geometries associated with the plurality of different individuals, a plurality of anatomical subspaces defining at least one of how the plurality of patches slide on underlying bones, in which directions the plurality of patches slide, or skin thicknesses associated with the plurality of patches, wherein each anatomical subspace included in the plurality of anatomical subspaces is generated based on an aggregation of a portion of the plurality of bone geometries associated with a patch included in the plurality of patches;
performing one or more optimization operations based on (i) a model that includes the plurality of local deformation subspaces and the plurality of anatomical subspaces and (ii) one or more constraints to determine parameter values associated with the model; and
generating a three-dimensional (3D) facial geometry based on the parameter values and the model.
12. The one or more non-transitory computer-readable storage media of claim 11 , wherein generating the plurality of local deformation subspaces comprises computing each local deformation subspace included in the plurality of local deformation subspaces based on an average of the second plurality of patch geometries for a corresponding patch.
13. The one or more non-transitory computer-readable storage media of claim 11 , wherein generating the plurality of local deformation subspaces comprises compressing the second plurality of patch geometries within each local deformation subspace included in the plurality of local deformation subspaces to reduce a dimensionality associated with the second plurality of patch geometries.
14. The one or more non-transitory computer-readable storage media of claim 11 , wherein the instructions that, when executed by one or more processing units, cause the one or more processing units to further perform the steps of:
receiving, via one or more user-specified sculpting operations that manipulate at least one of one or more vertices, a skin shape, or one or more bones associated with the 3D facial geometry, one or more characteristics associated with the 3D facial geometry and one or more regions of the 3D facial geometry that are affected by the one or more characteristics; and
determining the one or more constraints based on the one or more characteristics and the one or more regions of the 3D facial geometry that are affected by the one or more characteristics.
15. The one or more non-transitory computer-readable storage media of claim 14 , wherein the one or more characteristics comprise positions of one or more points included in the 3D facial geometry.
16. The one or more non-transitory computer-readable storage media of claim 14 , wherein the one or more characteristics comprise at least one of age, sex, body mass index, or ethnicity, and determining the one or more constraints comprises computing vertex displacements based on the one or more characteristics and a plurality of differential shape vectors that map changes in the one or more characteristics to changes in positions of vertices in the 3D facial geometry.
17. The one or more non-transitory computer-readable storage media of claim 11 , wherein the instructions that, when executed by one or more processing units, cause the one or more processing units to further perform the step of capturing each facial geometry included in the plurality of facial geometries with a neutral facial expression.
18. The one or more non-transitory computer-readable storage media of claim 11 , wherein the instructions that, when executed by one or more processing units, cause the one or more processing units to further perform the step of rendering at least one image using the 3D facial geometry.
19. A system, comprising:
one or more memories that include instructions; and
one or more processors that are coupled to the one or more memories and, when executing the instructions:
segment a plurality of facial geometries associated with a plurality of different individuals into a first plurality of patch geometries corresponding to a plurality of patches of skin,
generate, based on the segmented plurality of facial geometries, a plurality of local deformation subspaces defining how the plurality of patches deform, wherein each local deformation subspace included in the plurality of local deformation subspaces is generated based on an aggregation of a second plurality of patch geometries corresponding to a subset of the first plurality of patch geometries for a patch included in the plurality of patches,
generate, based on a plurality of bone geometries associated with the plurality of different individuals, a plurality of anatomical subspaces defining at least one of how the plurality of patches slide on underlying bones, in which directions the plurality of patches slide, or skin thicknesses associated with the plurality of patches, wherein each anatomical subspace included in the plurality of anatomical subspaces is generated based on an aggregation of a portion of the plurality of bone geometries associated with a patch included in the plurality of patches,
perform one or more optimization operations based on (i) a model that includes the plurality of local deformation subspaces and the plurality of anatomical subspaces and (ii) one or more constraints to determine parameter values associated with the model, and
generate a three-dimensional (3D ) facial geometry based on the parameter values and the model.
20. The system of claim 19 , wherein the one or more processors, when executing the instructions, further:
receive, via one or more user-specified sculpting operations, one or more characteristics associated with the 3D facial geometry; and
determine the one or more constraints based on the one or more characteristics.