Matching meshes for virtual avatars
Examples of systems and methods for matching a base mesh to a target mesh for a virtual avatar or object are disclosed. The systems and methods may be configured to automatically match a base mesh of an animation rig to a target mesh, which may represent a particular pose of the virtual avatar or object. Base meshes may be obtained by manipulating an avatar or object into a particular pose, while target meshes may be obtain by scanning, photographing, or otherwise obtaining information about a person or object in the particular pose. The systems and methods may automatically match a base mesh to a target mesh using rigid transformations in regions of higher error and non-rigid deformations in regions of lower error.
1. A method for matching a base mesh to a target mesh, the method comprising:
obtaining, using computing equipment, the base and target meshes, each of which comprises a plurality of vertices;
matching, using the computing equipment, the base mesh to the target mesh by:
(i) applying at least one iteration of a first type of transformation to at least some of the vertices of the base mesh until distance differences between the base and target meshes are below a first threshold; and
(ii) applying at least one iteration of a second type of transformation to at least some of the vertices of the base mesh until the distance differences between the base and target meshes are below a second threshold, wherein the second threshold is smaller than the first threshold; and
providing, using the computing equipment, an output based at least on the base mesh with transformed vertices,
wherein the first type of transformation comprises a rigid transformation of at least some of the vertices of the base mesh towards corresponding vertices of the target mesh, and
wherein applying at least one iteration of the first type of transformation comprises:
identifying a grouping of vertices of the base mesh that have distance differences above the first threshold, wherein a given vertex in the grouping of vertices has a largest distance difference amongst the grouping;
determining a size of a falloff region based at least in part on a magnitude of a largest distance difference;
applying the rigid transformation for the vertices of the base mesh in the falloff region; and
feathering the rigid transformation for the vertices of the base mesh outside of the falloff region.
2. The method of claim 1 , wherein the rigid transformation comprises a rigid nearest-neighbor transformation.
3. The method of claim 1 , wherein the second type of transformation comprises a non-rigid deformation of at least some of the vertices of the base mesh towards corresponding vertices of the target mesh.
4. The method of claim 3 , wherein the non-rigid deformation comprises a closest point on the surface (CPOS) deformation.
5. The method of claim 1 , wherein applying at least one iteration of the second type of transformation until the distance differences are below the second threshold comprises:
(a) applying the second type of transformation to at least some of the vertices of the base mesh;
(b) determining the distance differences between the vertices of the base mesh relative to corresponding vertices of the target mesh; and
iterating operations (a) and (b) until the distance differences have a mean error below the second threshold.
6. The method of claim 1 , wherein applying at least one iteration of the second type of transformation until the distance differences are below the second threshold comprises:
(1) applying the second type of transformation to at least some of the vertices of the base mesh;
(2) determining the distance differences between the vertices of the base mesh relative to corresponding vertices of the target mesh; and
iterating operations (1) and (2) until a maximum error in the distance differences is below the second threshold.
7. The method of claim 1 , wherein applying at least one iteration of the second type of transformation until the distance differences are below the second threshold comprises:
(A) applying the second type of transformation to at least some of the vertices of the base mesh;
(B) determining the distance differences between the vertices of the base mesh relative to corresponding vertices of the target mesh; and
iterating operations (A) and (B) until a maximum error in the distance differences is less than 0.5 cm.
8. The method of claim 1 , wherein applying at least one iteration of the first type of transformation comprises: determining the falloff region.
9. A system for matching a base mesh to a target mesh, the system comprising:
non-transitory computer storage configured to store the base and target meshes, each of which comprises a plurality of vertices; and
a hardware processor configured to match the base mesh to the target mesh, the hardware processor in communication with the non-transitory computer storage and the hardware processor programmed to:
(i) apply at least one iteration of a first type of transformation to at least some of the vertices of the base mesh until distance differences between the base and target meshes are below a first threshold;
(ii) apply at least one iteration of a second type of transformation to at least some of the vertices of the base mesh until the distance differences between the base and target meshes are below a second threshold, wherein the second threshold is smaller than the first threshold; and
(iii) provide an output based at least on the base mesh with transformed vertices,
wherein the first type of transformation comprises a rigid transformation of at least some of the vertices of the base mesh towards corresponding vertices of the target mesh, and
wherein applying at least one iteration of the first type of transformation comprises:
identifying a grouping of vertices of the base mesh that have distance differences above the first threshold, wherein a given vertex in the grouping of vertices has a largest distance difference amongst the grouping;
determining a size of a falloff region based at least in part on a magnitude of a largest distance difference;
applying the rigid transformation for the vertices of the base mesh in the falloff region; and
feathering the rigid transformation for the vertices of the base mesh outside of the falloff region.
10. The system of claim 9 , wherein the rigid transformation comprises a rigid nearest-neighbor transformation.
11. The system of claim 9 , wherein the second type of transformation comprises a non-rigid deformation of at least some of the vertices of the base mesh towards corresponding vertices of the target mesh.
12. The system of claim 11 , wherein the non-rigid deformation comprises a closest point on the surface (CPOS) deformation.
13. The system of claim 9 , wherein the hardware processor is programmed to:
(a) apply the second type of transformation to at least some of the vertices of the base mesh;
(b) determine the distance differences between the vertices of the base mesh relative to corresponding vertices of the target mesh; and
iterate operations (a) and (b) until the distance differences have a mean error below the second threshold.
14. The system of claim 9 , wherein the hardware processor is programmed to:
(1) apply the second type of transformation to at least some of the vertices of the base mesh;
(2) determine the distance differences between the vertices of the base mesh relative to corresponding vertices of the target mesh; and
iterate operations (1) and (2) until a maximum error in the distance differences is below the second threshold.
15. The system of claim 9 , wherein the hardware processor is programmed to:
(A) apply the second type of transformation to at least some of the vertices of the base mesh;
(B) determine the distance differences between the vertices of the base mesh relative to corresponding vertices of the target mesh; and
iterate operations (A) and (B) until a maximum error in the distance differences is less than 0.5 cm.
16. The system of claim 9 , and wherein the hardware processor is programmed to: determine the falloff region.