Dynamic mesh-object attachment system for 3D models
A computing device for displaying a 3D virtual object for an augmented reality (AR) scene is described. The computing device has a processor and a non-transitory computer-readable memory, where the processor is configured to carry out instructions from the memory that configure the computing device to: determine a polygon mesh representation of a target object within the AR scene, where the polygon mesh representation models a physical appearance of the target object; identify a target mesh surface of the polygon mesh representation onto which the 3D virtual object should be anchored; monitor the target object and updating the polygon mesh representation according to changes in the physical appearance of the target object; and display the 3D virtual object within the AR scene according to a position of the target mesh surface on the updated polygon mesh representation, where the target mesh surface moves along with the target object.
1 . A computing device for displaying a 3D virtual object for an augmented reality (AR) scene, the computing device comprising a processor and a non-transitory computer-readable memory, wherein the processor is configured to carry out instructions from the memory that configure the computing device to:
determine a polygon mesh representation of a target object within the AR scene, wherein the polygon mesh representation models a physical appearance of the target object;
identify a target mesh surface of the polygon mesh representation onto which the 3D virtual object should be anchored;
monitor the target object and updating the polygon mesh representation according to changes in the physical appearance of the target object;
perform a UV mapping of a texture onto the 3D virtual object according to an orientation of the 3D virtual object, wherein the UV mapping is used to adjust a surface of the 3D virtual object to match the polygon mesh representation of the target object; and
display the 3D virtual object within the AR scene according to a position of the target mesh surface on the updated polygon mesh representation, wherein the target mesh surface moves along with the target object.
2 . The computing device of claim 1 , wherein the target object is at least a portion of a subject in one of a captured image or a 3D avatar.
3 . The computing device of claim 2 , wherein the target object is a head, torso, arm, leg, or finger of the subject.
4 . The computing device of claim 2 , wherein the 3D virtual object is a facial feature decoration for a face of the subject.
5 . The computing device of claim 2 , wherein the 3D virtual object comprises one of a virtual clothing piece or a virtual make-up feature.
6 . The computing device of claim 1 , wherein:
the polygon mesh representation comprises a plurality of vertices and edges between the plurality of vertices that form polygons; and
the target mesh surface comprises one of a vertex normal from the polygon mesh representation or a normal vector of a face of the polygon mesh representation.
7 . The computing device of claim 6 , wherein the target object is a head of a subject and the target mesh surface corresponds to a moveable facial feature of the subject.
8 . The computing device of claim 7 , wherein the target mesh surface corresponds to one of a chin, eyelid, eyebrow, lip, jaw, cheek, or nose of the subject.
9 . The computing device of claim 1 , wherein displaying the 3D virtual object comprises rotating the 3D virtual object according to the position of the target mesh surface and an orientation of the target mesh surface.
10 . The computing device of claim 1 , wherein performing the UV mapping comprises adjusting a Vector 2 field associated with a displayed position of the texture on the 3D virtual object.
11 . The computing device of claim 1 , wherein performing the UV mapping comprises snapping a UV mapping value within the UV mapping to a maximum value of a range of the UV mapping.
12 . A method for displaying a 3D virtual object for an augmented reality (AR) scene, the method comprising:
determining a polygon mesh representation of a target object within the AR scene, wherein the polygon mesh representation models a physical appearance of the target object;
identifying a target mesh surface of the polygon mesh representation onto which the 3D virtual object should be anchored;
monitoring the target object and updating the polygon mesh representation according to changes in the physical appearance of the target object;
performing a UV mapping of a texture onto the 3D virtual object according to an orientation of the 3D virtual object, wherein the UV mapping is used to adjust a surface of the 3D virtual object to match the polygon mesh representation of the target object; and
displaying the 3D virtual object within the AR scene according to a position of the target mesh surface on the updated polygon mesh representation, wherein the target mesh surface moves along with the target object.
13 . The method of claim 12 , wherein the target object is at least a portion of a subject in one of a captured image or a 3D avatar.
14 . The method of claim 13 , wherein the target object is a head, torso, arm, leg, or finger of the subject.
15 . The method of claim 13 , wherein the 3D virtual object is a facial feature decoration for a face of the subject.
16 . The method of claim 13 , wherein the 3D virtual object comprises one of a virtual clothing piece or a virtual make-up feature.
17 . The method of claim 12 , wherein:
the polygon mesh representation comprises a plurality of vertices and edges between the plurality of vertices that form polygons; and
the target mesh surface comprises one of a vertex normal from the polygon mesh representation or a normal vector of a face of the polygon mesh representation.
18 . The method of claim 17 , wherein the target object is a head of a subject and the target mesh surface corresponds to a moveable facial feature of the subject.
19 . The method of claim 18 , wherein the target mesh surface corresponds to one of a chin, eyelid, eyebrow, lip, jaw, cheek, or nose of the subject.
20 . A computer readable non-transitory recording medium storing computer-executable program instructions that when executed by a processor cause a computer to execute displaying of a 3D virtual object for an augmented reality (AR) scene, the operation, comprising:
determining a polygon mesh representation of a target object within the AR scene, wherein the polygon mesh representation models a physical appearance of the target object;
identifying a target mesh surface of the polygon mesh representation onto which the 3D virtual object should be anchored;
monitoring the target object and updating the polygon mesh representation according to changes in the physical appearance of the target object;
performing a UV mapping of a texture onto the 3D virtual object according to an orientation of the 3D virtual object, wherein the UV mapping is used to adjust a surface of the 3D virtual object to match the polygon mesh representation of the target object; and
displaying the 3D virtual object within the AR scene according to a position of the target mesh surface on the updated polygon mesh representation, wherein the target mesh surface moves along with the target object.