System and method for mesh stabilization of facial motion capture data
View Patent ↗A method and system for removing head motion from facial motion capture data. The method includes receiving a set of measured points of a target model, wherein each point is associated with coordinates in a 3D space. The method includes computing an optimal affine transformation function. The computing includes selecting an unprocessed point from the set of measured points. The computing includes selecting two nearby neighboring points of the unprocessed point. The computing includes computing an affine transformation function that minimizes an L2-norm error. The computing includes identifying the optimal affine transformation function from a set of computed affine transformation functions. The method includes displaying an aligned target model and reference model utilizing the optimal affine transformation function. The method includes outputting the optimal affine function to a computer-readable storage medium.
1. A processor-implemented method for reducing error between motion capture sessions, comprising:
receiving a first and a second set of measured points, the first set of measured points from a first motion capture session and associated with a first set of facial skin markers of a target model in a first facial animation sequence of the target model, the second set of measured points from a second motion capture session and associated with a second set of facial markers of the target model in a second facial animation sequence of the target model, wherein each measured point is associated with a three-dimensional coordinate triple;
computing, by a computer system including a processor, an optimal affine transformation function for reducing error between the motion capture sessions, wherein the computing includes,
selecting an unprocessed point from the first set of measured points, the unprocessed point being a point of the first set of measured points that has not been used to compute any affine transformation function to minimize an error,
selecting two nearby neighboring points of the unprocessed point,
computing, as part of aligning the first set of measured points with the second set of measured points to reduce error between the motion capture sessions, an affine transformation function that minimizes an L 2 -norm error, and
identifying the optimal affine transformation function from a set of computed affine transformation functions; and
generating, by the computer system and utilizing the optimal affine transformation function, a head motion animation sequence.
2. The method of claim 1 , wherein the target model is a human actor and the measured points are measured from a video recording of marked points on the human actor's face, the marked points being associated with the facial markers.
3. The method of claim 2 , wherein the measured points vary with time and represent the facial animation sequence.
4. The method of claim 1 , wherein the affine transformation function is computed with Horn's solution to minimize the L 2 -norm error.
5. The method of claim 1 , wherein the optimal affine transformation function is computed with a RANSAC technique utilizing the unprocessed point and the two nearby neighboring points.
6. The method of claim 5 , wherein the unprocessed point is randomly selected from those points of the first set of measured points that have not been used to compute any affine transformation function to minimize an error.
7. The method of claim 1 , further comprising:
receiving a user-specified threshold, wherein the optimal affine transformation function produces at least one of: a minimum overall alignment error or an overall alignment error below the user-specified threshold.
8. The method of claim 1 , further comprising:
receiving a set of user-specified reliable measured points, wherein the unprocessed point is selected from those points of the set of user-specified reliable measured points that have not been used to compute any affine transformation function to minimize an error.
9. A server system for reducing error between motion capture sessions, comprising:
a computer-readable storage medium, and
a processor in communication with the computer-readable storage medium, the processor configured to execute a method including,
receiving a first and a second set of measured points, the first set of measured points from a first motion capture session and associated with a first set of facial skin markers of a target model in a first facial animation sequence of the target model, the second set of measured points from a second motion capture session and associated with a second set of facial markers of the target model in a second facial animation sequence of the target model, wherein each measured point is associated with a three-dimensional coordinate triple,
computing an optimal affine transformation function for reducing error between motion capture sessions, wherein the computing includes,
selecting an unprocessed point from the first set of measured points, the unprocessed point being a point of the first set of measured points that has not been used to compute any affine transformation function to minimize an error,
selecting two nearby neighboring points of the unprocessed point,
computing, as part of aligning the first set of measured points with the second set of measured points to reduce error between motion capture sessions, an affine transformation function that minimizes an L 2 -norm error, and
identifying the optimal affine transformation function from a set of computed affine transformation functions; and
generating, utilizing the optimal affine transformation function, a head motion animation sequence.
10. The server system of claim 9 , wherein the target model is a human actor and the measured points are measured from a video recording of marked points on the human actor's face, the marked points being associated with the facial markers.
11. The server system of claim 10 , wherein the measured points vary with time and represent the facial animation sequence.
12. The server system of claim 9 , wherein the affine transformation function is computed with Horn's solution to minimize the L 2 -norm error.
13. The server system of claim 9 , wherein the optimal affine transformation function is computed with a RANSAC technique utilizing the unprocessed point and the two nearby neighboring points.
14. The server system of claim 13 , wherein the unprocessed point is randomly selected from those points of the set of measured points that have not been used to compute any affine transformation function to minimize an error.
15. The server system of claim 9 , the method further including:
receiving a user-specified threshold, wherein the optimal affine transformation function produces at least one of: a minimum overall alignment error or an overall alignment error below the user-specified threshold.
16. The server system of claim 9 , the method further including:
receiving a set of user-specified reliable measured points, wherein the unprocessed point is selected from those points of the set of user-specified reliable measured points that have not been used to compute any affine transformation function to minimize an error.
17. A non-transitory computer-readable medium having executable programming instructions stored thereon to execute a method for reducing error between motion capture sessions, the method comprising:
receiving a first set and a second set of measured points, the first set of measured points from a first motion capture session and associated with a first set of facial skin markers of a target model in a first facial animation sequence of the target model, the second set of measured points from a second motion capture session and associated with a second set of facial skin markers of the target model in a second facial animation sequence of the target model, wherein each measured point is associated with a three-dimensional coordinate triple;
computing an optimal affine transformation function for reducing error between the motion capture sessions, wherein the computing includes,
selecting an unprocessed point from the first set of measured points, the unprocessed point being a point of the first set of measured points that has not been used to compute any affine transformation function to minimize an error,
selecting two nearby neighboring points of the unprocessed point,
computing, as part of aligning the first set of measured points with the second set of measured points to reduce error between the motion capture sessions, an affine transformation function that minimizes an L 2 -norm error, and
identifying the optimal affine transformation function from a set of computed affine transformation functions; and
generating, utilizing the optimal affine transformation function, a head motion animation sequence.
18. The medium of claim 17 , wherein the target model is a human actor and the measured points are measured from a video recording of marked points on the human actor's face, the marked points being associated with the facial markers.
19. The medium of claim 18 , wherein the measured points vary with time and represent the facial animation sequence.
20. The medium of claim 17 , wherein the affine transformation function is computed with Horn's solution to minimize the L 2 -norm error.
21. The medium of claim 17 , wherein the optimal affine transformation function is computed with a RANSAC technique utilizing the unprocessed point and the two nearby neighboring points.
22. The medium of claim 21 , wherein the unprocessed point is randomly selected from those points of the set of measured points that have not been used to compute any affine transformation function to minimize an error.
23. The medium of claim 17 , the method further comprising:
receiving a user-specified threshold, wherein the optimal affine transformation function produces at least one of: a minimum overall alignment error or an overall alignment error below the user-specified threshold.
24. The medium of claim 17 , the method further comprising:
receiving a set of user-specified reliable measured points, wherein the unprocessed point is selected from those points of the set of user-specified reliable measured points that have not been used to compute any affine transformation function to minimize an error.