Motion capture data prediction system for video game development
Implementations disclosed herein are directed to a method for predicting motion capture data implemented by a video game development system including one or more processors. The method can include: obtaining, by one or more of the processors of the video game development system, motion capture data; the motion capture data can include one or more missing values; generating, by one or more of the processors of the video game development system, a predicted value for the one or more missing values. Generating a predicted value can include processing the motion capture data using one or more machine learning models to determine the predicted value.
1 . A method for predicting motion capture data implemented by a video game development system comprising one or more processors, the method comprising:
obtaining, by one or more of the processors of the video game development system, motion capture data; wherein the motion capture data comprises one or more missing values; and
generating, by one or more of the processors of the video game development system, a predicted value for the one or more missing values comprising:
sequentially updating the motion capture data using a cascade of machine learning models to determine the predicted value, wherein each machine learning model is configured to predict values corresponding to a subset of markers in the motion capture data.
2 . The method of claim 1 , wherein generating a predicted value comprises updating the motion capture data with an initial estimated value for the one or more missing values prior to sequentially updating the motion capture data using the cascade of machine learning models; wherein the initial estimated values are determined using interpolation.
3 . The method of claim 1 , wherein generating a predicted value further comprises hip-centering the motion capture data prior to sequentially updating the motion capture data using the cascade of machine learning models.
4 . The method of claim 3 , wherein the motion capture data comprises data associated with one or more hip markers; and
wherein hip-centering the motion capture data comprises:
determining a position of the hips based upon the data associated with the one or more hip markers; and
centering the motion capture data based upon the determined position of the hips.
5 . The method of claim 3 , wherein generating a predicted value further comprises de-centering the predicted value determined by the cascade of machine learning models.
6 . The method of claim 5 , wherein de-centering is based upon the determined position of the hips.
7 . The method of claim 1 , further comprising performing one or more further iterations through the cascade of machine learning models to refine the predicted value.
8 . The method of claim 1 , wherein the cascade of machine learning models comprise one or more neural networks.
9 . The method of claim 8 , wherein the one or more neural networks comprise one or more one-dimensional convolutional layers.
10 . The method of claim 1 , wherein the cascade of machine learning models have been trained based upon a denoising objective.
11 . The method of claim 1 , wherein the cascade of machine learning models comprises one or more machine learning models configured to predict values for markers associated with the hips; one or more machine learning models configured to predict values for markers associated with the torso; one or more machine learning models configured to predict values for markers associated with the head; and one or more machine learning models configured to predict values for markers associated with the limbs.
12 . A video game development system comprising:
one or more processors; and
one or more computer readable storage media comprising processor readable instructions to cause the one or more processors to carry out a method comprising:
obtaining motion capture data; wherein the motion capture data comprises one or more missing values; and
generating a predicted value for the one or more missing values comprising:
sequentially updating the motion capture data using a cascade of machine learning models to determine the predicted value, wherein each machine learning model is configured to predict values corresponding to a subset of markers in the motion capture data.
13 . The system of claim 12 , wherein generating a predicted value comprises updating the motion capture data with an initial estimated value for the one or more missing values prior to sequentially updating the motion capture data using the cascade of machine learning models; wherein the initial estimated values are determined using interpolation.
14 . The system of claim 12 , wherein generating a predicted value further comprises hip-centering the motion capture data prior to sequentially updating the motion capture data using the cascade of machine learning models.
15 . The system of claim 14 , wherein generating a predicted value further comprises de-centering the predicted value determined by the cascade of machine learning models.
16 . One or more non-transitory computer-readable storage media comprising instructions which, when executed by one or more processors of a video game development system, cause the one or more processors to carry out a method comprising:
obtaining motion capture data; wherein the motion capture data comprises one or more missing values; and
generating a predicted value for the one or more missing values comprising:
sequentially updating the motion capture data using a cascade of machine learning models to determine the predicted value, wherein each machine learning model is configured to predict values corresponding to a subset of markers in the motion capture data.
17 . The method of claim 11 , wherein the sequential order is hips, torso, head and limbs.