Future pose predictor for a controller
Systems and techniques are described herein for predicting poses. An apparatus for predicting a pose includes at least one memory and at least one processor coupled to at least one memory and configured to: receive, at a pose estimation engine, pose data from a controller, the pose data including a plurality of previous poses of the controller; and predict, at a first time via the pose estimation engine, a future pose of the controller based on the pose data, the future pose comprising an expected pose of the controller at a second time that is after the first time.
1. An apparatus for predicting a pose, the apparatus comprising:
at least one memory; and
at least one processor coupled to at least one memory and configured to:
receive, via a pose estimation engine, pose data from a controller, the pose data including a plurality of previous poses of the controller; and
predict, at a first time via the pose estimation engine, a future pose of the controller based on the pose data, the future pose comprising an expected pose of the controller at a second time that is after the first time, wherein the pose estimation engine is trained using a six degrees of freedom pose loss comprising a total loss plus a position loss.
2. The apparatus of claim 1 , wherein the pose data comprises only current pose data and the plurality of previous poses from the controller.
3. The apparatus of claim 1 , wherein the pose estimation engine is configured on at least one of the controller, a headset in communication with the controller, or a computer in communication with at least one of the controller or the headset.
4. The apparatus of claim 1 , wherein the at least one processor is configured to:
determine a latency associated with at least one of transferring the pose data from the controller to one or more devices or processing the pose data via the pose estimation engine based on the latency.
5. The apparatus of claim 1 , wherein the pose data is configured in six degrees of freedom.
6. The apparatus of claim 1 , wherein the pose estimation engine is trained using at least one of position data, orientation data, velocity data, or acceleration data.
7. The apparatus of claim 1 , wherein the pose estimation engine comprises an input layer providing input layer data to an autoencoder having an encoder and a decoder trained on observed training data, and an output layer receiving autoencoder data from the autoencoder.
8. The apparatus of claim 7 , wherein the encoder learns time dependent features from the pose data and the decoder combines features output by the encoder into a future timestamp.
9. The apparatus of claim 8 , wherein the input layer comprises a fully connected layer, the output layer comprises a fully connected layer, and wherein the autoencoder comprises a bi-long short-term memory network.
10. The apparatus of claim 9 , wherein states from the encoder are shared to the decoder as an initial state as part of training the pose estimation engine.
11. The apparatus of claim 9 , wherein the output layer combines an output from the decoder to generate a six degrees of freedom pose for the future timestamp.
12. The apparatus of claim 7 , wherein the encoder comprises a first multi-layer bi-long short-term memory network and wherein the decoder comprises a second multi-layer bi-long short-term memory network.
13. The apparatus of claim 7 , wherein the autoencoder provides a repeated vector from the encoder to the decoder.
14. The apparatus of claim 1 , wherein the at least one processor is configured to:
render an image on a display based on the future pose.
15. The apparatus of claim 1 , wherein the pose data excludes velocity data associated with the controller.
16. The apparatus of claim 1 , wherein the pose estimation engine is trained further using at least one of an acceleration loss, a velocity loss, a quaternion loss, a rotation loss, or a rotation-translation loss, wherein the acceleration loss comprises a mean absolute error of acceleration, the velocity loss comprises a sum of the mean absolute error of acceleration and a mean absolute error of a gradient of velocity, and the position loss comprises a sum of a mean absolute error of position.
17. The apparatus of claim 1 , wherein the position loss comprises a sum of a mean absolute error of position.
18. A method of predicting a pose, the method comprising:
receiving, at a pose estimation engine, pose data from a controller, the pose data including a plurality of previous poses of the controller; and
predicting, at a first time via the pose estimation engine, a future pose of the controller based on the pose data, the future pose comprising an expected pose of the controller at a second time that is after the first time, wherein the pose estimation engine is trained using a six degrees of freedom pose loss comprising a total loss plus a position loss.
19. The method of claim 18 , wherein the pose data comprises only current pose data and the plurality of previous poses from the controller.
20. The method of claim 18 , wherein the pose estimation engine is configured on at least one of the controller, a headset in communication with the controller, or a computer in communication with at least one of the controller or the headset.
21. The method of claim 18 , further comprising:
determining a latency associated with at least one of transferring the pose data from the controller to one or more devices or processing the pose data via the pose estimation engine based on the latency.
22. The method of claim 18 , wherein the pose estimation engine comprises an input layer providing input layer data to an autoencoder having an encoder and a decoder trained on observed training data, and an output layer receiving autoencoder data from the autoencoder.
23. The method of claim 22 , wherein:
the encoder learns time dependent features from the pose data;
the decoder combines features output by the encoder into a future timestamp;
the input layer comprises a fully connected layer;
the output layer comprises a fully connected layer; and
the autoencoder comprises a bi-long short-term memory network.
24. The method of claim 23 , wherein states from the encoder are shared to the decoder as an initial state as part of training the pose estimation engine.
25. The method of claim 23 , wherein the output layer combines an output from the decoder to generate a six degrees of freedom pose for the future timestamp.
26. The method of claim 18 , further comprising:
rendering an image on a display based on the future pose.
27. The method of claim 18 , wherein the pose data excludes velocity data associated with the controller.
28. The method of claim 18 , wherein the pose estimation engine is trained further using at least one of an acceleration loss, a velocity loss, a quaternion loss, a rotation loss, or a rotation-translation loss, wherein the acceleration loss comprises a mean absolute error of acceleration, the velocity loss comprises a sum of the mean absolute error of acceleration and a mean absolute error of a gradient of velocity, and the position loss comprises a sum of a mean absolute error of position.
29. The method of claim 18 , wherein the position loss comprises a sum of a mean absolute error of position.