Future pose predictor for a controller
Systems and techniques are described herein for predicting poses. An apparatus for predicting a pose includes at least one memory and at least one processor coupled to at least one memory and configured to: receive, at a pose estimation engine, pose data from a controller, the pose data including a plurality of previous poses of the controller; and predict, at a first time via the pose estimation engine, a future pose of the controller based on the pose data, the future pose comprising an expected pose of the controller at a second time that is after the first time.
1 . An apparatus for predicting a pose, the apparatus comprising:
at least one memory; and
at least one processor coupled to at least one memory and configured to:
receive, via a pose estimation engine, pose data from a controller, the pose data including a plurality of previous poses of the controller, and wherein the pose data excludes velocity data associated with the controller;
determine a latency associated with at least one of transferring the pose data from the controller to one or more devices or processing the pose data via the pose estimation engine; and
predict, at a first time via the pose estimation engine based on the latency, a future pose of the controller based on the pose data, the future pose comprising an expected pose of the controller at a second time that is after the first time.
2 . The apparatus of claim 1 , wherein the pose data comprises only current pose data and the plurality of previous poses from the controller.
3 . The apparatus of claim 1 , wherein the pose estimation engine is configured on at least one of the controller, a headset in communication with the controller, or a computer in communication with at least one of the controller or the headset.
4 . The apparatus of claim 1 , wherein the pose data is configured in six degrees of freedom.
5 . The apparatus of claim 1 , wherein the pose estimation engine is trained using at least one of position data, orientation data, velocity data, or acceleration data.
6 . The apparatus of claim 1 , wherein the pose estimation engine comprises an input layer providing input layer data to an autoencoder having an encoder and a decoder trained on observed training data, and an output layer receiving autoencoder data from the autoencoder.
7 . The apparatus of claim 6 , wherein the encoder learns time dependent features from the pose data and the decoder combines features output by the encoder into a future timestamp.
8 . The apparatus of claim 7 , wherein the input layer comprises a fully connected layer, the output layer comprises a fully connected layer, and wherein the autoencoder comprises a bi-long short-term memory network.
9 . The apparatus of claim 8 , wherein states from the encoder are shared to the decoder as an initial state as part of training the pose estimation engine.
10 . The apparatus of claim 8 , wherein the output layer combines an output from the decoder to generate a six degrees of freedom pose for the future timestamp.
11 . The apparatus of claim 6 , wherein the encoder comprises a first multi-layer bi-long short-term memory network and wherein the decoder comprises a second multi-layer bi-long short-term memory network.
12 . The apparatus of claim 6 , wherein the autoencoder provides a repeated vector from the encoder to the decoder.
13 . The apparatus of claim 1 , wherein the pose estimation engine is trained using at least one of an acceleration loss, a velocity loss, a position loss, a quaternion loss, a rotation loss, a rotation-translation loss, or a six degrees of freedom pose loss.
14 . The apparatus of claim 13 , wherein the acceleration loss comprises a mean absolute error of acceleration, the velocity loss comprises a sum of the mean absolute error of acceleration and a mean absolute error of a gradient of velocity, and the position loss comprises a sum of a mean absolute error of position, the mean absolute error of a gradient of position, and a mean absolute error of a gradient of the gradient of position.
15 . The apparatus of claim 14 , wherein the six degrees of freedom pose loss comprises a total loss plus a position loss.
16 . The apparatus of claim 1 , wherein the at least one processor is configured to:
render an image on a display based on the future pose.
17 . A method of predicting a pose, the method comprising:
receiving, at a pose estimation engine, pose data from a controller, the pose data including a plurality of previous poses of the controller, wherein the pose data excludes velocity data associated with the controller;
determining a latency associated with at least one of transferring the pose data from the controller to one or more devices or processing the pose data via the pose estimation engine; and
predicting, at a first time via the pose estimation engine based on the latency, a future pose of the controller based on the pose data, the future pose comprising an expected pose of the controller at a second time that is after the first time.
18 . The method of claim 17 , wherein the pose data comprises only current pose data and the plurality of previous poses from the controller.
19 . The method of claim 17 , wherein the pose estimation engine is configured on at least one of the controller, a headset in communication with the controller, or a computer in communication with at least one of the controller or the headset.
20 . The method of claim 17 , wherein the pose estimation engine comprises an input layer providing input layer data to an autoencoder having an encoder and a decoder trained on observed training data, and an output layer receiving autoencoder data from the autoencoder.
21 . The method of claim 20 , wherein:
the encoder learns time dependent features from the pose data;
the decoder combines features output by the encoder into a future timestamp;
the input layer comprises a fully connected layer;
the output layer comprises a fully connected layer; and
the autoencoder comprises a bi-long short-term memory network.
22 . The method of claim 21 , wherein states from the encoder are shared to the decoder as an initial state as part of training the pose estimation engine.
23 . The method of claim 21 , wherein the output layer combines an output from the decoder to generate a six degrees of freedom pose for the future timestamp.
24 . The method of claim 17 , wherein the pose estimation engine is trained using at least one of an acceleration loss, a velocity loss, a position loss, a quaternion loss, a rotation loss, a rotation-translation loss, or a six degrees of freedom pose loss.
25 . The method of claim 24 , wherein the acceleration loss comprises a mean absolute error of acceleration, the velocity loss comprises a sum of the mean absolute error of acceleration and a mean absolute error of a gradient of velocity, the position loss comprises a sum of a mean absolute error of position, the mean absolute error of a gradient of position, and a mean absolute error of a gradient of the gradient of position, and the six degrees of freedom pose loss comprises a total loss plus a position loss.
26 . The method of claim 17 , further comprising:
rendering an image on a display based on the future pose.