Tracking pose of handheld object

1. At a computing device, a method for tracking a pose of a handheld object used with a head-mounted display device, the method comprising:

receiving image data from an image sensing system;

detecting a plurality of feature points of the handheld object in a frame of the image data, wherein the feature points comprise light sources and/or other fiducial markers;

receiving inertial measurement unit (IMU) data from an IMU of the handheld object;

based on detecting the plurality of feature points and receiving the IMU data, determining a first pose of the handheld object;

determining that at least a portion of the plurality of feature points is not detected in another frame of the image data;

based on determining that at least the portion of the plurality of feature points is not detected, using solely the IMU data, updating the first pose of the handheld object to a second pose; and

body-locking the second pose of the handheld object to a body location on a user wearing the head-mounted display device.

2. The method of claim 1 , wherein body-locking the second pose of the handheld object further comprises:

determining that a first threshold time has been reached;

determining that the handheld object is not stationary; and

based on determining that the first threshold time has been reached and that the handheld object is not stationary, body-locking the second pose of the handheld object to the body location on the user wearing the head-mounted display device.

3. The method of claim 2 , further comprising:

determining that a second threshold time has been reached after expiration of the first threshold time; and

based on determining that the first threshold time has been reached, the handheld object is not stationary, and the second threshold time has been reached, body-locking the second pose of the handheld object to the body location on the user wearing the head-mounted display device.

4. The method of claim 2 , further comprising:

determining that the handheld object is stationary after determining that the handheld object is not stationary; and

based on determining that the handheld object is stationary after determining that the handheld object is not stationary, using the IMU data to update an orientation of the second pose of the handheld object, and marking a position of the second pose as invalid.

5. The method of claim 1 , further comprising:

prior to body-locking the second pose of the handheld object, determining that a velocity of the handheld object is less than a threshold velocity; and

based on determining that the velocity of the handheld object is less than the threshold velocity, body-locking the second pose of the handheld object to the body location on the user wearing the head-mounted display device.

6. The method of claim 1 , further comprising:

determining that the handheld object is stationary; and

based on determining that the handheld object is stationary, world-locking the second pose of the handheld object.

7. The method of claim 1 , wherein the body location on the user wearing the head-mounted display device comprises a neck or an elbow of the user.

8. The method of claim 1 , further comprising detecting additional feature points in one or more additional frames of the image data; and

based on detecting the additional feature points, ceasing to body-lock the second pose of the handheld object and updating the second pose to a third pose.

9. The method of claim 1 , wherein the pose of the handheld object comprises a position and an orientation, and body-locking the second pose of the handheld object comprises locking the position and not locking the orientation.

10. The method of claim 1 , wherein the image sensing system is mounted on the head-mounted display device.

11. A computing device for tracking a pose of a handheld object used with a head-mounted display device, the computing device comprising:

a logic device configured to execute instructions; and

a storage device comprising instructions executable by the logic device to:

receive image data from an image sensing system;

detect a plurality of feature points of the handheld object in a frame of the image data, wherein the feature points comprise light sources and/or other fiducial markers;

receive inertial measurement unit (IMU) data from an IMU of the handheld object;

based on detecting the plurality of feature points and receiving the IMU data, determine a first pose of the handheld object;

determine that at least a portion of the plurality of feature points is not detected in another frame of the image data;

based on determining that at least the portion of the plurality of feature points is not detected, using solely the IMU data, update the first pose of the handheld object to a second pose; and

body-lock the second pose of the handheld object to a body location on a user wearing the head-mounted display device.

12. The computing device of claim 11 , wherein the instructions are executable to:

determine that a first threshold time has been reached;

determine that the handheld object is not stationary; and

based on determining that the first threshold time has been reached and that the handheld object is not stationary, body-lock the second pose of the handheld object to the body location on the user wearing the head-mounted display device.

13. The computing device of claim 12 , wherein the instructions are executable to:

determine that a second threshold time has been reached after expiration of the first threshold time; and

based on determining that the first threshold time has been reached, the handheld object is not stationary, and the second threshold time has been reached, body-lock the second pose of the handheld object to the body location on the user wearing the head-mounted display device.

14. The computing device of claim 12 , wherein the instructions are executable to:

determine that the handheld object is stationary after determining that the handheld object is not stationary; and

based on determining that the handheld object is stationary after determining that the handheld object is not stationary, use the IMU data to update an orientation of the second pose of the handheld object, and marking a position of the second pose as invalid.

15. The computing device of claim 11 , wherein the instructions are executable to:

prior to body-locking the second pose of the handheld object, determine that a velocity of the handheld object is less than a threshold velocity; and

based on determining that the velocity of the handheld object is less than the threshold velocity, body-lock the second pose of the handheld object to the body location on the user wearing the head-mounted display device.

16. The computing device of claim 11 , wherein the instructions are executable to:

determine that the handheld object is stationary; and

based on determining that the handheld object is stationary, world-lock the second pose of the handheld object.

17. The computing device of claim 11 , wherein the body location on the user wearing the head-mounted display device comprises a neck or an elbow of the user.

18. The computing device of claim 11 , wherein the instructions are executable to:

detect additional feature points in one or more additional frames of the image data; and

based on detecting the additional feature points in the image data, cease body-locking the second pose of the handheld object and update the second pose to a third pose.

19. The computing device of claim 11 , wherein the pose of the handheld object comprises a position and an orientation, and body-locking the second pose of the handheld object comprises locking the position and not locking the orientation.

20. A head-mounted display device, comprising:

an image sensing system; and

a computing device comprising:

a logic device configured to execute instructions; and

a storage device comprising instructions executable by the logic device to:

receive image data from the image sensing system;

detect a plurality of feature points of the handheld object in a frame of the image data;

receive inertial measurement unit (IMU) data from an IMU of the handheld object;

based on detecting the plurality of feature points and receiving the IMU data, determine a first pose of the handheld object;

determine that at least a portion of the plurality of feature points is not detected in another frame of the image data;

using the IMU data, update the first pose of the handheld object to a second pose;

determine that the handheld object is not stationary;

determine that a first threshold time has been reached; and

based on determining that the first threshold time has been reached and that the handheld object is not stationary, body-lock the second pose of the handheld object to a body location on a user wearing the head-mounted display device.