Systems and methods for tracking a controller

1 . A method comprising:

obtaining first geometric information from a controller, the first geometric information including a first point cloud of the controller associated with a scene;

obtaining second geometric information based on image data from a camera of a wearable device, the second geometric information including a second point cloud of the wearable device associated with the scene;

determining a correspondence between the first point cloud and the second point cloud based on three-dimensional (3D) points in the first point cloud that correspond to 3D points in the second point cloud;

outputting, based on the correspondence, a third point cloud including points from the first point cloud and points from the second point cloud; and

determining a six degree of freedom (6DoF) pose of the controller or the wearable device based on the third point cloud.

2 . The method of claim 1 , wherein the third point cloud includes at least one point from the first point cloud that is not included in the second point cloud and at least one point from the second point cloud that is not included in the first point cloud.

3 . The method of claim 1 , further comprising storing the third point cloud.

4 . The method of claim 1 , wherein determining the 6DoF pose comprises determining a 6DoF pose of the controller based on the third point cloud.

5 . The method of claim 4 , further comprising:

receiving, from the controller, at least one of image data or inertial measurement unit (IMU) data; and

determining the 6DoF pose of the controller based on at least of the image data or the IMU data.

6 . The method of claim 5 , wherein determining the 6DoF pose of the controller comprises:

determining a three-dimensional orientation of the controller within the first point cloud of the controller based on at least one of timestamped IMU data or timestamped image data received from the controller.

7 . The method of claim 4 , further comprising:

determine, based on the correspondence, a transformation from a coordinate system of the controller to a coordinate system of the wearable device;

converting, using the transformation, the 6DoF pose of the controller to the coordinate system of the wearable device; and

rendering content in an application based on the converted 6DoF pose.

8 . The method of claim 7 , further comprising:

receiving, from the controller, a second 6DoF pose of the controller determined by a 6DoF tracking algorithm implemented on the controller; and

rendering content in the application based on the second 6DoF pose.

9 . The method of claim 7 , further comprising:

receiving, from the controller, updated geometric information and updated IMU data from which a 6DoF tracking algorithm implemented on the wearable device determines a second 6DoF pose of the controller; and

rendering content in the application based on the second 6DoF pose.

10 . The method of claim 7 , further comprising:

receiving a point cloud update from the controller; and

synchronizing the coordinate system of the wearable device with the coordinate system of the controller based on the point cloud update.

11 . The method of claim 1 , wherein determining the 6DoF pose comprises determining a 6DoF pose of the wearable device based on the third point cloud.

12 . The method of claim 11 , further comprising:

rendering content based on the 6DoF pose of the wearable device.

13 . The method of claim 1 , further comprising:

receiving a point cloud update from the controller; and

determining an updated 6DoF pose of the wearable device based on the point cloud update.

14 . The method of claim 1 , further comprising implementing a 6DoF tracking algorithm on the wearable device to determine a 6DoF pose of the wearable device based on the third point cloud.

15 . The method of claim 1 , further comprising determining the second point cloud of the wearable device based on image data received from the wearable device.

16 . An apparatus, comprising:

a memory; and

a processor in communication with the memory, the processor configured to:

obtain first geometric information from a controller, the first geometric information including a first point cloud of the controller associated with a scene;

obtain second geometric information based on image data from a camera of a wearable device, the second geometric information including a second point cloud of the wearable device associated with the scene;

determine a correspondence between the first point cloud and the second point cloud based on three-dimensional (3D) points in the first point cloud that correspond to 3D points in the second point cloud;

output, based on the correspondence, a third point cloud including points from the first point cloud and points from the second point cloud; and

determine a six degree of freedom (6DoF) pose of the controller or the wearable device based on the third point cloud.

17 . The apparatus of claim 16 , wherein the third point cloud includes at least one point from the first point cloud that is not included in the second point cloud and at least one point from the second point cloud that is not included in the first point cloud.

18 . The apparatus of claim 16 , further comprising storing the third point cloud.

19 . The apparatus of claim 16 , wherein determining the 6DoF pose comprises determining a 6DoF pose of the controller based on the third point cloud.

20 . The apparatus of claim 19 , further comprising:

receiving, from the controller, at least one of image data or inertial measurement unit (IMU) data; and

determining the 6DoF pose of the controller based on at least of the image data or the IMU data.

21 . The apparatus of claim 20 , wherein determining the 6DoF pose of the controller comprises:

determining a three-dimensional orientation of the controller within the first point cloud of the controller based on at least one of timestamped IMU data or timestamped image data received from the controller.

22 . The apparatus of claim 19 , further comprising:

determine, based on the correspondence, a transformation from a coordinate system of the controller to a coordinate system of the wearable device;

converting, using the transformation, the 6DoF pose of the controller to the coordinate system of the wearable device; and

rendering content in an application based on the converted 6DoF pose.

23 . The apparatus of claim 22 , further comprising:

receiving, from the controller, a second 6DoF pose of the controller determined by a 6DoF tracking algorithm implemented on the controller; and

rendering content in the application based on the second 6DoF pose.

24 . The apparatus of claim 22 , further comprising:

receiving, from the controller, updated geometric information and updated IMU data from which a 6DoF tracking algorithm implemented on the wearable device determines a second 6DoF pose of the controller; and

rendering content in the application based on the second 6DoF pose.

25 . The apparatus of claim 22 , further comprising:

receiving a point cloud update from the controller; and

synchronizing the coordinate system of the wearable device with the coordinate system of the controller based on the point cloud update.

26 . The apparatus of claim 16 , wherein determining the 6DoF pose comprises determining a 6DoF pose of the wearable device based on the third point cloud.

27 . The apparatus of claim 26 , further comprising:

rendering content based on the 6DoF pose of the wearable device.

28 . The apparatus of claim 16 , further comprising:

receiving a point cloud update from the controller; and

determining an updated 6DoF pose of the wearable device based on the point cloud update.

29 . The apparatus of claim 16 , further comprising implementing a 6DoF tracking algorithm on the wearable device to determine a 6DoF pose of the wearable device based on the third point cloud.

30 . The apparatus of claim 16 , further comprising determining the second point cloud of the wearable device based on image data received from the wearable device.