Pose tracking system with multi device shared scene map
A virtual environment or mixed reality environment system including a headset device and controller. Both the headset device and the controller are configured to access and update a shared scene map. In some cases, the headset device may implement pose tracking with respect to the headset device and the controller and the controller may also implement pose tracking with respect to the headset device and the controller.
1. A method comprising:
determining, by a controller associate with a mixed reality image system, a six-degree of freedom (6DOF) pose of a headset device relative to the controller based at least in part on image data of a physical environment captured by the controller, the headset device associate with the mixed reality image system;
determining, by the controller, a 6DOF pose of the controller relative to a scene map shared between the controller and the headset device; and
determining a 6DOF pose of the headset device relative to the scene map based at least in part on the 6DOF pose of the headset device relative to the controller and the 6DOF pose of the controller relative to the scene map.
2. The method as recited in claim 1 , further comprising determining an input to the mixed reality image system based at least in part on the 6DOF pose of the headset device relative to the scene map.
3. The method as recited in claim 2 , wherein determining the input to the mixed reality image system is also based at least in part on the 6DOF pose of the controller relative to the scene map.
4. The method as recited in claim 1 , wherein the 6DOF pose of the headset device relative to the scene map is determined by the controller.
5. The method as recited in claim 1 , further comprising:
sending the 6DOF pose of the headset device relative to the controller and the 6DOF pose of the controller relative to the scene map to the headset device; and
wherein the 6DOF pose of the headset device relative to the scene map is determined by the headset device.
6. The method as recited in claim 1 , further comprising
updating, by the controller, the scene map based on image data captured by the controller; and
updating, by the headset device, the scene map based on image data captured by the headset device.
7. The method as recited in claim 1 , wherein the controller and the headset device are performing self-tracking and localization within the scene.
8. The method as recited in claim 1 , further comprising:
updating the scene map, by the controller, based on image data captured by the controller; and
updating the scene map, by the headset device, based on image data captured by the headset device.
9. The method as recited in claim 1 , wherein determining the 6DOF pose of the headset device relative to the scene map based at least in part on the 6DOF pose of the headset device relative to the controller and the 6DOF pose of the controller relative to the scene map includes composing the 6DOF pose of the headset device relative to the controller and the 6DOF pose of the controller relative.
10. A method comprising:
determining, by a headset device associated with a mixed reality image system, a six-degree of freedom (6DOF) pose of a controller associated with the mixed reality image system based at least in part on image data of a physical environment, the 6DOF pose of the controller relative to the headset device;
determining, by the controller, a 6DOF pose of the controller relative to a scene map shared between the controller and the headset device;
sending the 6DOF pose of the controller relative to the scene map from the controller to the headset device;
determining a 6DOF pose of the headset device relative to the scene map based at least in part on the 6DOF pose of the controller relative to the headset device and the 6DOF pose of the controller relative to the scene map; and
determining an input to the mixed reality image system based at least in part on the 6DOF pose of the headset device relative to the scene map.
11. The method as recited in claim 10 , further comprising outputting the 6DOF pose of the headset device relative to the scene map.
12. The method as recited in claim 10 , further comprising
updating, by the controller, the scene map based on image data captured by the controller; and
updating, by the headset device, the scene map based on image data captured by the headset device.
13. The method as recited in claim 10 , further comprising setting a perspective for image data associated with the mixed reality image system displayed to a user based at least in part on the 6DOF pose of the headset device relative to the scene map.
14. The method as recited in claim 10 , further comprising re-localizing, by the headset device, within the scene map based at least in part on the 6DOF pose of the headset device relative to the scene map.
15. A method comprising:
determining, by a headset device associate with a mixed reality image system, a six-degree of freedom (6DOF) pose of a controller associate with the mixed reality image system based at least in part on image data of a physical environment, the 6DOF pose of the controller relative to the headset device;
determining, by the headset device, a 6DOF pose of the headset device relative to a scene map shared between the controller and the headset device;
determining a 6DOF pose of the controller relative to the scene map based at least in part on the 6DOF pose of the controller relative to the headset device and the 6DOF pose of the headset device relative to the scene map; and
outputting the 6DOF pose of the controller relative to the scene map.
16. The method as recited in claim 14 , wherein outputting the 6DOF pose of the controller relative to the scene map includes determining a user input to the mixed reality image system based at least in part on the 6DOF pose of the controller relative to the scene map.
17. The method as recited in claim 15 , further comprising:
wherein outputting the 6DOF pose of the controller relative to the scene map includes sending the 6DOF pose of the controller relative to the scene from the headset device to the controller; and
re-localizing, by the controller, within the scene map based at least in part on the 6DOF pose of the controller relative to the scene map.
18. The method as recited in claim 15 , further comprising capturing, by the headset device, the image data of the physical environment, the physical environment in proximity to the controller and the headset device while the 6DOF pose of headset device relative to the scene map is determined.
19. The method as recited in claim 15 , further comprising capturing, by the headset device, orientation data associated with the headset device.
20. The method as recited in claim 15 , further comprising substantially simultaneously updating the scene map by the controller based on image data captured by the controller and by the headset device based on image data captured by the headset device.