Controller movement tracking with light emitters
A head-worn computer includes a camera system positioned to capture a surrounding environment in front of a user, a processor that identifies a position of a plurality of light emitters mounted on a hand-held controller from images captured by the camera system and tracks the position of the plurality of light emitters as the hand-held controller moves in the surrounding environment and interprets the tracked position as positional changes of the hand-held controller. The processor uses the position of the plurality of light emitters as markers in three dimensional space, the markers used as an anchor for virtual content presented in a see-through display of the head-worn computer.
1. A system comprising:
a wearable head device;
a camera system configured to capture images of a surrounding environment of a user of the wearable head device;
a see-through display of the wearable head device; and
one or more processors configured to perform a method comprising:
determining a position of a hand-held controller at a first time,
determining a predicted position of the hand-held controller at a second time later than the first time, wherein the determining is based on an input from a first inertial measurement unit (IMU) of the hand-held controller and based further on an image captured by the camera system, the image comprising a view of a plurality of light emitters of the hand-held controller,
determining, based on the predicted position of the hand-held controller, a display position of virtual content in the see-through display of the wearable head device, and
presenting the virtual content at the display position in the see-through display of the wearable head device,
wherein the plurality of light emitters comprises a first light emitter of a first wavelength and a second light emitter of a second wavelength, different from the first wavelength, and
wherein a first tracking sensitivity associated with the first light emitter is greater than a second tracking sensitivity associated with the second light emitter.
2. The system of claim 1 , wherein the camera system comprises a fisheye lens and capturing the images of the surrounding environment comprises capturing the images of the surrounding environment via the fisheye lens.
3. The system of claim 1 , wherein the camera system comprises two or more cameras mounted separately on the wearable head device and wherein each camera views the plurality of light emitters from a different perspective.
4. The system of claim 1 , wherein the plurality of light emitters is mounted on the hand-held controller and wherein the predicted position of the hand-held controller is predicted based on a perspective view of the plurality of light emitters.
5. The system of claim 1 , further comprising a second IMU, wherein the predicted position of the hand-held controller is predicted further based on an output from the second IMU.
6. The system of claim 1 , wherein the method further comprises:
providing the predicted position as a control input for a software application operating on the wearable head device.
7. The system of claim 1 , wherein the method further comprises:
using the position of the plurality of light emitters as a marker in three dimensional space, wherein the marker is used as an anchor for virtual content presented in the see-through display of the wearable head device.
8. A method comprising:
determining a position of a hand-held controller at a first time;
determining a predicted position of the hand-held controller at a second time later than the first time, wherein the determining is based on an input from a first inertial measurement unit (IMU) of the hand-held controller and based further on an image captured by a camera system, the image comprising a view of a plurality of light emitters of the hand-held controller;
determining, based on the predicted position of the hand-held controller, a display position of virtual content in a see-through display of a wearable head device; and
presenting the virtual content at the display position in the see-through display of the wearable head device,
wherein the plurality of light emitters comprises a first light emitter of a first wavelength and a second light emitter of a second wavelength, different from the first wavelength, and
wherein a first tracking sensitivity associated with the first light emitter is greater than a second tracking sensitivity associated with the second light emitter.
9. The method of claim 8 , wherein the camera system comprises a fisheye lens and the image comprises an image captured via the fisheye lens.
10. The method of claim 8 , wherein the camera system comprises two or more cameras mounted separately on the wearable head device and wherein each camera views the plurality of light emitters from a different perspective.
11. The method of claim 8 , wherein the plurality of light emitters is mounted on the hand-held controller and wherein the predicted position of the hand-held controller is predicted based on a perspective view of the plurality of light emitters.
12. The method of claim 8 , wherein the wearable head device comprises a second IMU, and wherein the predicted position of the hand-held controller is predicted further based on an output from the second IMU.
13. The method of claim 8 , further comprising providing the predicted position as a control input for a software application operating on the wearable head device.
14. The method of claim 8 , further comprising using a position of the plurality of light emitters as a marker in three dimensional space, wherein the marker is used as an anchor for virtual content presented in the see-through display of the wearable head device.
15. A non-transitory computer readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising:
determining a position of a hand-held controller at a first time;
determining a predicted position of the hand-held controller at a second time later than the first time, wherein the determining is based on an input from a first inertial measurement unit (IMU) of the hand-held controller and based further on an image captured by a camera system, the image comprising a view of a plurality of light emitters of the hand-held controller;
determining, based on the predicted position of the hand-held controller, a display position of virtual content in a see-through display of a wearable head device; and
presenting the virtual content at the display position in the see-through display of the wearable head device,
wherein the plurality of light emitters comprises a first light emitter of a first wavelength and a second light emitter of a second wavelength, different from the first wavelength, and
wherein a first tracking sensitivity associated with the first light emitter is greater than a second tracking sensitivity associated with the second light emitter.
16. The non-transitory computer readable medium of claim 15 , wherein the camera system comprises a fisheye lens and the image comprises an image captured via the fisheye lens.
17. The non-transitory computer readable medium of claim 15 , wherein the camera system comprises two or more cameras mounted separately on the wearable head device and wherein each camera views the plurality of light emitters from a different perspective.
18. The non-transitory computer readable medium of claim 15 , wherein the plurality of light emitters is mounted on the hand-held controller and wherein the predicted position of the hand-held controller is predicted based on a perspective view of the plurality of light emitters.
19. The non-transitory computer readable medium of claim 15 , wherein the wearable head device comprises a second IMU, and wherein the predicted position of the hand-held controller is predicted further based on an output from the second IMU.
20. The non-transitory computer readable medium of claim 15 , wherein the method further comprises providing the predicted position as a control input for a software application operating on the wearable head device.