Drift cancelation for portable object detection and tracking
The technology disclosed can provide capabilities such as using motion sensors and/or other types of sensors coupled to a motion-capture system to monitor motions within a real environment. A virtual object can be projected to a user of a portable device integrated into an augmented rendering of a real environment about the user. Motion information of a user body portion is determined based at least in part upon sensory information received from imaging or acoustic sensory devices. Control information is communicated to a system based in part on a combination of the motion of the portable device and the detected motion of the user. The virtual device experience can be augmented in some implementations by the addition of haptic, audio and/or other sensory information projectors.
1. A method of integrating real three-dimensional (3D) space sensing with a virtual reality head mounted device, the method including:
at a first time t 0 , using a sensor attached to the virtual reality head mounted device, sensing a first position of at least one hand in a first reference frame of a three-dimensional (3D) sensory space, including tracking at least portions of the hand for causing interaction between the hand and the virtual reality head mounted device;
causing display of a first virtual representation of the hand at the first position, wherein the first virtual representation is rendered in a virtual environment of the virtual reality head mounted device;
at a second time t 1 , sensing, in the 3D sensory space, a second position of the hand different from the first position responsive to repositioning of the virtual reality head mounted device and the attached sensor due to body movement, wherein the hand has not moved in the 3D sensory space between t 0 and t 1 ; and
causing display of a second virtual representation of the hand at an actual second position by:
sensing motion of the attached sensor and calculating a second reference frame that accounts for repositioning of the attached sensor;
calculating a transformation that renders the first position in the first reference frame and the second position in the second reference frame into a common reference frame; and
transforming the first and second positions of the hand into the common reference frame, wherein the common reference frame has a fixed point of reference and an initial orientation of axes, whereby the sensed second position is transformed to the actual second position.
2. A system of integrating real three-dimensional (3D) space sensing with a virtual reality head mounted device, the system including:
a processor and a computer readable storage medium storing computer instructions configured to cause the processor to implement the method of claim 1 .
3. One or more non-transitory computer readable media having instructions stored thereon for performing a method of claim 1 .
4. A method of integrating real three-dimensional (3D) space sensing with an augmented reality head mounted device, the method including:
at a first time t 0 , using a sensor attached to the augmented reality head mounted device, sensing a first position of at least one hand in a first reference frame of a three-dimensional (3D) sensory space located in a real environment, including tracking at least portions of the hand for causing interaction between the hand and the augmented reality head mounted device;
generating data representing a first virtual representation of the hand at the first position, wherein the first virtual representation is rendered in a virtual environment of the augmented reality head mounted device superimposed on the real environment;
at a second time t 1 , sensing in the 3D sensory space, a second position of the hand different from the first position responsive to repositioning of the augmented reality head mounted device and the attached sensor due to body movement, wherein the hand has not moved in the 3D sensory space between t 0 and t 1 ; and
generating data representing a second virtual representation of the hand at an actual second position by:
sensing motion of the attached sensor and calculating a second reference frame that accounts for repositioning of the attached sensor;
calculating a transformation that renders the first position in the first reference frame and the second position in the second reference frame into a common reference frame; and
transforming the first and second positions of the hand into the common reference frame, wherein the common reference frame has a fixed point of reference and an initial orientation of axes, whereby the sensed second position is transformed to the actual second position.
5. The method of claim 4 , wherein the common reference frame is a world reference frame that does not change as the attached sensor is repositioned.
6. The method of claim 4 , wherein the common reference frame is the second reference frame.
7. The method of claim 4 , wherein the transforming the first and second positions of the hand into the common reference frame further includes applying an affine transformation.
8. The method of claim 4 , further including determining the orientation of the hand at the first position with respect to the first reference frame and causing interaction between the hand and the augmented reality accordingly.
9. The method of claim 4 , further including determining the orientation of the hand at the second position with respect to the second reference frame and causing interaction between the hand and the augmented reality accordingly.
10. The method of claim 4 , wherein determining the position of the hand at the first position further includes calculating a translation of the hand with respect to the common reference frame and causing interaction between the hand and the augmented reality accordingly.
11. The method of claim 4 , wherein determining the position of the hand at the second position further includes calculating a translation of the hand with respect to the common reference frame and causing interaction between the hand and the augmented reality accordingly.
12. A system of integrating real three-dimensional (3D) space sensing with an augmented reality head mounted device, the system including:
a processor and a computer readable storage medium storing computer instructions configured to cause the processor to implement the method of claim 4 .
13. One or more non-transitory computer readable media having instructions stored thereon for performing a method of claim 4 .
14. A method of integrating real three-dimensional (3D) space sensing with a virtual reality head mounted device that renders a virtual background and one or more virtual objects, the method including:
at a first time, using a sensor attached to the virtual reality head mounted device, sensing a first position of at least one hand in a first reference frame of a three-dimensional (3D) sensory space, including tracking at least portions of the hand for causing interaction between the hand and the virtual reality head mounted device;
at a second time,
sensing a second position of the hand;
responsive to repositioning of the head mounted device and the attached sensor due to body movement, sensing motion of the attached sensor and calculating a second reference frame that accounts for repositioning of the attached sensor; and
calculating a transformation that renders the first position in the first reference frame and the second position in the second reference frame into a common reference frame; and
transforming the first and second positions of the hand into the common reference frame, wherein the common reference frame has a fixed point of reference and an initial orientation of axes.
15. The method of claim 14 , wherein the common reference frame is a world reference frame that does not change as the attached sensor is repositioned.
16. The method of claim 14 , wherein the common reference frame is the second reference frame.
17. The method of claim 14 , wherein the attached sensor is integrated into a unit with the virtual reality head mounted device.
18. The method of claim 14 , wherein the transforming the first and second positions of the hand into the common reference frame further includes applying at least one affine transformation.
19. A system of integrating real three-dimensional (3D) space sensing with a virtual reality head mounted device that renders a virtual background and one or more virtual objects, the system including:
a processor and a computer readable storage medium storing computer instructions configured to cause the processor to implement the method of claim 14 .
20. One or more non-transitory computer readable media having instructions stored thereon for performing a method of claim 14 .