Drift cancelation for portable object detection and tracking
The technology disclosed can provide capabilities such as using motion sensors and/or other types of sensors coupled to a motion-capture system to monitor motions within a real environment. A virtual object can be projected to a user of a portable device integrated into an augmented rendering of a real environment about the user. Motion information of a user body portion is determined based at least in part upon sensory information received from imaging or acoustic sensory devices. Control information is communicated to a system based in part on a combination of the motion of the portable device and the detected motion of the user. The virtual device experience can be augmented in some implementations by the addition of haptic, audio and/or other sensory information projectors.
1. A method of integrating real three-dimensional (3D) space sensing with an augmented reality or virtual reality head mounted device, the method including:
obtaining a first position, at a first time t0, of a hand in a first reference frame of a three-dimensional (3D) sensory space;
obtaining, in the 3D sensory space, a second position, at a second time t1, of the hand different from the first position responsive to repositioning of the virtual reality head mounted device and an attached sensor wherein the hand has not moved in the 3D sensory space between t0 and t1;
obtaining an actual second position for the hand, the actual second position generated by:
obtaining a second reference frame that accounts for repositioning of the attached sensor; and
obtaining transformed first and second positions of the hand into a common reference frame using a transformation that renders the first position in the first reference frame and the second position in the second reference frame into the common reference frame, wherein the common reference frame has a fixed point of reference and an initial orientation of axes, whereby the second position is transformed to the actual second position; and
providing to another process for display, a virtual representation of the hand at one or more of the first position and the second position, wherein the virtual representation is to be rendered in a virtual environment of the augmented reality or virtual reality head mounted device.
2. The method of claim 1 , wherein the common reference frame is a world reference frame that does not change as the attached sensor is repositioned.
3. The method of claim 1 , wherein the common reference frame is the second reference frame.
4. The method of claim 1 , wherein the transformed first and second positions of the hand into a common reference frame further includes applying an affine transformation.
5. The method of claim 1 , further including determining an orientation of the hand at the first position with respect to the first reference frame and causing the display of the hand accordingly.
6. The method of claim 1 , further including determining an orientation of the hand at the second position with respect to the second reference frame and causing the display of the hand accordingly.
7. The method of claim 1 , wherein determining a position of the hand at the first position further includes calculating a translation of the hand with respect to the common reference frame and causing the display of the hand accordingly.
8. The method of claim 1 , wherein determining a position of the hand at the second position further includes calculating a translation of the hand with respect to the common reference frame and causing the display of the hand accordingly.
9. One or more non-transitory computer readable media having instructions stored thereon for performing a method of claim 1 .
10. The method of claim 1 , wherein the common reference frame has a fixed point of reference and an initial orientation of axes.
11. A method of integrating real three-dimensional (3D) space sensing with an augmented reality or virtual reality head mounted device that renders a virtual background and one or more virtual objects, the method including:
at a first time, obtaining a first position of a hand in a first reference frame of a three-dimensional (3D) sensory space;
at a second time,
obtaining a second position of the hand;
obtaining a second reference frame that accounts for repositioning of an attached sensor; and
obtaining transformed first and second positions of the hand into a common reference frame using a transformation that renders the first position in the first reference frame and the second position in the second reference frame into a common reference frame.
12. The method of claim 11 , wherein the common reference frame is a world reference frame that does not change as the attached sensor is repositioned.
13. The method of claim 11 , wherein the common reference frame is the second reference frame.
14. The method of claim 11 , wherein the attached sensor is integrated into a unit with the augmented reality or virtual reality head mounted device.
15. The method of claim 11 , wherein the transforming first and second positions of the hand into the common reference frame further includes applying at least one affine transformation.
16. One or more non-transitory computer readable media having instructions stored thereon for performing a method of claim 11 .
17. A system of integrating real three-dimensional (3D) space sensing with an augmented reality or virtual reality head mounted device, the system including:
a processor and a computer readable storage medium storing computer instructions configured to cause the processor to:
obtain a first position, at a first time t0, of a hand in a first reference frame of a three-dimensional (3D) sensory space, including at least portions of the hand;
obtain, in the 3D sensory space, a second position, at a second time t1, of the hand different from the first position responsive to repositioning of the virtual reality head mounted device and an attached sensor wherein the hand has not moved in the 3D sensory space between t0 and t1; and
obtain an actual second position for the hand, the actual second position generated by:
obtaining a second reference frame that accounts for repositioning of the attached sensor; and
obtaining transformed first and second positions of the hand into a common reference frame using a transformation that renders the first position in the first reference frame and the second position in the second reference frame into the common reference frame, whereby the second position is transformed to the actual second position.
18. A system of integrating real three-dimensional (3D) space sensing with an augmented reality or virtual reality head mounted device that renders a virtual background and one or more virtual objects, the system including:
a processor and a computer readable storage medium storing computer instructions configured to cause the processor to:
at a first time, obtain a first position of a hand in a first reference frame of a three-dimensional (3D) sensory space;
at a second time,
obtain a second position of the hand;
obtain a second reference frame that accounts for repositioning of an attached sensor; and
obtain transformed first and second positions of the hand into a common reference frame using a transformation that renders the first position in the first reference frame and the second position in the second reference frame into a common reference frame.