Simultaneous localization and mapping (SLAM) compensation for gesture recognition in virtual, augmented, and mixed reality (xR) applications
Embodiments of systems and methods for Simultaneous Localization and Mapping (SLAM) compensation for gesture recognition in virtual, augmented, and mixed reality (xR) applications are described. In an illustrative, non-limiting embodiment, an Information Handling System (IHS) may include a processor and a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution by the processor, cause the IHS to: receive a first set of images from a SLAM camera, wherein the first set of images captures movement of a Head-Mounted Device (HMD); calculate a transformation matrix based upon the first set of images; receive a second set of images from a gesture camera, wherein the second set of images captures a gesture; and apply the transformation matrix to the second set of images prior to recognizing the gesture.
1. An Information Handling System (IHS), comprising:
a processor; and
a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution by the processor, cause the IHS to:
receive a first set of images from a Simultaneous Localization and Mapping (SLAM) camera, wherein the first set of images captures movement of a Head-Mounted Device (HMD);
detect, based upon the first set of images, movement of the HMD above a threshold value;
in response to the detection, reduce a number of candidate gestures;
calculate a transformation matrix based upon the first set of images;
receive a second set of images from a gesture camera, wherein the second set of images captures a gesture; and
apply the transformation matrix to the second set of images prior to recognizing the gesture.
2. The IHS of claim 1 , wherein the SLAM camera comprises an infra-red (IR) camera, and wherein the gesture camera comprises a visible spectrum camera.
3. The IHS of claim 1 , wherein the first and second sets of images are received concurrently.
4. The IHS of claim 1 , wherein the transformation matrix comprises at least one of: a translation matrix, a rotation matrix, or a scaling matrix.
5. The IHS of claim 1 , wherein to apply the transformation matrix, the program instructions, upon execution by the processor, further cause the IHS to:
convert the second set of images from a gesturing coordinate system into a global coordinate system;
multiply the one or more images of the second set of images by the transformation matrix to produce SLAM-compensated images; and
convert the SLAM-compensated images from the global coordinate system into the gesturing coordinate system.
6. The IHS of claim 5 , wherein to produce the SLAM-compensated images, the program instructions, upon execution by the processor, further cause the IHS to subtract an HMD velocity from a gesture velocity.
7. The IHS of claim 1 , wherein the SLAM camera and the gesture camera are offset by a physical distance.
8. The IHS of claim 7 , wherein the program instructions, upon execution by the processor, further cause the IHS to apply an offset matrix to the second set of images prior to recognizing the gesture, wherein the offset matrix corresponds to the physical distance.
9. The IHS of claim 1 , wherein the movement is detected as a rotation to the right, and wherein the program instructions, upon execution by the processor, further cause the IHS to eliminate a right swiping gesture from the candidate gestures.
10. The IHS of claim 1 , wherein the movement is detected as a rotation to the right, and wherein the program instructions, upon execution by the processor, further cause the IHS to eliminate a left swiping gesture from the candidate gestures.
11. The IHS of claim 1 , wherein the movement is detected as a rotation to the left, and wherein the program instructions, upon execution by the processor, further cause the IHS to eliminate a right swiping gesture from the candidate gestures.
12. The IHS of claim 1 , wherein the movement is detected as a rotation to the left, and wherein the program instructions, upon execution by the processor, further cause the IHS to eliminate a left swiping gesture from the candidate gestures.
13. The IHS of claim 1 , wherein the movement is detected as a rotation upward, and wherein the program instructions, upon execution by the processor, further cause the IHS to eliminate an upward swiping gesture from the candidate gestures.
14. The IHS of claim 1 , wherein the movement is detected as a rotation upward, and wherein the program instructions, upon execution by the processor, further cause the IHS to eliminate a downward swiping gesture from the candidate gestures.
15. The IHS of claim 1 , wherein the movement is detected as a rotation downward, and wherein the program instructions, upon execution by the processor, further cause the IHS to eliminate an upward swiping gesture from the candidate gestures.
16. The IHS of claim 1 , wherein the movement is detected as a rotation downward, and wherein the program instructions, upon execution by the processor, further cause the IHS to eliminate a downward swiping gesture from the candidate gestures.
17. The IHS of claim 1 , wherein the movement is detected as a translation, and wherein the program instructions, upon execution by the processor, further cause the IHS to eliminate a forward or backward swiping gesture from the candidate gestures.
18. A hardware memory of an Information Handling System (IHS), the hardware memory having program instructions stored thereon that, upon execution, cause the IHS to:
receive a first set of images from a Simultaneous Localization and Mapping (SLAM) camera, wherein the first set of images captures movement of a Head-Mounted Device (HMD);
calculate a transformation matrix based upon the first set of images;
receive a second set of images from a gesture camera, wherein the second set of images captures a gesture, and wherein the SLAM camera and the gesture camera are offset by a physical distance;
apply an offset matrix to the second set of images prior to recognizing the gesture, wherein the offset matrix corresponds to the physical distance; and
apply the transformation matrix to the second set of images prior to recognizing the gesture.
19. A method, comprising:
receiving a first set of images from a Simultaneous Localization and Mapping (SLAM) camera, wherein the first set of images captures movement of a Head-Mounted Device (HMD);
calculating a transformation matrix based upon the first set of images;
receiving a second set of images from a gesture camera, wherein the second set of images captures a gesture;
converting the second set of images from a gesturing coordinate system into a global coordinate system;
multiplying the one or more images of the second set of images by the transformation matrix to produce SLAM-compensated images, at least in part, by subtracting an HMD velocity from a gesture velocity;
converting the SLAM-compensated images from the global coordinate system into the gesturing coordinate system; and
applying the transformation matrix to the second set of images prior to recognizing the gesture.