Scalable gesture and eye-gaze tracking in virtual, augmented, and mixed reality (xR) applications
Systems and methods for scalable gesture and eye-gaze tracking in virtual, augmented, and mixed reality (xR) applications are described. In some embodiments, an Information Handling System (IHS) may include a processor and a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution, cause the IHS to: monitor utilization of an IHS resource; capture an image of a gesture performed by a user wearing a Head-Mounted Device (HMD) during an xR application; and generate a feature vector usable to identify the gesture based upon the image, wherein the feature vector has a maximum number of features selected based upon the IHS resource utilization.
1. An Information Handling System (IHS), comprising:
a processor; and
a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution, cause the IHS to:
monitor utilization of an IHS resource;
capture an image of a gesture performed by a user wearing a Head-Mounted Device (HMD) during a virtual, augmented, or mixed reality (xR) application;
generate a feature vector usable to identify the gesture based upon the image, wherein the feature vector has a maximum number of features selected, wherein the maximum number of features varies as the utilization level of the IHS resource varies and is automatically selected based upon the current IHS resource utilization level;
rank features usable to identify a first gesture and a second gesture;
determine that the second gesture is indistinguishable from the first gesture based upon a comparison between a higher-ranked subset of features of the first gesture a higher-ranked subset of features of the second gesture; and
disable recognition of the second gesture if the second gesture is indistinguishable from the first gesture.
2. The IHS of claim 1 , wherein the IHS resource comprises the processor.
3. The IHS of claim 1 , wherein the IHS resource comprises the memory.
4. The IHS of claim 1 , wherein the maximum number features is selected to maintain the IHS resource utilization below a predetermined level.
5. The IHS of claim 1 , wherein the maximum number of features is selected based upon a calibration curve that associates a reduction in a number of used features with a corresponding reduction in IHS resource utilization.
6. The IHS of claim 1 , wherein the maximum number of features is selected based upon a calibration curve that associates an increase in number of used features with a corresponding increase in IHS resource utilization.
7. The IHS of claim 1 , wherein the program instructions, upon execution, further cause the IHS to:
identify an increase in IHS resource utilization; and
reduce the maximum number of features to prevent reaching a maximum IHS resource utilization.
8. The IHS of claim 1 , wherein the program instructions, upon execution, further cause the IHS to:
identify a decrease in IHS resource utilization; and
increase the maximum number of features without reaching a maximum IHS resource utilization.
9. The IHS of claim 1 , wherein to rank the features, the program instructions, upon execution, further cause the IHS to:
calculate a Pearson's Correlation Coefficient (PCC) for each of the features; and
order the features by respective PCCs, wherein a feature having a value closest to zero is ranked highest.
10. The IHS of claim 1 , wherein the program instructions, upon execution, further cause the IHS to select the first and second gestures based upon an Euclidian distance between the first and second gestures.
11. The IHS of claim 1 , wherein the program instructions, upon execution, further cause the IHS to select one of a plurality of eye-gaze tracking (EGT) methods based upon the IHS resource utilization.
12. A method, comprising:
monitoring utilization of an Information Handling System (IHS) resource;
selecting an eye gaze tracking (EGT) method usable to detect eye movement of a user wearing a Head Mounted Device (HMD) coupled to the IHS during a virtual, augmented, or mixed reality (xR) application based upon the IHS resource utilization;
capturing an image of a gesture performed by a user;
generating a feature vector usable to identify the gesture based upon the image, wherein the feature vector has a maximum number of features selected based upon the IHS resource utilization, wherein the maximum number of features varies as the utilization level of the IHS resource varies and is automatically selected based upon the current IHS resource utilization level; and
disabling recognition of the gesture if the maximum number of features renders the gesture indistinguishable from another gesture.
13. The method of claim 12 , wherein the EGT method is selected from the group consisting of: a two-dimensional (2D) method, or a three-dimensional (3D) method.
14. The method of claim 12 , wherein the EGT method is selected based upon a calibration that associates a reduction in EGT features or accuracy, with a corresponding reduction in IHS resource utilization.
15. The method of claim 12 , wherein the EGT method is selected based upon a calibration that associates an increase in EGT features or accuracy with a corresponding increase in IHS resource utilization.
16. A hardware memory device of an Information Handling System (IHS) having program instructions stored thereon that, upon execution by a hardware processor, cause the IHS to:
monitor utilization of an IHS resource;
select an eye gaze tracking (EGT) method based upon the IHS resource utilization, wherein the EGT method is usable to detect eye movement of a user wearing a Head Mounted Device (HMD) during a virtual, augmented, or mixed reality (xR) application;
capture an image of a gesture performed by a user;
generate a feature vector usable to identify the gesture based upon the image, wherein the feature vector has a maximum number of features selected, wherein the maximum number of features varies as the utilization level of the IHS resource varies and is automatically selected based upon the current IHS resource utilization level; and
disable recognition of the gesture if the maximum number of features renders the gesture indistinguishable from another gesture.
17. The hardware memory device of claim 16 , wherein the eye-gaze tracking method is selected based upon a calibration that associates: (i) an increase in EGT features or accuracy with a corresponding increase in IHS resource utilization; or (ii) a reduction in EGT features or accuracy with a corresponding reduction in IHS resource utilization.
18. The hardware memory device of claim 16 , wherein the maximum number of features is selected based upon a calibration that associates: (i) an increase in number of used features with a corresponding increase in IHS resource utilization; or a reduction in a number of used features with a corresponding reduction in IHS resource utilization.