Single-handed microgesture inputs
A method for single-handed microgesture input comprises receiving hand tracking data for a hand of a user. A set of microgesture targets that include software functions are assigned to positions along a length of a first finger. A visual affordance including indicators for two or more assigned microgesture targets is provided to the user. The received hand tracking data is analyzed by a gesture recognition machine. A location of a thumbtip of the hand of the user is determined relative to the positions along the first finger. Responsive to determining that the thumbtip is within a threshold distance of the first finger at a first position along the length of the first finger, an indicator for a corresponding first microgesture target is augmented, and then further augmented based on a duration the thumbtip is at the first position. Responsive to detecting a confirmation action, the corresponding microgesture target executes.
1. A method for single-handed microgesture input, comprising:
receiving hand tracking data for a hand of a user;
assigning a set of microgesture targets to a plurality of positions along a length of a first finger of the hand of the user, each microgesture target including one or more software functions;
providing a visual affordance to the user, the visual affordance including indicators for two or more assigned microgesture targets; and
based on an output of a gesture recognition machine configured to analyze the received hand tracking data:
determining a location of a thumbtip of the hand of the user relative to the plurality of positions along the length of the first finger;
responsive to determining that the thumbtip is within a threshold distance of the first finger at a first position along the length of the first finger, augmenting an indicator of a first microgesture target corresponding to such first position; and
further augmenting the indicator of the first microgesture target based on a duration the thumbtip is at the first position; and
responsive to detecting a confirmation action, executing the corresponding first microgesture target.
2. The method of claim 1 , wherein the visual affordance includes a first affordance that is tethered to the hand of the user at a predetermined distance, so that the indicators for the assigned microgesture targets are not coincident with corresponding positions along the length of the first finger.
3. The method of claim 2 , wherein an appearance of the first affordance is progressively reduced based on the user repeatedly selecting microgesture targets over time.
4. The method of claim 2 , wherein the visual affordance includes a second affordance that is presented within a field-of-view of the user at a display location not dependent on a location of the hand of the user.
5. The method of claim 4 , wherein the display location of the second affordance is based on a gaze direction of the user.
6. The method of claim 4 , wherein the second affordance is body-locked to the user.
7. The method of claim 1 , wherein the indicators for the microgesture targets are presented in an order based on corresponding positions along the length of the first finger.
8. The method of claim 1 , further comprising:
activating single-handed microgesture input responsive to recognizing an activation command from the user.
9. The method of claim 1 , further comprising:
responsive to determining that the thumbtip has moved to a second position along the length of the first finger, augmenting an indicator for a corresponding second microgesture target; and
further augmenting the indicator of the second microgesture target based on a duration the thumbtip is within the threshold distance of the second position.
10. The method of claim 9 , wherein the gesture recognition machine is configured to determine a position of the thumbtip along a continuum of positions along the length of the first finger.
11. The method of claim 10 , where the continuum of positions along the length of the first finger are divided into to a plurality of discrete positions for microgesture target assignment.
12. The method of claim 9 , wherein the visual affordance displays a subset of the indicators for the assigned microgesture targets, the displayed subset of indicators based on the location of the thumbtip of the hand of the user relative to the plurality of positions along the length of the first finger.
13. The method of claim 1 , wherein the confirmation action includes a dwell time at the first position greater than a threshold.
14. The method of claim 1 , wherein the confirmation action includes a predetermined gesture.
15. The method of claim 1 , wherein sets of microgesture targets are assigned to pluralities of positions along a length of two or more fingers of the hand of the user.
16. The method of claim 1 , further comprising:
receiving hand tracking data for a second hand of a user;
assigning a second set of microgesture targets to a plurality of positions along a length of a first finger of the second hand of the user;
based on output of the gesture recognition machine, determining a location of a thumbtip of the second hand of the user relative to the plurality of positions along the length of the first finger of the second hand of the user; and
responsive to detecting a confirmation action, executing the corresponding microgesture target.
17. The method of claim 1 , wherein receiving hand tracking data for the hand of the user includes:
receiving depth data for an environment;
fitting a virtual skeleton to point clouds of the received depth data;
assigning hand joints to the virtual skeleton based at least in part on image data of the user performing the first-stage gesture and the second-stage gesture; and
tracking positions of the assigned hand joints across sequential depth images.
18. A system for a head-mounted display, comprising:
one or more outward-facing image sensors;
a display device configured to present virtual content; and
a gesture recognition machine configured to:
receive hand tracking data for a hand of a user via the one or more outward facing image sensors;
assign a set of microgesture targets to a plurality of positions along a length of a first finger of the hand of the user, each microgesture target including one or more software functions;
provide a visual affordance to the user, the visual affordance including indicators for two or more assigned microgesture targets;
determine a location of a thumbtip of the hand of the user relative to the plurality of positions along the length of the first finger;
responsive to determining that the thumbtip is within a threshold distance of the first finger at a first position along the length of the first finger, augment an indicator for a corresponding first microgesture target;
further augment the indicator of the first microgesture target based on a duration the thumbtip is at the first position; and
responsive to detecting a confirmation action, execute the corresponding microgesture target.
19. The system of claim 18 , wherein the gesture recognition machine is further configured to:
responsive to determining that the thumbtip has moved to a second position along the length of the first finger, augment an indicator for a corresponding second microgesture target; and
further augment the indicator of the second microgesture target based on a duration the thumbtip is within the threshold distance of the second position.
20. A method for single-handed microgesture input, comprising:
receiving hand tracking data for a hand of a user;
assigning a set of microgesture targets to a plurality of positions along a length of a first finger of the hand of the user;
providing a visual affordance to the user, the visual affordance including indicators for two or more assigned microgesture targets, the visual affordance including a first affordance that is tethered to the hand of the user at a threshold distance, so that the indicators for the assigned microgesture targets are not coincident with the corresponding positions along the length of the first finger, and further including a second affordance that is presented within a field-of-view of the user at a position not dependent on a position of the hand of the user; and
based on an output of a gesture recognition machine configured to analyze the received hand tracking data:
determining a location of a thumbtip of the hand of the user along a continuum of positions along the length of the first finger, the continuum of positions along the length of the first finger divided into to a plurality of discrete positions for microgesture target assignment;
responsive to determining that the thumbtip is within a threshold distance of the first finger at a first position along the length of the first finger, augmenting an indicator for a corresponding first microgesture target; and
further augmenting the indicator of the first microgesture target based on a duration the thumbtip is at the first position; and
responsive to detecting a confirmation action, executing the corresponding microgesture target.