Determining user intent during two-handed gestures
Devices, systems, and methods that interpret user activity, such as two hand gestures, as user interactions with virtual elements (e.g., UI elements) positioned within in a 3D space such as an XR environment. For example, an example process may include receiving data corresponding to user activity involving two hands of a user in a 3D coordinate system. The process may further include identifying actions performed by the two hands based on the data corresponding to the user activity, each of the two hands performing one of the identified actions. The process may further include determining whether the identified actions satisfy a criterion based on the data corresponding to the user activity. The process may further include determining to associate each of the actions performed by each of the two hands with a user intended action in accordance with determining that the identified actions satisfy the criterion.
1 . A method comprising:
at a device having a processor and one or more sensors:
receiving data corresponding to user activity involving two hands of a user in a 3D coordinate system;
determining one or more gestures performed by the two hands based on the data corresponding to the user activity;
identifying actions performed by the two hands based on the determined one or more gestures, each of the two hands performing one of the identified actions;
determining a delta corresponding to coordinate differences between the two hands based on tracking each hand during the user activity;
decomposing the delta into a plurality of components of a motion; and
suppressing a component of the plurality of components of the decomposed delta associated with movement outside of the identified actions in order to interpret the user activity.
2 . The method of claim 1 , wherein the user activity is determined to provide a single-handed gesture.
3 . The method of claim 1 , wherein the user activity is determined to provide a dual-handed gesture.
4 . The method of claim 1 , wherein the identified action comprises a pan interaction, a zoom interaction, or a rotation of one or more elements of a user interface.
5 . The method of claim 1 , wherein identifying actions performed by the two hands comprises:
determining a velocity of movement of the two hands; and
determining, based on the velocity of movement of the two hands, whether the two hands are moving away or towards each other or if the two hands satisfy a same direction criterion.
6 . The method of claim 5 , wherein, in response to determining that the two hands are moving away or towards each other, applying a suppression of a determined component of the movement of the two hands for determining a user intended action.
7 . The method of claim 5 , wherein, in response to determining that the two hands satisfy the same direction criterion, applying a limitation of a suppression of a determined component of the movement of the two hands for determining a user intended action.
8 . The method of claim 7 , wherein the limitation of the suppression of the determined component of the movement of the two hands is based on a weighted scale.
9 . The method of claim 1 , wherein identifying actions performed by the two hands is based on determining whether the user activity comprises motion associated with a pan motion, a zoom motion, a rotation motion, or a combination thereof.
10 . The method of claim 1 , wherein determining the identified action comprises determining that the user activity is associated with two simultaneous two-handed gestures.
11 . The method of claim 1 , wherein determining the identified action comprises determining that the user activity is associated with simultaneous one-handed gestures.
12 . The method of claim 1 , further comprising:
using at least a portion of the identified actions performed by the two hands to directly enable a functional component displayed within an extended reality (XR) environment.
13 . The method of claim 1 , wherein suppressing the component of the plurality of components of the decomposed delta associated with movement outside of the identified actions in order to interpret the user activity comprises applying a suppression of a particular movement component based on the identified action associated with the movement of the two hands.
14 . The method of claim 1 , further comprising:
interpreting user input based on the suppressing the component of the plurality of components of the decomposed delta associated with movement outside of the identified actions.
15 . A device comprising:
one or more sensors;
a non-transitory computer-readable storage medium; and
one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising:
receiving data corresponding to user activity involving two hands of a user in a 3D coordinate system;
determining one or more gestures performed by the two hands based on the data corresponding to the user activity;
identifying actions performed by the two hands based on the determined one or more gestures, each of the two hands performing one of the identified actions;
determining a delta corresponding to coordinate differences between the two hands based on tracking each hand during the user activity;
decomposing the delta into a plurality of components of a motion; and
suppressing a component of the plurality of components of the decomposed delta associated with movement outside of the identified actions in order to interpret the user activity.
16 . The device of claim 15 , wherein the user activity is determined to provide a single-handed gesture.
17 . The device of claim 15 , wherein the user activity is determined to provide a dual-handed gesture.
18 . The device of claim 15 , wherein the identified action comprises a pan interaction, a zoom interaction, or a rotation of one or more elements of a user interface.
19 . The device of claim 15 , wherein identifying actions performed by the two hands comprises:
determining a velocity of movement of the two hands; and
determining, based on the velocity of movement of the two hands, whether the two hands are moving away or towards each other or if the two hands satisfy a same direction criterion.
20 . The device of claim 19 , wherein, in response to determining that the two hands are moving away or towards each other, applying a suppression of a determined component of the movement of the two hands for determining a user intended action.
21 . The device of claim 19 , wherein, in response to determining that the two hands satisfy the same direction criterion, applying a limitation of a suppression of a determined component of the movement of the two hands for determining a user intended action.
22 . A non-transitory computer-readable storage medium, storing program instructions executable on a device to perform operations comprising:
receiving data corresponding to user activity involving two hands of a user in a 3D coordinate system;
determining one or more gestures performed by the two hands based on the data corresponding to the user activity;
identifying actions performed by the two hands based on the determined one or more gestures, each of the two hands performing one of the identified actions;
determining a delta corresponding to coordinate differences between the two hands based on tracking each hand during the user activity;
decomposing the delta into a plurality of components of a motion; and
suppressing a component of the plurality of components of the decomposed delta associated with movement outside of the identified actions in order to interpret the user activity.