Wrist-stabilized projection casting
A wrist-pose isolation system can infer a wrist pose (e.g., the user's hand position relative to her forearm) and can reduce wrist-induced jitter for projection casting in an XR environment. A user's projection cast can be determined as a combination of a “low-wrist contribution” component (e.g., a body-and-arm component) and a “high-wrist contribution” component (e.g., the pose of the wrist with respect to the arm). Using input from a gesture-tracking system, the contribution of the user's wrist pose to the user's current projection cast is calculated as a “wrist-contribution vector.” A projection cast direction can be determined as the interpolation of the current low-wrist contribution component and the high-wrist contribution component. This interpolation can be performed by weighting each by a specified amount and combining them.
1. A method for interacting with objects in an artificial reality environment, the method comprising:
tracking portions of a user's hand;
determining, from the tracked portions of the user's hand, a low-wrist contribution point on the user's hand;
determining, from the tracked portions of the user's hand, a high-wrist contribution point on the user's hand;
calculating a low-wrist projection vector based on the low-wrist contribution point on the user's hand;
calculating a high-wrist projection vector based on the high-wrist contribution point on the user's hand;
calculating a resultant projection vector by interpolating between the low-wrist projection vector and the high-wrist projection vector; and
casting a projection ray, in the artificial reality environment, along the direction of the resultant projection vector.
2. The method of claim 1 , wherein the interpolating between the low-wrist projection vector and the high-wrist projection vector comprises:
weighting each of the low-wrist projection vector and the high-wrist projection vector according to contribution factors pre-defined for each; and
combining the weighted low-wrist projection vector and the high-wrist projection vector.
3. The method of claim 1 , further comprising:
determining a second low-wrist projection vector based on an updated position of the low-wrist contribution point on the user's hand; and
resultant projection vector based on the second low-wrist projection vector.
4. The method of claim 1 , wherein the high-wrist contribution point is at a pinch point between the user's thumb and index finger or at a tip of a finger of the user.
5. The method of claim 1 , wherein the low-wrist contribution point on the user's hand is at a base of a wrist of the user.
6. The method of claim 1 , wherein the projection ray is used to perform an action including one or more of: selecting one or more objects, moving one or more objects, operating a control assigned to an object, interacting with a menu presented by an object, or any combination thereof.
7. The method of claim 1 , wherein the low-wrist projection vector and the high-wrist projection vector are each further based on an origin point on the body of the user that is a point between a shoulder and a hip of the user.
8. The method of claim 1 ,
wherein the low-wrist projection vector and the high-wrist projection vector are each further based on an origin point on the body of the user that is a point between a shoulder and a hip of the user; and
wherein the point between the shoulder and the hip of the user is selected according to a determined angle of the user's gaze.
9. A non-transitory computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform a process for casting a projection in an artificial reality environment, the process comprising:
tracking portions of a user's hand;
determining, from the tracked portions of the user's hand, a low-wrist contribution point on the user's hand;
determining, from the tracked portions of the user's hand, a high-wrist contribution point on the user's hand;
calculating a low-wrist projection vector based on the low-wrist contribution point on the user's hand;
calculating a high-wrist projection vector based on the high-wrist contribution point on the user's hand;
calculating a resultant projection vector by interpolating between the low-wrist projection vector and the high-wrist projection vector; and
casting a projection ray, in the artificial reality environment, along the direction of the resultant projection vector.
10. The non-transitory computer-readable storage medium of claim 9 , wherein the interpolating between the low-wrist projection vector and the high-wrist projection vector comprises:
weighting each of the low-wrist projection vector and the high-wrist projection vector according to contribution factors pre-defined for each; and
combining the weighted low-wrist projection vector and the high-wrist projection vector.
11. The non-transitory computer-readable storage medium of claim 9 , wherein the interpolating between the low-wrist projection vector and the high-wrist projection vector comprises:
determining a gaze vector between an eye of the user and the high-wrist contribution point; and
combining the low-wrist projection vector, the high-wrist projection vector, and the gaze vector.
12. The non-transitory computer-readable storage medium of claim 9 , wherein the high-wrist contribution point is at a pinch point between the user's thumb and index finger or at a tip of a finger of the user.
13. The non-transitory computer-readable storage medium of claim 9 , wherein the low-wrist contribution point on the user's hand is at a base of a wrist of the user.
14. The non-transitory computer-readable storage medium of claim 9 , wherein the projection ray is used to perform an action including one or more of: selecting one or more objects, moving one or more objects, operating a control assigned to an object, interacting with a menu presented by an object, or any combination thereof.
15. The non-transitory computer-readable storage medium of claim 9 , wherein the low-wrist projection vector and the high-wrist projection vector are each further based on an origin point on the body of the user that is a point between a shoulder and a hip of the user.
16. The non-transitory computer-readable storage medium of claim 9 ,
wherein the low-wrist projection vector and the high-wrist projection vector are each further based on an origin point on the body of the user that is a point between a shoulder and a hip of the user; and
wherein the point between the shoulder and the hip of the user is selected according to a determined angle of the user's gaze.
17. A computing system for interacting with objects in an artificial reality environment, the computing system comprising:
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:
tracking portions of a user's hand;
determining, from the tracked portions of the user's hand, a low-wrist contribution point on the user's hand;
determining, from the tracked portions of the user's hand, a high-wrist contribution point on the user's hand;
calculating a low-wrist projection vector based on the low-wrist contribution point on the user's hand;
calculating a high-wrist projection vector based on the high-wrist contribution point on the user's hand;
calculating a resultant projection vector by interpolating between the low-wrist projection vector and the high-wrist projection vector; and
casting a projection ray, in the artificial reality environment, along the direction of the resultant projection vector.
18. The computing system of claim 17 , wherein the interpolating between the low-wrist projection vector and the high-wrist projection vector comprises:
weighting each of the low-wrist projection vector and the high-wrist projection vector according to contribution factors pre-defined for each; and
combining the weighted low-wrist projection vector and the high-wrist projection vector.
19. The computing system of claim 17 , wherein the interpolating between the low-wrist projection vector and the high-wrist projection vector comprises:
determining a gaze vector between an eye of the user and the high-wrist contribution point; and
combining the low-wrist projection vector, the high-wrist projection vector, and the gaze vector.
20. The computing system of claim 17 ,
wherein the high-wrist contribution point is at a pinch point between the user's thumb and index finger or at a tip of a finger of the user; and
wherein the low-wrist contribution point on the user's hand is at a base of a wrist of the user.