Automatic projection type selection in an artificial reality environment
The present technology relates to artificial reality systems. Such systems provide projections a user can create to specify object interactions. For example, when a user wishes to interact with an object outside her immediate reach, she can use a projection to select, move, or otherwise interact with the distant object. The present technology also includes object selection techniques for identifying and disambiguating between objects, allowing a user to select objects both near and distant from the user. Yet further aspects of the present technology include techniques for interpreting various bimanual (two-handed) gestures for interacting with objects. The present technology further includes a model for differentiating between global and local modes for, e.g., providing different input modalities or interpretations of user gestures.
1. A non-transitory computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform a process for selecting multiple objects in an artificial reality environment, the process comprising:
determining a control point and casting direction based on one or more tracked positions of one or more body parts of a user;
identifying a target object with a corresponding surface type;
selecting a type of the projection based on mappings between surface types and projection types;
generating a projection, of the selected type, that begins at the control point and that extends along the casting direction;
identifying multiple objects that at least partially intersect with at least part of the projection, either simultaneously or at different times; and
selecting the multiple objects.
2. The non-transitory computer-readable storage medium of claim 1 , wherein the type of the projection is equivalent to a cylinder type, specifying that the projection includes a cylinder that extends, beginning at the control point, outward from the user and is centered on the casting direction.
3. The non-transitory computer-readable storage medium of claim 1 ,
wherein the type of the projection is equivalent to a cone type, specifying that the projection includes a cone; and
wherein:
the tip of the cone is at the control point;
the cone extends, from the cone tip, outward from the user; and
the cone is centered, from the cone tip to the center of the circular base of the cone, on the casting direction.
4. The non-transitory computer-readable storage medium of claim 1 ,
wherein the type of the projection is equivalent to a ray type, specifying that the projection includes a ray; and
wherein:
at least one point along the ray corresponds to a hook; and
each particular one of the multiple objects is identified based on the hook intersecting with that particular object when a selection gesture from the user is identified.
5. The non-transitory computer-readable storage medium of claim 1 , wherein:
the type of the projection is equivalent to a distorted cone type;
the casting direction is along a line connecting one of the user's eyes to the control point;
the process further comprises:
continuously monitoring, according to movements of the control point and the one of the user's eyes, a path of a ray that extends away from the user, begins at the control point, and falls along the line that intersects the control point and the user's dominant eye; and
identifying a distorted cone formed with a point of the distorted cone at the one of the user's eyes and the sides of the distorted cone extending through the path of the ray; and
the projection becomes the distorted cone and the multiple objects are identified based on their intersection with at least a part of the distorted cone.
6. The non-transitory computer-readable storage medium of claim 1 , wherein the type of the projection is equivalent to a ray type, specifying that the projection includes a line extending from the control point.
7. A method for selecting multiple objects in an artificial reality environment, the method comprising:
determining a control point and casting direction based on one or more tracked positions of one or more body parts of a user;
identifying a target object with a corresponding surface type;
selecting a type of the projection based on mappings between surface types and projection types;
generating a projection, of the selected type, that begins at the control point and that extends along the casting direction;
identifying multiple objects that at least partially intersect with at least part of the projection, either simultaneously or at different times; and
selecting the multiple objects.
8. The method of claim 7 , wherein the type of the projection is equivalent to a cylinder type, specifying that the projection includes a cylinder that extends, beginning at the control point, outward from the user and is centered on the casting direction.
9. The method of claim 7 ,
wherein the type of the projection is equivalent to a cone type, specifying that the projection includes a cone; and
wherein:
the tip of the cone is at the control point;
the cone extends, from the cone tip, outward from the user; and
the cone is centered, from the cone tip to the center of the circular base of the cone, on the casting direction.
10. The method of claim 7 ,
wherein the type of the projection is equivalent to a ray type, specifying that the projection includes a ray; and
wherein:
at least one point along the ray corresponds to a hook; and
each particular one of the multiple objects is identified based on the hook intersecting with that particular object when a selection gesture from the user is identified.
11. The method of claim 7 , wherein:
the type of the projection is equivalent to a distorted cone type;
the casting direction is along a line connecting one of the user's eyes to the control point;
the method further comprises:
continuously monitoring, according to movements of the control point and the one of the user's eyes, a path of a ray that extends away from the user, begins at the control point, and falls along the line that intersects the control point and the user's dominant eye; and
identifying a distorted cone formed with a point of the distorted cone at the one of the user's eyes and the sides of the distorted cone extending through the path of the ray; and
the projection becomes the distorted cone and the multiple objects are identified based on their intersection with at least a part of the distorted cone.
12. The method of claim 7 , wherein the type of the projection is equivalent to a ray type, specifying that the projection includes a line extending from the control point.
13. A computing system for performing an action in relation to multiple objects in an artificial reality environment, the computing system comprising:
one or more processors; and
one or more memories storing instructions that, when executed by a computing system, cause the computing system to perform a process comprising:
determining a control point and casting direction based on one or more tracked positions of one or more body parts of a user;
identifying a target object with a corresponding surface type;
selecting a type of the projection based on mappings between surface types and projection types;
generating a projection, of the selected type, that begins at the control point and that extends along the casting direction;
identifying multiple objects that at least partially intersect with at least part of the projection, either simultaneously or at different times; and
performing an action in relation to the multiple objects.
14. The computing system of claim 13 , wherein the type of the projection is equivalent to a cylinder type, specifying that the projection includes a cylinder that extends, beginning at the control point, outward from the user and is centered on the casting direction.
15. The computing system of claim 13 ,
wherein the type of the projection is equivalent to a cone type, specifying that the projection includes a cone; and
wherein:
the tip of the cone is at the control point;
the cone extends, from the cone tip, outward from the user; and
the cone is centered, from the cone tip to the center of the circular base of the cone, on the casting direction.
16. The computing system of claim 13 ,
wherein the type of the projection is equivalent to a ray type, specifying that the projection includes a ray; and
wherein:
at least one point along the ray corresponds to a hook; and
each particular one of the multiple objects is identified based on the hook intersecting with that particular object when a selection gesture from the user is identified.
17. The computing system of claim 13 , wherein:
the type of the projection is equivalent to a distorted cone type;
the casting direction is along a line connecting one of the user's eyes to the control point;
the process further comprises:
continuously monitoring, according to movements of the control point and the one of the user's eyes, a path of a ray that extends away from the user, begins at the control point, and falls along the line that intersects the control point and the user's dominant eye; and
identifying a distorted cone formed with a point of the distorted cone at the one of the user's eyes and the sides of the distorted cone extending through the path of the ray; and
the projection becomes the distorted cone and the multiple objects are identified based on their intersection with at least a part of the distorted cone.
18. The computing system of claim 13 , wherein the type of the projection is equivalent to a ray type, specifying that the projection includes a line extending from the control point.