Interactions with 3D virtual objects using poses and multiple-DOF controllers
A wearable system can comprise a display system configured to present virtual content in a three-dimensional space, a user input device configured to receive a user input, and one or more sensors configured to detect a user's pose. The wearable system can support various user interactions with objects in the user's environment based on contextual information. As an example, the wearable system can adjust the size of an aperture of a virtual cone during a cone cast (e.g., with the user's poses) based on the contextual information. As another example, the wearable system can adjust the amount of movement of virtual objects associated with an actuation of the user input device based on the contextual information.
1. A system comprising:
a display system of a wearable device configured to provide a three-dimensional (3D) view to a user and permit a user interaction with objects in a field of view (FOV) of the user, the FOV comprising a portion of an environment around the user that is capable of being perceived by the user via the display system;
a hardware processor in communication with the display system, the hardware processor programmed to:
initiate a cone cast of a virtual cone;
determine contextual information in the FOV of the user;
anchor a proximal end of the virtual cone to an anchoring location associated with a physical object designated by the user via a user input, wherein the user is enabled to anchor the proximal end to a pointing device and wherein the user is enabled to anchor the proximal end to the user, wherein movement of the anchoring location, designated by the user, associated with the physical object causes corresponding movement of the virtual cone so that the proximal end of the virtual cone remains at the anchoring location associated with the physical object, wherein the virtual cone further comprises a distal end, or a depth, and the hardware processor is programmed to anchor the the distal end, and/or the depth of the virtual cone to the physical object based on at least one of: a user input, a body gesture, a body pose, a direction of gaze, or a voice command;
determine one or more objects within the virtual cone anchored to the physical object; and
perform an action on one or more of the objects within the virtual cone anchored to the physical object.
2. The system of claim 1 , wherein the hardware processor is further programmed to determine the contextual information regarding the environment of the user, wherein the contextual information comprises at least one of: a type, a layout, a location, a size, a distance, or a density of objects within the FOV of the user.
3. The system of claim 2 , wherein to determine the density of objects within the FOV of the user, the hardware processor is programmed to:
calculate a number of objects in the FOV;
calculate a fraction of the FOV that is covered by the objects; or
calculate a contour map for the objects.
4. The system of claim 2 , wherein the contextual information comprises at least one of: a preference of the user, a physical condition associated with the user, or information associated with the environment.
5. The system of claim 1 , wherein an aperture is dynamically-adjustable, and wherein to update the dynamically-adjustable aperture, the hardware processor is programmed to select the aperture between a minimum size and a maximum size.
6. The system of claim 5 , wherein the minimum size is zero.
7. The system of claim 1 , wherein the action comprises rendering a focus indicator associated with the one or more objects.
8. The system of claim 5 , wherein the hardware processor is further programmed to detect a collision between the virtual cone and one or more objects in the environment, wherein to detect the collision between the virtual cone and the one or more objects, the hardware processor is programmed to determine that the one or more objects intersect with a virtual surface of the virtual cone or fall within the aperture of the virtual cone.
9. The system of claim 1 , wherein the one or more objects comprise multiple collided objects, and the hardware processor is programmed to apply an occlusion disambiguation technique to the multiple collided objects to identify an occluded object or to determine a depth ordering or position among occluded objects.
10. The system of claim 1 , wherein the one or more objects comprise multiple collided objects, and the hardware processor is programmed to present a user interface element to select one or more of the multiple collided objects.
11. The system of claim 1 , wherein the action with the one or more objects comprises one or more of:
a selection of the one or more objects;
a movement of the one or more objects;
display of a menu or toolbar associated with the one or more objects; or
a game operation on a virtual avatar in a game.
12. The system of claim 1 , wherein the virtual cone comprises a central ray, wherein an aperture of the virtual cone is transverse to the central ray, and a direction of the central ray is based on a pose of the user.
13. The system of claim 1 , wherein the hardware processor is configured to:
determine a density of objects in the FOV of the user; and
update an aperture of the virtual cone based on the density of objects in the FOV of the user.
14. The system of claim 1 , wherein the hardware processor is further programmed to detect a collision between the virtual cone and one or more objects in the environment, wherein the virtual cone comprises a distal end, and to detect the collision, the hardware processor is programmed to scan for collisions with the one or more objects within the distal end of the virtual cone.
15. The system of claim 1 , wherein the virtual cone further comprises a distal end or a depth, and the hardware processor is programmed to anchor the distal end or the depth of the virtual cone based in part on the contextual information.
16. The system of claim 1 , further comprising:
a pose sensor configured to acquire pose data associated with a pose of the user; and
wherein the hardware processor is programmed to:
translate the virtual cone based on the pose data associated with the pose of the user.
17. The system of claim 1 , wherein the hardware processor is further programmed to:
resize an aperture of the virtual cone based at least in part on the contextual information in the FOV of the user.
18. The system of claim 1 , wherein to initiate the cone cast, the hardware processor is programmed to extend a distal end of the virtual cone until the distal end reaches a termination threshold.
19. The system of claim 18 , wherein the termination threshold comprises a threshold distance or a boundary of the environment.
20. The system of claim 1 , wherein the hardware processor is further programmed to:
render to the user, via the display system, a visual representation of at least a portion of the virtual cone.