Contextual awareness of user interface menus
Examples of systems and methods for a wearable system to automatically select or filter available user interface interactions or virtual objects are disclosed. The wearable system can select a group of virtual objects for user interaction based on contextual information associated with the user, the user's environment, physical or virtual objects in the user's environment, or the user's physiological or psychological state.
1. A method for generating virtual content in a three-dimensional (3D) physical environment of a user, the method comprising:
under control of a hardware processor;
analyzing data acquired from a pose sensor to identify a pose of a user;
identifying a physical surface in a 3D physical environment of the user and at least an object function associated with an object in the 3D physical environment having the physical surface, based at least partly on the pose;
receiving an indication to initiate an interaction with the physical surface;
determining environmental information associated with the physical surface, the environmental information comprising an affordance of the physical surface, wherein the affordance of the physical surface is determined based on a physical environment in which the physical surface is located;
accessing a set of available user interface options associated with the physical surface based on the affordance of the physical surface;
determining a maximum number of user interface options from the set of available user interface options that can legibly fit in a virtual user interface associated with the physical surface;
selecting, based on the maximum number of user interface options that can legibly fit in the virtual user interface on the affordance of the physical surface, and on the object function associated with the object, a subset of user interface options from the set of available user interface options associated with the physical surface, wherein the subset of user interface options selected is less than the set of available user interface options; and
generating an instruction for presenting the virtual user interface containing the subset of user interface options that can legibly fit in the virtual user interface in a 3D view to the user.
2. The method of claim 1 , wherein the environmental information comprises one or more of: functions of the user's environment, objects in the user's environment, objects and their relationships with the environment, or information associated with the user's environment.
3. The method of claim 1 , wherein the virtual user interface comprises a virtual menu.
4. The method of claim 1 , wherein the method further comprises:
identifying a person with whom the user is interacting, and wherein the selecting of the subset of user interface options is further based on the person with whom the user is interacting.
5. The method of claim 1 , wherein the object is a table, a desk, or a wall.
6. The method of claim 1 , wherein the object function associated with the object is variable depending on the environment, and wherein the selecting of the subset of user interface options is based on the object function associated with the object in view of the environment.
7. The method of claim 1 , wherein the selecting of the subset of user interface options is further based on one or more of an age, gender, educational level, occupation, and preference of the user.
8. The method of claim 1 , wherein the selecting of the subset of user interface options is further based on a physiological state of the user, a psychological state of the user, or autonomic nerve system activity of the user.
9. The method of claim 1 , wherein the selecting of the subset of user interface options is further based on a fiducial marker associated with the object.
10. A method for generating virtual content in a three-dimensional (3D) physical environment of a user, the method comprising:
under control of a hardware processor;
analyzing data acquired from a pose sensor to identify a pose of a user;
identifying a physical surface in a 3D physical environment of the user and at least an object function associated with an object in the 3D physical environment having the physical surface, based at least partly on the pose;
receiving an indication to initiate an interaction with the physical surface;
determining object information associated with the physical surface, the object information comprising an affordance of the physical surface;
accessing a set of available user interface options associated with the physical surface based on the object information;
determining a maximum number of user interface options from the set of available user interface options that can legibly fit in a virtual user interface associated with the physical surface;
selecting, based on the maximum number of user interface options that can legibly fit in the virtual user interface and on the affordance of the physical surface and on the object function associated with the object, a subset of user interface options from the set of available user interface options associated with the physical surface, wherein the subset of user interface options selected is less than the set of available user interface options; and
generating an instruction for presenting the virtual user interface containing the subset of user interface options that can legibly fit in the virtual user interface in a 3D view to the user.
11. The method of claim 10 , wherein the object information comprises one or more of: a shape of a physical object, functions of the physical object, a type of the object, information associated with objects in the user's environment, or an affordance of the physical object.
12. The method of claim 11 , wherein the object comprises one or more persons in a field of view of the user.
13. The method of claim 10 , wherein the virtual user interface comprises a virtual menu.
14. The method of claim 10 , wherein the object is a table, a desk, or a wall.
15. The method of claim 10 , wherein the object function associated with the object is variable depending on the environment, and wherein the selecting of the subset of user interface options is based on the object function associated with the object in view of the environment.
16. The method of claim 10 , wherein the selecting of the subset of user interface options is further based on one or more of an age, gender, educational level, occupation, and preference of the user.
17. The method of claim 10 , wherein the selecting of the subset of user interface options is further based on a physiological state of the user, a psychological state of the user, or autonomic nerve system activity of the user.
18. The method of claim 10 , wherein the selecting of the subset of user interface options is further based on a fiducial marker associated with the object.
19. A method for generating virtual content in a three-dimensional (3D) physical environment of a user, the method comprising:
under control of a hardware processor:
analyzing data acquired from a pose sensor to identify a pose of a user; identifying a physical surface in a 3D physical environment of the user and at least an object function associated with an object in the 3D physical environment having the physical surface, based at least partly on the pose;
receiving an indication to initiate an interaction with the physical surface; determining environmental information associated with a fiducial marker associated with the physical surface, the environmental information comprising an affordance of the physical surface, wherein the affordance of the physical surface is determined based on a physical environment in which the physical surface is located;
accessing a set of available user interface options associated with the physical surface based on the information associated with the fiducial marker;
determining a maximum number of user interface options from the set of available user interface options that can legibly fit in a virtual user interface associated with the physical surface;
selecting, based on the maximum number of user interface options that can legibly fit in the virtual user interface and the information associated with the fiducial marker and based on the affordance of the physical surface and on the object function associated with the object, a subset of user interface options from the set of available user interface options associated with the physical surface, wherein the subset of user interface options selected is less than the set of available user interface options; and
generating an instruction for presenting the virtual user interface containing the subset of user interface options that can legibly fit in the virtual user interface in a 3D view to the user.
20. The method of claim 19 , wherein the virtual user interface comprises a virtual menu.