CONTEXTUAL AWARENESS OF USER INTERFACE MENUS
Examples of systems and methods for a wearable system to automatically select or filter available user interface interactions or virtual objects are disclosed. The wearable system can select a group of virtual objects for user interaction based on contextual information associated with the user, the user's environment, physical or virtual objects in the user's environment, or the user's physiological or psychological state.
1 . A method for selectively presenting virtual content to a user in a three-dimensional space (3D), the method comprising:
under control of a wearable device comprising a computer processor, a display, and a physiological sensor configured to measure a physiological parameter of a user:
obtaining, using the physiological sensor, data associated with the physiological parameter of the user;
determining, based at least partly on the data, a physiological state of the user; determining virtual content to be presented to the user based at least partly on the physiological state;
determining a spatial location for displaying the virtual content in a 3D space;
generating a virtual user interface comprising at least the determined virtual content; and
displaying to the user, via the display of the wearable device, virtual content at the determined spatial location.
2 . The method of claim 1 , wherein the physiological parameter comprises at least one of: a heart rate, a pupil dilation, a galvanic skin response, a blood pressure, an encephalographic state, a respiration rate, or an eye movement.
3 . The method of claim 1 , further comprising obtaining data associated with the physiological parameter of the user using an inward-facing imaging system configured to image one or both eyes of the user.
4 . The method of claim 1 , further comprising determining a psychological state based on the data.
5 . The method of claim 1 , wherein the virtual content comprises a virtual menu.
6 . The method of claim 1 , wherein the virtual content is further determined based on at least one of the following: an affordance of a physical object associated with the virtual content; a function of an environment of the user; a characteristic of the user; individuals present in the environment; information encoded in a fiducial marker associated with the physical object; or a current or past interaction of the user with the wearable system.
7 . The method of claim 6 , wherein the affordance of the physical object comprises a relation between the physical object and the environment of the physical object which affords an opportunity for an action or use associated with the physical object.
8 . The method of claim 6 , wherein the affordance of the physical object is based at least partly on one or more of the following: a function of the physical object, an orientation, a type, a location, a shape, a size, or the environment in which the physical object is located.
9 . The method of claim 8 , wherein the orientation of the physical object comprises horizontal or vertical.
10 . The method of claim 6 , wherein the environment is a living environment or a working environment.
11 . The method of claim 6 , wherein the environment is a private environment or a public environment.
12 . The method of claim 6 , wherein the characteristic of the user comprises one or more of the following: an age, a gender, an educational level, an occupation, or a preference.
13 . The method of claim 12 , wherein the preference is based at least partly on a previous usage pattern of the user, wherein the previous usage pattern comprises information on a location or a time for which the virtual object is used.
14 . The method of claim 6 , wherein the current interaction comprises a telepresence session between the user of the AR system and another user.
15 . The method of claim 1 , wherein the wearable device comprises an augmented reality system.
16 . A wearable device for generating virtual content in a three-dimensional (3D) environment of a user, the wearable system comprising:
an augmented reality display configured to present virtual content in a 3D view to a user;
a physiological sensor configured to measure a physiological parameter of the user;
a hardware processor in communication with the physiological sensor and the augmented reality display, the hardware processor programmed to:
obtain, using the physiological sensor, data associated with the physiological parameter of the user;
determine, based at least partly on the data, a physiological state of the user; determining virtual content to be presented to the user based at least partly on the physiological state;
determine a spatial location for displaying the virtual content in a 3D space;
generate a virtual user interface comprising at least the determined virtual content; and
display to the user, via the display of the wearable device, virtual content at the determined spatial location.
17 . The wearable device of claim 16 , wherein the physiological parameter comprises at least one of: a heart rate, a pupil dilation, a galvanic skin response, a blood pressure, an encephalographic state, a respiration rate, or an eye movement.
18 . The wearable device of claim 16 , further comprising obtaining data associated with the physiological parameter of the user using an inward-facing imaging system configured to image one or both eyes of the user.
19 . The wearable device of claim 16 , wherein the virtual content is further determined based on at least one of the following: an affordance of a physical object associated with the virtual content; a function of an environment of the user; a characteristic of the user; individuals present in the environment; information encoded in a fiducial marker associated with the physical object; or a current or past interaction of the user with the wearable system.
20 . The wearable device of claim 19 , wherein the affordance of the physical object comprises a relation between the physical object and the environment of the physical object which affords an opportunity for an action or use associated with the physical object, and wherein the affordance of the physical object is based at least partly on one or more of the following: a function of the physical object, an orientation, a type, a location, a shape, a size, or the environment in which the physical object is located.