TECHNIQUES FOR IDENTIFYING GRAPHICAL ELEMENTS DISPLAYED AT A HEAD-WORN DEVICE BASED ON AN EYE GESTURE AND A MOTIONAL GESTURE
A method for managing a user interface in a smart glass is provided. The method includes identifying an eye gesture of the user of the smart glass, based on an eye-tracking signal indicative of a pupil location of the user, assessing a user intention for an application displayed in a user interface in the headset based on the eye gesture, and activating the application based on the user intention. A smart glass for use with the above method is also provided.
1 . (canceled)
2 . A method comprising:
while a user is wearing a wrist-worn device and a head-worn device comprising (1) a first eyepiece with a display and (2) one or more cameras:
receiving, from the wrist-worn device, an indication of a motional gesture performed by the user;
obtaining image data from the one or more cameras;
identifying an object from the image data;
determining, based on the indication of the motional gesture and the object, a user intention to interact with an interface displayed on the head-worn device, wherein the interface is associated with an application;
activating, based on the user intention, a function of the application; and
displaying, on the interface, information associated with the object on the display of the first eyepiece.
3 . The method of claim 2 , wherein the motional gesture comprises one or more of a scrolling, a pinching, a panning, or a flipping gesture from the user.
4 . The method of claim 2 , wherein the user intention is further determined based on a gaze of the user.
5 . The method of claim 2 , wherein the user intention is further determined based on a voice of the user captured by one or more microphones of the head-worn device.
6 . The method of claim 2 , wherein displaying the information on the interface comprises highlighting an icon associated with the function of the application.
7 . The method of claim 2 , wherein the application is a virtual assistant application.
8 . The method of claim 7 , wherein the virtual assistant application provides a recommendation associated with the object to the user.
9 . The method of claim 2 , wherein the head-worn device further comprises a second eyepiece that does not include a display.
10 . A non-transitory, computer-readable storage medium including executable instruction that, when executed by one or more processors, cause the one or more processors to:
while a user is wearing a wrist-worn device and a head-worn device comprising (1) a first eyepiece with a display and (2) one or more cameras:
receive, from the wrist-worn device, an indication of a motional gesture performed by the user;
obtain image data from the one or more cameras;
identify an object from the image data;
determine, based on the indication of the motional gesture and the object, a user intention to interact with an interface displayed on the head-worn device, wherein the interface is associated with an application;
cause a function of the application to be activated based on the user intention; and
cause information associated with the object to be displayed on the interface at the display of the first eyepiece.
11 . The non-transitory, computer-readable storage medium of claim 10 , wherein the motional gesture comprises one or more of a scrolling, a pinching, a panning, or a flipping gesture from the user.
12 . The non-transitory, computer-readable storage medium of claim 10 , wherein the user intention is further determined based on a gaze of the user.
13 . The non-transitory, computer-readable storage medium of claim 10 , wherein the user intention is further determined based on a voice of the user captured by one or more microphones of the head-worn device.
14 . The non-transitory, computer-readable storage medium of claim 10 , wherein displaying the information on the interface comprises highlighting an icon associated with the function of the application.
15 . The non-transitory, computer-readable storage medium of claim 10 , wherein the head-worn device further comprises a second eyepiece that does not include a display.
16 . A system comprising:
wrist-worn device; and
a head-worn device comprising (1) a first eyepiece with a display and (2) one or more cameras, wherein the system is configured to:
while a user is wearing the wrist-worn device and the head-worn device:
receive, from the wrist-worn device, an indication of a motional gesture performed by the user;
obtain image data from the one or more cameras;
identify an object from the image data;
determine, based on the indication of the motional gesture and the object, a user intention to interact with an interface displayed on the head-worn device, wherein the interface is associated with an application;
activate, based on the user intention, a function of the application; and
display, on the interface, information associated with the object on the display of the first eyepiece.
17 . The system of claim 16 , wherein the motional gesture comprises one or more of a scrolling, a pinching, a panning, or a flipping gesture from the user.
18 . The system of claim 16 , wherein the user intention is further determined based on a gaze of the user.
19 . The system of claim 16 , wherein the user intention is further determined based on a voice of the user captured by one or more microphones of the head-worn device.
20 . The system of claim 16 , wherein displaying the information on the interface comprises highlighting an icon associated with the function of the application.
21 . The system of claim 16 , wherein the head-worn device further comprises a second eyepiece that does not include a display.