Techniques for identifying graphical elements displayed at a head-worn device based on an eye gesture and a motional gesture
A method for managing a user interface in a smart glass is provided. The method includes identifying an eye gesture of the user of the smart glass, based on an eye-tracking signal indicative of a pupil location of the user, assessing a user intention for an application displayed in a user interface in the headset based on the eye gesture, and activating the application based on the user intention. A smart glass for use with the above method is also provided.
1 . A method, comprising:
providing information to cause a graphical user interface, of a head-worn device worn by a user, to be displayed on a display of the head-worn device;
detecting an eye gesture performed by the user;
identifying, in the graphical user interface displayed on the display of the head-worn device, a graphical element based at least in part upon the detected eye gesture;
receiving, from a wearable device worn by the user, an indication of a motional gesture performed by the user outside of the graphical user interface, wherein the motional gesture, when performed, is associated with the graphical element and reproduced in the graphical user interface; and
performing an action associated with the graphical element based at least in part upon the indication of the motional gesture performed by the user.
2 . The method of claim 1 , wherein the wearable device is a wristband worn on the wrist of the user.
3 . The method of claim 1 , wherein the wearable device is worn on a finger or a hand of the user.
4 . The method of claim 1 , wherein the motional gesture is performed by a finger, a hand, or a wrist of the user.
5 . The method of claim 1 , further comprising:
causing a change in a visual appearance of the graphical element based at least in part upon the detected eye gesture.
6 . The method of claim 1 , wherein the eye gesture comprises setting a gaze of the user towards the graphical element.
7 . The method of claim 1 , further comprising:
displaying, on the display of the head-worn device, a virtual assistant indicating that an attribute of the graphical element is relevant to the user; and
activating the virtual assistant when the eye gesture includes a gaze direction towards the virtual assistant.
8 . The method of claim 1 , further comprising highlighting, based on a direction of the eye gesture, at least a portion of an application displayed on the display of the head-worn device.
9 . The method of claim 1 , wherein the action is one of scrolling, pinching, or flipping the graphic element.
10 . A system, comprising:
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, cause the system to:
provide information to cause a graphical user interface, of a head-worn device worn by a user, to be displayed on a display of the head-worn device;
detect an eye gesture performed by the user;
identify, in the graphical user interface displayed on the display of the head-worn device, a graphical element based at least in part upon the detected eye gesture;
receive, from a wearable device worn by the user, an indication of a motional gesture performed by the user outside of the graphical user interface, wherein the motional gesture, when performed, is associated with the graphical element and reproduced in the graphical user interface; and
perform an action associated with the graphical element based at least in part upon the indication of the motional gesture performed by the user.
11 . The system of claim 10 , wherein the wearable device is a wristband worn on the wrist of the user.
12 . The system of claim 10 , wherein the wearable device is worn on a finger or a hand of the user.
13 . The system of claim 10 , wherein the eye gesture comprises setting a gaze of the user towards the graphical element.
14 . The system of claim 10 , wherein the instructions, when executed by the one or more processors, cause the system to: cause a change in a visual appearance of the graphical element based at least in part upon the detected eye gesture.
15 . The system of claim 10 , wherein the action is one of scrolling, pinching, or flipping the graphic element.
16 . A non-transitory computer-readable storage medium having instructions embodied thereon, wherein the instructions, when executed by one or more processors, cause a system to:
provide information to cause a graphical user interface, of a head-worn device worn by a user, to be displayed on a display of the head-worn device;
detect an eye gesture performed by the user;
identify, in the graphical user interface displayed on the display of the head-worn device, a graphical element based at least in part upon the detected eye gesture;
receive, from a wearable device worn by the user, an indication of a motional gesture performed by the user outside of the graphical user interface, wherein the motional gesture, when performed, is associated with the graphical element and reproduced in the graphical user interface; and
perform an action associated with the graphical element based at least in part upon the indication of the motional gesture performed by the user.
17 . The non-transitory computer-readable storage medium of claim 16 , wherein the wearable device is a wristband worn on the wrist of the user.
18 . The non-transitory computer-readable storage medium of claim 16 , wherein the motional gesture is performed by a finger, a hand, or a wrist of the user.
19 . The non-transitory computer-readable storage medium of claim 16 , wherein the instructions, when executed by one or more processors, cause a system to: cause a change in a visual appearance of the graphical element based at least in part upon the detected eye gesture.
20 . The non-transitory computer-readable storage medium of claim 16 , wherein the action is one of scrolling, pinching, or flipping the graphic element.