Granted Patent B2

US 12,517,627 · App. 18/477,188 · Granted Jan 6, 2026

Inventors: Jochen Weber (San Francisco, CA); Tobias Toft (San Mateo, CA); Ryan Alexander West (Carleton Place, CA); Cosmo Rettig (Mill Valley, CA); Prasanthi Gurumurthy (San Jose, CA); Joost Korngold (Morgan Hill, CA); Tarik Hany Abdel-Gawad (San Francisco, CA); Samuel Legge (Waterloo, CA); Jingying Hu (Menlo Park, CA)

Assignee: GOOGLE LLC

G06F3/0482G02B27/017G06F3/012G06F3/013G02B2027/0178

Abstract

Claims (80)

1 . A non-transitory computer-readable storage medium comprising instructions stored thereon that, when executed by at least one processor, are configured to cause a computing system to:

display a user interface (UI) including a grid of blocks;

identify a direction based on an eye-gaze, the direction being from a first block to a second block of the grid of blocks;

identify a UI element displayed on the UI based on the direction;

determine an eye-gaze characteristic of a user of a wearable device;

determine a head movement of the user, the UI being configured to remain stationary on a display of the wearable device in relation to the head movement;

determine a gesture based on the eye-gaze characteristic, the head movement, and a correlation between the eye-gaze characteristic and the head movement;

select the UI element based on the gesture; and

trigger a UI operation in response to selecting the UI element.

2 . The non-transitory computer-readable storage medium of claim 1 , wherein the eye-gaze characteristic includes at least one of an eye-gaze saccade and a fixation point associated with a vector.

3 . The non-transitory computer-readable storage medium of claim 1 , wherein

the direction is a pointing vector, and

the head movement includes a head motion vector.

4 . The non-transitory computer-readable storage medium of claim 1 , wherein

the UI element is associated with two or more blocks of the UI, and

the UI operation causes a UI element characteristic to change.

5 . The non-transitory computer-readable storage medium of claim 1 , wherein

the correlation between the eye-gaze characteristic and the head movement indicates that the user is looking outside a boundary of the UI, and

the UI operation causes the UI to be hidden.

6 . The non-transitory computer-readable storage medium of claim 1 , the instructions further cause the computing system to:

identify one of a false eye-gaze detection, head motion detection, or missed UI interaction, and

in response to identifying one of a false eye-gaze detection, head motion detection, or missed UI interaction, causing the UI operation not to be performed.

7 . The non-transitory computer-readable storage medium of claim 6 , the instructions further cause the computing system to:

determine a second eye-gaze characteristic;

determine a second head movement;

select a second UI element based on a correlation between the second eye-gaze characteristic and the second head movement as an indicator correlation; and

trigger a user interface (UI) operation in response to selecting the second UI element.

8 . A method comprising:

displaying a user interface (UI) including a grid of blocks;

identifying a direction based on an eye-gaze, the direction being from a first block to a second block of the grid of blocks;

identifying a UI element displayed on the UI based on the direction;

determining an eye-gaze characteristic of a user of a wearable device;

determining a head movement of the user, the UI being configured to remain stationary on a display of the wearable device in relation to the head movement;

determining whether the user of the wearable device intends to perform a gesture based on at least one of the eye-gaze characteristic and the head movement; and

in response to determining the user of the wearable device intends to perform a gesture;

determining a gesture based on the eye-gaze characteristic, the head movement, and a correlation between the eye-gaze characteristic and the head movement;

selecting the UI element based on the gesture; and

triggering a UI operation in response to selecting the UI element.

9 . The method of claim 8 , wherein the eye-gaze characteristic includes at least one of an eye-gaze saccade and a fixation point associated with a vector.

10 . The method of claim 8 , wherein

the direction is a pointing vector, and

the head movement includes a head motion vector.

11 . The method of claim 8 , wherein

the UI element is associated with two or more blocks of the UI, and

the UI operation causes a UI element characteristic to change.

12 . The method of claim 8 , wherein

the correlation between the eye-gaze characteristic and the head movement indicates that the user is looking outside a boundary of the UI, and

the UI operation causes the UI to be hidden.

13 . The method of claim 8 , further comprising:

identifying one of a false eye-gaze detection, head motion detection, or missed UI interaction, and

in response to identifying one of a false eye-gaze detection, head motion detection, or missed UI interaction, causing the UI operation not to be performed.

14 . The method of claim 13 , further comprising:

determining a second eye-gaze characteristic;

determining a second head movement;

selecting a second UI element based on a correlation between the second eye-gaze characteristic and the second head movement as an indicator correlation; and

triggering a user interface (UI) operation in response to selecting the second UI element.

15 . An apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to:

display a user interface (UI) including a grid of blocks;

identify a direction based on an eye-gaze, the direction being from a first block to a second block of the grid of blocks;

identify a UI element displayed on the UI based on the direction;

determine an eye-gaze characteristic of a user of a wearable device;

determine a head movement of the user, the UI being configured to remain stationary on a display of the wearable device in relation to the head movement;

determine a gesture based on the eye-gaze characteristic, the head movement, and a correlation between the eye-gaze characteristic and the head movement;

select the UI element based on the gesture; and

trigger a UI operation in response to selecting the UI element.

16 . The apparatus of claim 15 , wherein the eye-gaze characteristic includes at least one of an eye-gaze saccade and a fixation point.

17 . The apparatus of claim 15 , wherein

the direction is a pointing vector, and

the head movement includes a head motion vector.

18 . The apparatus of claim 15 , wherein

the UI element is associated with two or more blocks of the UI, and

the UI operation causes a UI element characteristic to change.

19 . The apparatus of claim 15 , the computer program code further causes the apparatus to:

identify one of a false eye-gaze detection, head motion detection, or missed UI interaction, and

in response to identifying one of a false eye-gaze detection, head motion detection, or missed UI interaction, causing the UI operation not to be performed.

20 . The apparatus of claim 19 , the computer program code further causes the apparatus to:

determine a second eye-gaze characteristic;

determine a second head movement;

select a second UI element based on a correlation between the second eye-gaze characteristic and the second head movement as an indicator correlation; and

trigger a user interface (UI) operation in response to selecting the second UI element.

Assignments (1)

Continuity (1)

References Cited (13)