IP Library Granted Patent US 9,804,753
Granted Patent B2
US 9,804,753 · App. 14/221,147 · Granted Oct 31, 2017

Selection using eye gaze evaluation over time

Inventors: Scott Ramsby (Kirkland, WA); Tony Ambrus (Seattle, WA); Michael Scavezze (Bellevue, WA); Abby Lin Lee (Seattle, WA); Brian Mount (Seattle, WA); Ian Douglas McIntyre (Redmond, WA); Aaron Mackay Burns (Newcastle, WA); Russ McMackin (Kirkland, WA); Katelyn Elizabeth Doran (Seattle, WA); Gerhard Schneider (Seattle, WA); Quentin Simon Charles Miller (Sammamish, WA)
Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
G06F3/04842G06F3/013G06F3/048G06F3/0482G06F3/04883G06K9/00604
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,804,753
App. No.
14/221,147
Granted
Oct 31, 2017
Kind
B2
Abstract

Various embodiments relating to selection of a user interface object displayed on a graphical user interface based on eye gaze are disclosed. In one embodiment, a selection input may be received. A plurality of eye gaze samples at different times within a time window may be evaluated. The time window may be selected based on a time at which the selection input is detected. A user interface object may be selected based on the plurality of eye gaze samples.

Claims (60)

1. On a computing device comprising an eye tracking system, a method for selecting a user interface object displayed on a graphical user interface based on eye gaze, the method comprising:

sampling eye gaze samples via the eye tracking system;

storing the eye gaze samples;

detecting a selection input, via a user input mechanism other than the eye tracking system, at a time when an eye gaze location intersects a first user interface object;

upon receipt of the selection input, evaluating a plurality of the eye gaze samples at different times within a time window selected based on the time at which the selection input is received to determine a combined location of the plurality of the eye gaze samples;

when the combined location of the plurality of the eye gaze samples is closer to a second user interface object than the first user interface object, then selecting the second user interface object as a result of the selection input;

evaluating a different plurality of the eye gaze samples in the time window to determine a combined location of the different plurality of the eye gaze samples; and

confirming the selecting of the second user interface object or selecting a different user interface object as a result of the selection input based upon the combined location of the different plurality of the eye gaze samples.

2. The method of claim 1 , wherein the second user interface object is selected based on having a highest weighting according to a selection rule.

3. The method of claim 1 , wherein the time window includes eye gaze samples taken before and after the time at which the selection input is received, and wherein the method further comprises selecting the first user interface object when the combined location of the plurality of the eye gaze samples is closer to the first user interface object than the second user interface object.

4. The method of claim 1 , further comprising:

setting a center of the time window to a different time that is prior to the time at which the selection input is received.

5. The method of claim 1 , further comprising:

evaluating the plurality of the eye gaze samples compared to a stability metric, and not including any eye gaze samples that do not meet the stability metric in evaluating the plurality of the eye gaze samples.

6. The method of claim 1 , further comprising:

in response to selecting the second user interface object, providing visual feedback of the selecting of the second user interface object on the graphical user interface.

7. The method of claim 1 , wherein the time window is a first time window, wherein selecting the different user interface object comprises

selecting the different user interface object if the different user interface object is nearest to the combined location of the different plurality of the eye gaze samples, and undoing the selecting of the second user interface object.

8. The method of claim 7 , further comprising:

upon evaluation of the different plurality of eye gaze samples in the second time window, removing evaluated eye gaze samples in the first time window and the second time window from being eligible for future evaluations of eye gaze samples.

9. The method of claim 7 , wherein the selection input is a first selection input, the method further comprising:

receiving a second selection input after the first selection input is received; and

if the second selection input is detected before the different plurality of the eye gaze samples are evaluated, then shortening the second time window.

10. A data-holding device, comprising an eye tracking system, holding instructions executable by a logic device to:

sample eye gaze samples via the eye tracking system;

store the eye gaze samples;

detect a selection input made via a user input mechanism other than the eye tracking system, at a time when an eye gaze location intersects a first user interface object;

upon receipt of the selection input, evaluate a plurality of the eye gaze samples at different eye gaze locations and at different times within a time window selected based on the time at which the selection input is detected to determine a combined location of the plurality of the eye gaze samples;

when the combined location of the plurality of the eye gaze samples is closer to a second user interface object than the first user interface object, then select the second user interface object as a result of the selection input and based on the second user interface object having a highest weighting according to a selection rule;

evaluate a different plurality of the eye gaze samples in the time window to determine a combined location of the different plurality of the eye gaze samples; and

confirm the selecting of the second user interface object or select a different user interface object as a result of the selection input based upon the combined location of the different plurality of gaze samples.

11. The data-holding device of claim 10 , wherein the highest weighting is assigned to a user interface object that is a nearest user interface object to the combined location at which the plurality of the eye gaze samples intersect with a graphical user interface, the nearest user interface object being the first user interface object when the combined location is closer to the first user interface object than the second user interface object.

12. The data-holding device of claim 10 , wherein the time window includes eye gaze samples taken before and after the time at which the selection input is detected.

13. The data-holding device of claim 10 , further holding instructions executable by the logic device to:

set a center of the time window to a different time prior to the time at which the selection input is detected.

14. The data-holding device of claim 10 , further holding instructions executable by the logic device to:

evaluate the plurality of the eye gaze samples compared to a stability metric, and not include any eye gaze samples that do not meet the stability metric in evaluating the plurality of the eye gaze samples.

15. The data-holding device of claim 10 , further holding instructions executable by the logic device to:

in response to selecting the second user interface object, provide visual feedback of selection of the second user interface object on the graphical user interface.

16. The data-holding device of claim 10 , wherein the time window is a first time window, and the data-holding device further holds instructions executable by the logic device to:

after the second user interface object is selected, evaluate a different plurality of the eye gaze samples in a second time window that is centered later than the first time window to determine the combined location of the different plurality of the eye gaze samples; and

if the different user interface object is nearest to the combined location of the different plurality of the eye gaze samples, then undo the selecting of the second user interface object and select the different user interface object.

17. The data-holding device of claim 16 , further holding instructions executable by the logic device to:

upon evaluation of the different plurality of the eye gaze samples, remove evaluated eye gaze samples in the first time window and the second time window from being eligible for future evaluations of eye gaze samples.

18. The data-holding device of claim 16 , wherein the selection input is a first selection input, and the data-holding device further holds instructions executable by the logic device to:

detect a second selection input after the time at which the first selection input is detected;

if the second selection input is detected before the different plurality of the eye gaze samples are evaluated, then shorten the second time window.

19. A head-mounted display device, comprising:

a display configured to display a graphical user interface including one or more user interface objects;

an eye tracking system configured to detect a location at which an eye gaze of a wearer of the head-mounted display device intersects with the display;

a logic device; and

a data-holding device holding instructions executable by the logic device to:

sample eye gaze samples via the eye tracking system;

store the eye gaze samples;

detect a selection input made via a user input mechanism other than the eye tracking system, at a time when an eye gaze location intersects a first user interface object,

upon receipt of the selection input, evaluate a plurality of the eye gaze samples at different eye gaze locations and at different times within a time window selected based on the time at which the selection input is detected to determine a combined location of the plurality of the eye gaze samples, and

when the combined location of the plurality of the eye gaze samples is closer to a second user interface object than the first user interface object, then select the second user interface object as a result of the selection input;

evaluate a different plurality of the eye gaze samples in the designated time window to determine a combined location of the different plurality of the eye gaze samples; and

confirm the selecting of the second user interface object or select a different user interface object as a result of the selection input based upon the combined location of the different plurality of eye gaze samples.

20. The head-mounted display device of claim 19 , wherein the time window includes eye gaze samples taken before and after the time at which the selection input is detected, and wherein the first user interface object is selected when the combined location of the plurality of the gaze samples is closer to the first user interface object than the second user interface object.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 20, 2016
From: RAMSBY, SCOTT; AMBRUS, TONY; SCAVEZZE, MICHAEL; LEE, ABBY LIN; MOUNT, BRIAN; MCINTYRE, IAN DOUGLAS; BURNS, AARON MACKAY; MCMACKIN, RUSS; DORAN, KATELYN ELIZABETH; SCHNEIDER, GERHARD; MILLER, QUENTIN SIMON CHARLES
To: MICROSOFT CORPORATION
Reel/Frame 039085/0812 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 9, 2015
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 039025/0454 →
Continuity (1)
Related Publication 20150268821A1 · Sep 24, 2015