IP Library › Granted Patent US 10,627,900
Granted Patent B2
US 10,627,900 · App. 15/897,536 · Granted Apr 21, 2020

Eye-signal augmented control

Inventors: Nelson G. Publicover (Reno, NV); Lewis James Marggraff (Mountain View, CA); Spencer James Connaughton (Palo Alto, CA)
Assignee: GOOGLE LLC
G06F3/013G06F3/04812G06K9/00597G06K9/00604
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,627,900
App. No.
15/897,536
Filed
Feb 15, 2018
Granted
Apr 21, 2020
Kind
B2
Art Unit
2694
USPC
345/157
Abstract

An electronic system tracks a user's gaze to rapidly transport a cursor to a location within a focal region of the user's eye. The electronic system transports the cursor from an initial location across one or more displays to a new location in response to detecting one or more saccadic and/or vergence movements of the user's eye or in response to a signal indicating that the user desires to move the cursor to a new location within a focal region of the user's eye or eyes. In some embodiments, the electronic system moves the cursor to the new location along a trajectory wherein the cursor is visible along at least a portion of the trajectory, enabling the user to find the cursor more easily.

Claims (36)

1. A method comprising:

tracking, by a processing unit, a gaze location of a first user's eyes based on images captured by a user-facing camera;

filtering, by the processing unit, movements of the first user's eyes based on one or more of angular velocity and divergent movement of one eye relative to the other eye to identify saccadic or vergence eye movements;

computing, by the processing unit, the gaze location relative to a location of a cursor at a first display; and

selectively moving, by the processing unit, the cursor from a first location to a second location along a trajectory in response to detecting that the first user's gaze is focused at a focal region comprising the second location following a saccadic or vergence eye movement.

2. The method of claim 1 , wherein the cursor is visible as it moves along at least a portion of the trajectory.

3. The method of claim 1 , wherein selectively moving comprises moving the cursor essentially instantaneously from the first location to the second location.

4. The method of claim 1 , further comprising moving the cursor along the trajectory at a first rate for a first duration, and decelerating the cursor relative to the first rate as the cursor approaches the second location.

5. The method of claim 1 , wherein the first location is at a first display and the second location is at a second display and selectively moving the cursor comprises transferring the cursor from the first display to the second display along the trajectory.

6. The method of claim 5 , wherein the first display and the second display are controlled by different processing units.

7. The method of claim 1 , wherein moving the cursor is further in response to detecting a signal generated at one of a mouse, keyboard, joystick, touchpad, or microphone, the signal indicating that the cursor should be moved to the second location.

8. The method of claim 1 , further comprising refraining from moving the cursor to the second location in response to determining that the cursor is within a threshold distance of the second location.

9. The method of claim 1 , further comprising selectively moving, by the processing unit, one or more of a function, icon, text, or graphic associated with the cursor from the first location to the second location in response to detecting that the first user's gaze is focused at a focal region comprising the second location following a purposeful saccadic movement.

10. A method, comprising:

tracking, by a processing unit, a gaze location of a first user's eyes based on images captured by a user-facing camera;

filtering, by the processing unit, movements of the first user's eyes based on one or more of angular velocity and divergent movement of one eye relative to the other eye to identify saccadic or vergence eye movements;

computing, by the processing unit, the gaze location relative to a location of a cursor at a first display;

receiving, at the processing unit, a signal indicating that the first user desires to move the cursor from a first location at the first display; and

moving the cursor from the first location to a second location along a trajectory in response to detecting that the first user's gaze is focused at a focal region comprising the second location following a saccadic or vergence eye movement.

11. The method of claim 10 , wherein cursor is visible as it moves along at least a portion of the trajectory.

12. The method of claim 10 , further comprising moving the cursor along the trajectory at a first rate for a first duration, and decelerating the cursor relative to the first rate as the cursor approaches the second location.

13. The method of claim 10 , wherein the first location is at a first display and the second location is at a second display and moving the cursor comprises transferring the cursor from the first display to the second display along the trajectory.

14. The method of claim 13 , wherein the first display and the second display are controlled by different processing units.

15. The method of claim 10 , wherein moving the cursor is further in response to detecting a saccadic movement of the first user's eyes toward the second location.

16. The method of claim 10 , wherein the signal is generated at one of a mouse, keyboard, joystick, touchpad, switch, or microphone.

17. The method of claim 10 , further comprising selectively moving, by the processing unit, one or more of a function, icon, text, or graphic associated with the cursor from the first location to the second location in response to detecting that the first user's gaze is focused at a focal region comprising the second location.

18. An electronic device, comprising:

a user-facing camera configured to track a gaze of a first user's eyes;

a first display;

a cursor displayed at a first location at the first display; and

a processor configured to:

filter movements of the first user's eyes based on one or more of angular velocity and divergent movement of one eye relative to the other eye to identify saccadic or vergence eye movements; and

selectively move the cursor from the first location to a second location along a trajectory in response to receiving a signal indicating that the first user desires to move the cursor from the first location and further in response to detecting that the first user's gaze is focused at a focal region comprising the second location following a saccadic or vergence eye movement.

19. The electronic device of claim 18 , wherein cursor is visible as it approaches the second location.

20. The electronic device of claim 18 , wherein the second location is at a second display and the processor is configured to selectively move the cursor by transferring the cursor from the first display to the second display along the trajectory.

21. The electronic device of claim 20 , wherein the first display and the second display are controlled by different processing units.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 19, 2018
From: PUBLICOVER, NELSON G.; MARGGRAFF, LEWIS JAMES; CONNAUGHTON, SPENCER JAMES
To: GOOGLE LLC
Reel/Frame 044964/0147 →
Continuity (2)
Provisional Application 62475709 · Mar 23, 2017
Related Publication 20180275753A1 · Sep 27, 2018