IP Library Granted Patent US 10,860,098
Granted Patent B1
US 10,860,098 · App. 16/730,997 · Granted Dec 8, 2020

Gesture-based eye tracking

Inventor: Josh Stratton (Issaquah, WA)
Assignee: HULU, LLC
G06F3/013G02B27/0093G06F3/017G06K9/00335G06K9/00355G02B2027/0187
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,860,098
App. No.
16/730,997
Granted
Dec 8, 2020
Kind
B1
Abstract

In some embodiments, a method receives images of eyes of a user. The method computes a vertical resting position of the eyes and computes a horizontal resting position of the eyes based on an average horizontal position of both of the eyes. An image of the eyes of the user is received. Then, the method calculates when an offset for a direction and a magnitude of the eyes is within a region in a plurality of regions and determines a command for the region. A call for the command is generated to a device to cause the command to be performed on a user interface.

Claims (54)

1. A method comprising:

receiving, by a computing device, images of eyes of a user;

computing, by the computing device, a vertical resting position of the eyes;

computing, by the computing device, a horizontal resting position of the eyes based on an average horizontal position of both of the eyes;

receiving, by the computing device, an image of the eyes of the user;

calculating, by the computing device, when an offset for a direction and a magnitude of the eyes is within a region in a plurality of regions;

determining, by the computing device, a command for the region; and

generating, by the computing device, a call for the command to a device to cause the command to be performed on a user interface.

2. The method of claim 1 , further comprising:

computing boxes around the eyes of the user, wherein the boxes are used to compute the vertical resting position and the horizontal resting position.

3. The method of claim 2 , wherein the vertical resting position is a vertical position in the boxes.

4. The method of claim 2 , wherein the horizontal resting position is the average horizontal position of the eyes in the boxes.

5. The method of claim 1 , wherein the direction and the magnitude is an offset from a resting position that is based on the horizontal resting position and the vertical resting position.

6. The method of claim 5 , wherein the offset in a horizontal direction is a difference between an average of the horizontal position of the eyes from the image and the horizontal resting position.

7. The method of claim 1 , wherein the offset in a vertical direction is a difference between an average of the vertical position of the eyes from the image and the vertical resting position.

8. The method of claim 1 , further comprising:

measuring a time in which the eyes are closed; and

mapping the time to another command.

9. The method of claim 8 , wherein mapping the time to another command comprises:

mapping the time to one of a plurality of commands, wherein each of the plurality of commands is associated with a different range of times in which the eyes are closed.

10. The method of claim 1 , further comprising:

determining whether both eyes are looking in a direction towards the region;

when both eyes are looking in the direction towards the region, generating the call for the command; and

when both eyes are not looking in the direction towards the region, not generating the call for the command.

11. The method of claim 1 , wherein the offset is determined based on an X position offset and a Y position offset of the eyes with the resting position as a center point.

12. The method of claim 1 , wherein calculating when the offset for the direction and the magnitude of the eyes is within the region comprises:

determining when the direction and the magnitude of the eyes is outside of a threshold and within the region.

13. The method of claim 1 , wherein the call comprises a call to an application programming interface of the device.

14. A non-transitory computer-readable storage medium containing instructions, that when executed, control a computer system to be operable for:

receiving images of eyes of a user;

computing a vertical resting position of the eyes;

computing a horizontal resting position of the eyes based on an average horizontal position of both of the eyes;

receiving an image of the eyes of the user;

calculating when an offset for a direction and a magnitude of the eyes is within a region in a plurality of regions;

determining a command for the region; and

generating a call for the command to a device to cause the command to be performed on a user interface.

15. The non-transitory computer-readable storage medium of claim 14 , further comprising:

computing boxes around the eyes of the user, wherein the boxes are used to compute the vertical resting position and the horizontal resting position.

16. The non-transitory computer-readable storage medium of claim 14 , wherein the direction and the magnitude is an offset from a resting position that is based on the horizontal resting position and the vertical resting position.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the offset in a horizontal direction is a difference between an average of the horizontal position of the eyes from the image and the horizontal resting position.

18. The non-transitory computer-readable storage medium of claim 14 , wherein the offset in a vertical direction is a difference between an average of the vertical position of the eyes from the image and the vertical resting position.

19. The non-transitory computer-readable storage medium of claim 14 , further comprising:

measuring a time in which the eyes are closed; and

mapping the time to another command.

20. An apparatus comprising:

one or more computer processors; and

a non-transitory computer-readable storage medium comprising instructions, that when executed, control the one or more computer processors to be operable for:

receiving images of eyes of a user;

computing a vertical resting position of the eyes;

computing a horizontal resting position of the eyes based on an average horizontal position of both of the eyes;

receiving an image of the eyes of the user;

calculating when an offset for a direction and a magnitude of the eyes is within a region in a plurality of regions;

determining a command for the region; and

generating a call for the command to a device to cause the command to be performed on a user interface.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 5, 2020
From: STRATTON, JOSH
To: HULU, LLC
Reel/Frame 051732/0211 →
Cited By (3)
US 12,462,845 US 12,609,845 US 12,656,862