IP Library Granted Patent US 8,723,798
Granted Patent B2
US 8,723,798 · App. 13/278,606 · Granted May 13, 2014

Systems and methods for obtaining user command from gaze direction

Inventor: Matthew T. Vernacchia (Upper St. Clair, PA)
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,723,798
App. No.
13/278,606
Granted
May 13, 2014
Kind
B2
Abstract

In one embodiment of the present invention, a computer-implemented method for obtaining a command of a user includes capturing an image of the eyes of a user. The coordinates of a glint center point and the coordinates of an eye center point are calculated from the image. The gaze direction of the user is calculated from the coordinates of a glint center point and the coordinates of an eye center point to determine a corresponding command.

Claims (59)

1. A system for obtaining a command of a user comprising:

a camera configured to capture an image of the user;

an image capture module configured to retrieve the image from the camera;

an image analysis module configured to i) determine whether brightness values of pixels in the image are within a color threshold, and average coordinates of pixels within the color threshold to determine coordinates of a glint center point from the image, and ii) determine coordinates of an eye center point from the image;

a gaze direction determiner module configured to determine a gaze direction from the coordinates of the glint center point and the coordinates of the eye center point;

a typing interface module configured to determine a command corresponding to the gaze direction; and

a display configured to visually present the command to the user.

2. The system as claimed in claim 1 , wherein the camera is an infrared camera.

3. The system as claimed in claim 1 , further comprising a speaker configured to audibly present the command to the user.

4. The system as claimed in claim 1 , further comprising a tutorial module configured to present a tutorial to the user.

5. The system as claimed in claim 1 , further comprising a text-to-speech module configured to convert text of the command to speech.

6. A computer-implemented method for obtaining a command of a user comprising:

capturing an image of a user;

determining coordinates of a glint center point from the image;

determining coordinates of an eye center point from the image, wherein the step of determining coordinates of the eye center point includes:

a) identifying a first set of edge points having a shape from the image;

b) identifying a second set of edge points from the shape of the first set of edge points; and

c) averaging coordinates of the second set of edge points;

determining a gaze direction from the coordinates of the glint center point and the coordinates of the eye center point; and

determining a command corresponding to the gaze direction.

7. The method as claimed in claim 6 , wherein the second set of edge points represents a pupil.

8. The method as claimed in claim 6 , wherein the second set of edge points represents an iris.

9. The method as claimed in claim 6 , wherein the step of determining coordinates of the eye center point further comprises:

d) identifying a region of pixels in proximity to the glint center point;

e) determining whether pixels in the region is within a color threshold;

f) averaging coordinates of the pixels within the color threshold; and

g) calculating a weighted average of the coordinates obtained from steps a)-c) and from the coordinates obtained from steps d)-f), wherein the weighted average of the coordinates is weighted based on a confidence of the second set of edge points conforming to a circular pattern.

10. The method as claimed in claim 9 , wherein the second set of edge points represents a pupil.

11. The method as claimed in claim 9 , wherein the second set of edge points represents an iris.

12. A computer-implemented method for obtaining a command of a user comprising:

capturing an image of a user;

determining coordinates of a glint center point from the image, wherein the step of determining coordinates of the glint center point comprises:

determining whether brightness values of pixels in the image is within a color threshold; and

averaging coordinates of pixels within the color threshold;

determining coordinates of an eye center point from the image;

determining a gaze direction from the coordinates of the glint center point and the coordinates of the eye center point; and

determining a command corresponding to the gaze direction.

13. The method as claimed in claim 12 , wherein the step of determining coordinates of the glint center point further comprises removing regions of pixels in the image wherein the regions are not within a size threshold.

14. The method as claimed in claim 12 , wherein the step of determining coordinates of the glint center point further comprises removing regions of pixels without counterparts from the image.

15. A computer-implemented method for obtaining a command of a user comprising:

capturing an image of a user;

determining coordinates of a glint center point from the image;

determining coordinates of an eye center point from the image, wherein the step of determining coordinates of the eye center point comprises:

identifying a region of pixels in proximity to the glint center point;

determining whether pixels in the region is within a color threshold; and

averaging coordinates of the pixels within the color threshold;

determining a gaze direction from the coordinates of the glint center point and the coordinates of the eye center point; and

determining a command corresponding to the gaze direction.

16. A system for obtaining a command of a user comprising:

a camera configured to capture an image of the user;

an image capture module configured to retrieve the image from the camera;

an image analysis module configured to determine coordinates of a glint center point from the image, and further configured to a) identify a region of pixels in proximity to the glint center point; b) determine whether pixels in the region is within a color threshold; and c) average coordinates of the pixels within the color threshold to determine coordinates of an eye center point from the image;

a gaze direction determiner module configured to determine a gaze direction from the coordinates of the glint center point and the coordinates of the eye center point;

a typing interface module configured to determine a command corresponding to the gaze direction; and

a display configured to visually present the command to the user.

17. The system as claimed in claim 16 , wherein the camera is an infrared camera.

18. The system as claimed in claim 16 , further comprising a speaker configured to audibly present the command to the user.

19. The system as claimed in claim 16 , further comprising a tutorial module configured to present a tutorial to the user.

20. The system as claimed in claim 16 , further comprising a text-to-speech module configured to convert text of the command to speech.

Continuity (1)
Related Publication 20130100025A1 · Apr 25, 2013