Systems and methods for obtaining user command from gaze direction
View Patent ↗In one embodiment of the present invention, a computer-implemented method for obtaining a command of a user includes capturing an image of the eyes of a user. The coordinates of a glint center point and the coordinates of an eye center point are calculated from the image. The gaze direction of the user is calculated from the coordinates of a glint center point and the coordinates of an eye center point to determine a corresponding command.
1. A system for obtaining a command of a user comprising:
a camera configured to capture an image of the user;
an image capture module configured to retrieve the image from the camera;
an image analysis module configured to i) determine whether brightness values of pixels in the image are within a color threshold, and average coordinates of pixels within the color threshold to determine coordinates of a glint center point from the image, and ii) determine coordinates of an eye center point from the image;
a gaze direction determiner module configured to determine a gaze direction from the coordinates of the glint center point and the coordinates of the eye center point;
a typing interface module configured to determine a command corresponding to the gaze direction; and
a display configured to visually present the command to the user.
2. The system as claimed in claim 1 , wherein the camera is an infrared camera.
3. The system as claimed in claim 1 , further comprising a speaker configured to audibly present the command to the user.
4. The system as claimed in claim 1 , further comprising a tutorial module configured to present a tutorial to the user.
5. The system as claimed in claim 1 , further comprising a text-to-speech module configured to convert text of the command to speech.
6. A computer-implemented method for obtaining a command of a user comprising:
capturing an image of a user;
determining coordinates of a glint center point from the image;
determining coordinates of an eye center point from the image, wherein the step of determining coordinates of the eye center point includes:
a) identifying a first set of edge points having a shape from the image;
b) identifying a second set of edge points from the shape of the first set of edge points; and
c) averaging coordinates of the second set of edge points;
determining a gaze direction from the coordinates of the glint center point and the coordinates of the eye center point; and
determining a command corresponding to the gaze direction.
7. The method as claimed in claim 6 , wherein the second set of edge points represents a pupil.
8. The method as claimed in claim 6 , wherein the second set of edge points represents an iris.
9. The method as claimed in claim 6 , wherein the step of determining coordinates of the eye center point further comprises:
d) identifying a region of pixels in proximity to the glint center point;
e) determining whether pixels in the region is within a color threshold;
f) averaging coordinates of the pixels within the color threshold; and
g) calculating a weighted average of the coordinates obtained from steps a)-c) and from the coordinates obtained from steps d)-f), wherein the weighted average of the coordinates is weighted based on a confidence of the second set of edge points conforming to a circular pattern.
10. The method as claimed in claim 9 , wherein the second set of edge points represents a pupil.
11. The method as claimed in claim 9 , wherein the second set of edge points represents an iris.
12. A computer-implemented method for obtaining a command of a user comprising:
capturing an image of a user;
determining coordinates of a glint center point from the image, wherein the step of determining coordinates of the glint center point comprises:
determining whether brightness values of pixels in the image is within a color threshold; and
averaging coordinates of pixels within the color threshold;
determining coordinates of an eye center point from the image;
determining a gaze direction from the coordinates of the glint center point and the coordinates of the eye center point; and
determining a command corresponding to the gaze direction.
13. The method as claimed in claim 12 , wherein the step of determining coordinates of the glint center point further comprises removing regions of pixels in the image wherein the regions are not within a size threshold.
14. The method as claimed in claim 12 , wherein the step of determining coordinates of the glint center point further comprises removing regions of pixels without counterparts from the image.
15. A computer-implemented method for obtaining a command of a user comprising:
capturing an image of a user;
determining coordinates of a glint center point from the image;
determining coordinates of an eye center point from the image, wherein the step of determining coordinates of the eye center point comprises:
identifying a region of pixels in proximity to the glint center point;
determining whether pixels in the region is within a color threshold; and
averaging coordinates of the pixels within the color threshold;
determining a gaze direction from the coordinates of the glint center point and the coordinates of the eye center point; and
determining a command corresponding to the gaze direction.
16. A system for obtaining a command of a user comprising:
a camera configured to capture an image of the user;
an image capture module configured to retrieve the image from the camera;
an image analysis module configured to determine coordinates of a glint center point from the image, and further configured to a) identify a region of pixels in proximity to the glint center point; b) determine whether pixels in the region is within a color threshold; and c) average coordinates of the pixels within the color threshold to determine coordinates of an eye center point from the image;
a gaze direction determiner module configured to determine a gaze direction from the coordinates of the glint center point and the coordinates of the eye center point;
a typing interface module configured to determine a command corresponding to the gaze direction; and
a display configured to visually present the command to the user.
17. The system as claimed in claim 16 , wherein the camera is an infrared camera.
18. The system as claimed in claim 16 , further comprising a speaker configured to audibly present the command to the user.
19. The system as claimed in claim 16 , further comprising a tutorial module configured to present a tutorial to the user.
20. The system as claimed in claim 16 , further comprising a text-to-speech module configured to convert text of the command to speech.