IP Library Granted Patent US 7,580,545
Granted Patent B2
US 7,580,545 · App. 11/243,442 · Granted Aug 25, 2009

Method and system for determining gaze direction in a pupil detection system

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,580,545
App. No.
11/243,442
Granted
Aug 25, 2009
Kind
B2
Abstract

A detector captures one or more images of the face, eyes, or pupils of a person using light propagating at two or more angles of incidence with respect to the optical axis of the detector. A difference image is then generated by subtracting one image from another image. A computed location associated with the pupils in the difference image is then compared with a reference location to determine the gaze direction of the pupils.

Claims (44)

1. A pupil detection system, comprising:

an imager operable to capture images of the pupils, wherein the imager comprises a hybrid filter comprising:

a dual-band narrowband filter with a first transmission peak at a first wavelength of a first light source and a second transmission peak at a second wavelength of a second light source, wherein the dual-band narrowband filter is configured to substantially block wavelengths of light other than approximately the first wavelength and the second wavelength; and

a patterned filter layer to receive light at the first and second wavelength from the dual-band narrowband filter, wherein the patterned filter layer comprises:

areas of a first filter characteristic to transmit light at approximately the first wavelength to corresponding pixels of the imager; and

areas of a second filter characteristic to transmit light at approximately the second wavelength to other corresponding pixels of the imager; and

a processing unit operable to generate a difference image using the captured images, wherein the difference image indicates a position of the pupils due to differential reflectivity from retinal return in the captured images, and to determine a gaze direction for the pupils using the difference image.

2. The pupil detection system of claim 1 , further comprising one or more light sources operable to emit light towards the pupils, wherein the light strikes the pupils at two or more different angles of incidence, wherein a difference between the angles of incidence corresponds to the differential reflectivity from the retinal return in the captured images.

3. The pupil detection system of claim 2 , wherein the one or more light sources comprise a broadband light source.

4. The pupil detection system of claim 1 , wherein the processing unit is operable to determine the gaze location by determining a difference between a current point location and a reference point location associated with the pupils, wherein the reference point location comprises a reference midpoint location between the pupils determined over a plurality of the captured images, and the current point location comprises a current midpoint location between the pupils.

5. The pupil detection system of claim 1 , wherein the imager is operable to capture the images of the pupils in a composite image, wherein the composite image comprises at least two sub-frame images which comprise separate portions of the composite image and together form a complete image, wherein the at least two sub-frame images are captured using light propagating at different wavelengths.

6. The pupil detection system of claim 1 , wherein the captured images comprise two distinct images.

7. The pupil detection system of claim 1 , wherein the captured images are captured simultaneously.

8. The pupil detection system of claim 1 , wherein the captured images are captured successively.

9. A method for determining a gaze direction in a pupil detection system, the method comprising:

capturing one or more images of a person's pupils using an imager which comprises a hybrid filter comprising:

a dual-band narrowband filter with a first transmission peak at a first wavelength of a first light source and a second transmission peak at a second wavelength of a second light source, wherein the dual-band narrowband filter is configured to substantially block wavelengths of light other than approximately the first wavelength and the second wavelength; and

a patterned filter layer to receive light at the first and second wavelength from the dual-band narrowband filter, wherein the patterned filter layer comprises:

areas of a first filter characteristic to transmit light at approximately the first wavelength to corresponding pixels of the imager; and

areas of a second filter characteristic to transmit light at approximately the second wavelength to other corresponding pixels of the imager;

generating a difference image using the captured images, wherein the difference image indicates a position of the pupils due to differential reflectivity from retinal return in the captured images; and

determining a gaze direction for the pupils using the difference image.

10. The method of claim 9 , further comprising emitting light towards the pupils.

11. The method of claim 9 , wherein:

capturing the images comprises capturing a composite image; and

generating the difference image using the captured images comprises subtracting a first sub-frame in the composite image from a second sub-frame in the composite image.

12. The method of claim 9 , wherein:

capturing the images comprises capturing two distinct images; and

generating the difference image comprises subtracting one of the captured image from the other captured image.

13. The method of claim 9 , wherein determining a gaze direction comprises:

determining a current point location in the difference image, wherein the current point location comprises a current midpoint location between the pupils;

determining a difference between the current point location and a reference point location, wherein the reference point location comprises a reference midpoint location between the pupils determined over a plurality of the captured images; and

determining a gaze direction for the pupils using the difference between the current and reference point locations.

14. The method of claim 13 , further comprising determining the reference point location, wherein determining the reference point location comprises:

capturing a plurality of images of the pupils;

determining a plurality of difference images using the captured images;

determining a point location in each of the difference images; and

determining a reference point location using the point locations for each of the difference images, wherein the reference point location comprises a statistical representation of the plurality of point locations in the plurality of difference images.

15. The method of claim 14 , further comprising incrementing a count of the number of captured images each time an image of the pupils is captured.

16. The method of claim 15 , further comprising re-computing the reference point location after a predetermined number of images has been captured.

17. The method of claim 14 , wherein the statistical representation of the plurality of point locations in the plurality of difference images comprises a mean for each of a plurality of horizontal values (x) and a plurality of vertical values (y).

18. The method of claim 9 , wherein capturing one or more images of the pupils comprises successively capturing images of the pupils.

19. The method of claim 9 , wherein capturing one or more images of the pupils comprises simultaneously capturing one or more images of the pupils.

20. The pupil detection system of claim 1 , wherein the processing unit is further operable to activate an alarm in response to a determination that the gaze direction for the pupils of a driver of a vehicle is directed at a location other than a road ahead gaze direction, wherein the road ahead gaze direction corresponds to the pupils looking in a direction toward a road.