IP Library Granted Patent US 10,503,973
Granted Patent B2
US 10,503,973 · App. 16/213,043 · Granted Dec 10, 2019

Pupil detection device

Inventors: Yu-Hao Huang (Hsin-Chu County, TW); Ming-Tsan Kao (Hsin-Chu County, TW); Yi-Fang Lee (Hsin-Chu County, TW); En-Feng Hsu (Hsin-Chu County, TW); Meng-Huan Hsieh (Hsin-Chu County, TW); Nien-Tse Chen (Hsin-Chu County, TW)
Assignee: PIXART IMAGING INC.
G06K9/00604G06F3/013H04N5/2354
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,503,973
App. No.
16/213,043
Granted
Dec 10, 2019
Kind
B2
Abstract

There is provided a pupil tracking device including an active light source, an image sensor and a processing unit. The active light source emits light toward an eyeball alternatively in a first brightness value and a second brightness value. The image sensor captures a first brightness image corresponding to the first brightness value and a second brightness image corresponding to the second brightness value. The processing unit identifies a brightest region at corresponding positions of the first brightness image and the second brightness image as an active light image.

Claims (46)

1. A pupil tracking device, comprising:

a single light source configured to emit light toward an eyeball in a first brightness value at a first time to form a first reflection spot on the eyeball, and the single light source configured to emit light toward the eyeball in a second brightness value at a second time to form a second reflection spot on the eyeball, wherein the first brightness value is different from the second brightness value, and the first time is different from the second time;

an image sensor configured to capture a first image containing a brightest region associated with the first reflection spot on the eyeball corresponding to the first brightness value and a second image containing a brightest region associated with the second reflection spot on the eyeball corresponding to the second brightness value, wherein the image sensor is further configured to capture a third image when the single light source is being turned off; and

a processing unit electrically coupled to the image sensor and the single light source, and configured to

calculate a first differential image between the first image and the third image, and calculate a second differential image between the second image and the third image;

calculate a differential image between the first differential image and the second differential image, and

identify a darkest region in the differential image as an image of the single light source projected on the eyeball when a position of the darkest region in the differential image corresponds to a position of the brightest region in the first differential image or the brightest region in the second differential image.

2. The pupil tracking device as claimed in claim 1 , wherein the processing unit is further configured to identify a pupil position according to the first differential image or the second differential image.

3. The pupil tracking device as claimed in claim 2 , wherein

the first brightness value is higher than the second brightness value; and

the processing unit is configured to

identify a plurality of pixels around a lowest gray level in the first differential image and having gray levels within a gray level range as a pupil region, and

calculate a gravity center or a center of the pupil region as the pupil position.

4. The pupil tracking device as claimed in claim 2 , further comprising a memory unit configured to save pupil coordinate information associated with the image of the single light source and the pupil position.

5. The pupil tracking device as claimed in claim 1 , wherein the image sensor is configured to

capture the first image with a first exposure time and a first gain, and

capture the second image with a second exposure time and a second gain, wherein the first exposure time is not equal to the second exposure time or the first gain is not equal to the second gain.

6. The pupil tracking device as claimed in claim 1 , wherein the pupil tracking device is disposed on a head accessory.

7. The pupil tracking device as claimed in claim 1 , wherein the single light source is configured to emit light in the first brightness value and the second brightness value alternatively.

8. A pupil tracking device, comprising:

a single light source configured to emit light toward an eyeball in a first brightness value at a first time to form a first reflection spot on the eyeball, and the single light source configured to emit light toward the eyeball in a second brightness value at a second time to form a second reflection spot on the eyeball, wherein the first brightness value is different from the second brightness value, and the first time is different from the second time;

an image sensor configured to capture a first image containing a brightest region associated with the first reflection spot on the eyeball corresponding to the first brightness value and a second image containing a brightest region associated with the second reflection spot on the eyeball corresponding to the second brightness value, wherein the image sensor is further configured to capture a third image when the single light source is being turned off; and

a processing unit electrically coupled to the image sensor and the single light source, and configured to

calculate a first differential image between the first image and the third image, and calculate a second differential image between the second image and the third image;

calculate a differential image by subtracting the second differential image from the first differential image, and

identify one of two darkest regions in the differential image having a smaller area and corresponding to a position of the brightest region in the first or second differential image as an image of the single light source projected on the eyeball, and the other one of the two darkest regions in the differential image having a larger area as a pupil image.

9. The pupil tracking device as claimed in claim 8 , wherein the pupil tracking device is disposed on a head accessory.

10. The pupil tracking device as claimed in claim 8 , wherein the image sensor is configured to

capture the first image with a first exposure time and a first gain, and

capture the second image with a second exposure time and a second gain, wherein the first exposure time is not equal to the second exposure time or the first gain is not equal to the second gain.

11. The pupil tracking device as claimed in claim 8 , further comprising a memory unit configured to save pupil coordinate information associated with the image of the single light source and the pupil image.

12. The pupil tracking device as claimed in claim 8 , wherein the single light source is configured to emit light in the first brightness value and the second brightness value alternatively.

13. A pupil tracking device, comprising:

a single light source configured to emit light toward an eyeball in a first brightness value at a first time to form a first reflection spot on the eyeball, and the single light source configured to emit light toward the eyeball in a second brightness value at a second time to form a second reflection spot on the eyeball, wherein the first brightness value is equal to the second brightness value, and the first time is different from the second time;

an image sensor configured to capture, using a first exposure time, a first image containing a brightest region associated with the first reflection spot on the eyeball corresponding to the first brightness value and capture, using a second exposure time, a second image containing a brightest region associated with the second reflection spot on the eyeball corresponding to the second brightness value, wherein the first exposure time is different from the second exposure time, wherein the image sensor is further configured to capture a third image when the single light source is being turned off; and

a processing unit electrically coupled to the image sensor and the single light source, and configured to

calculate a first differential image between the first image and the third image, and calculate a second differential image between the second image and the third image;

calculate a differential image between the first differential image and the second differential image, and

identify a darkest region in the differential image as an image of the single light source projected on the eyeball when a position of the darkest region in the differential image corresponds to a position of the brightest region in the first differential image or the brightest region in the second differential image.

14. The pupil tracking device as claimed in claim 13 , wherein the image sensor is further configured to

capture the first image using a first gain,

capture the second image using a second gain, and

the first gain is different from the second gain.

15. The pupil tracking device as claimed in claim 13 , wherein the processing unit is further configured to identify a pupil position according to the first differential image or the second differential image.

16. The pupil tracking device as claimed in claim 15 , further comprising a memory unit configured to save pupil coordinate information associated with the image of the single light source and the pupil position.

17. The pupil tracking device as claimed in claim 13 , wherein the pupil tracking device is disposed on a head accessory.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 27, 2019
From: PIXART IMAGING INC.
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 051431/0464 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 7, 2018
From: HUANG, YU-HAO; KAO, MING-TSAN; LEE, YI-FANG; HSU, EN-FENG; HSIEH, MENG-HUAN; CHEN, NIEN-TSE
To: PIXART IMAGING INC.
Reel/Frame 047705/0424 →
Priority Claims (1)
TW 101139163 A · Oct 23, 2012 · national
Continuity (3)
Continuation 15253503 · Aug 31, 2016
Continuation 14054724 · Oct 15, 2013
Related Publication 20190108395A1 · Apr 11, 2019