IP Library Granted Patent US 10,198,631
Granted Patent B2
US 10,198,631 · App. 15/253,503 · Granted Feb 5, 2019

Pupil detection device

Inventors: Yu-Hao Huang (Hsin-Chu County, TW); Ming-Tsan Kao (Hsin-Chu County, TW); Yi-Fang Lee (Hsin-Chu County, TW); En-Feng Hsu (Hsin-Chu County, TW); Meng-Huan Hsieh (Hsin-Chu County, TW); Nien-Tse Chen (Hsin-Chu County, TW)
Assignee: PIXART IMAGING INC.
G06K9/00604G06F3/013H04N5/2354
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,198,631
App. No.
15/253,503
Granted
Feb 5, 2019
Kind
B2
Abstract

There is provided a pupil tracking device including an active light source, an image sensor and a processing unit. The active light source emits light toward an eyeball alternatively in a first brightness value and a second brightness value. The image sensor captures a first brightness image corresponding to the first brightness value and a second brightness image corresponding to the second brightness value. The processing unit identifies a brightest region at corresponding positions of the first brightness image and the second brightness image as an active light image.

Claims (48)

1. A pupil tracking device, comprising:

an active light source configured to emit light toward an eyeball in a first brightness value at a first time to form a first reflection spot on the eyeball and the same active light source configured to emit light toward the eyeball in a second brightness value at a second time to form a second reflection spot on the eyeball, wherein the first brightness value is different from the second brightness value, and the first time is different from the second time;

an image sensor configured to capture a first brightness image containing a brightest region associated with the first reflection spot on the eyeball corresponding to the first brightness value and a second brightness image containing a brightest region associated with the second reflection spot on the eyeball corresponding to the second brightness value; and

a processing unit electrically coupled to the image sensor and the active light source, and configured to

calculate a differential image between the first brightness image and the second brightness image, and

identify a darkest region in the differential image as an image of the active light source projected on the eyeball when a position of the darkest region in the differential image corresponds to a position of the brightest region in the first brightness image or the brightest region in the second brightness image.

2. The pupil tracking device as claimed in claim 1 , wherein the processing unit is further configured to identify a pupil position according to the first brightness image or the second brightness image.

3. The pupil tracking device as claimed in claim 2 , wherein

the first brightness value is higher than the second brightness value; and

the processing unit is configured to

identify a plurality of pixels around a lowest gray level in the first brightness image and having gray levels within a gray level range as a pupil region, and

calculate a gravity center or a center of the pupil region as the pupil position.

4. The pupil tracking device as claimed in claim 2 , further comprising a memory unit configured to save pupil coordinate information associated with the image of the active light source and the pupil position.

5. The pupil tracking device as claimed in claim 1 , wherein

the image sensor is configured to

capture the first brightness image with a first exposure time and a first gain, and

capture the second brightness image with a second exposure time and a second gain, wherein the first exposure time is not equal to the second exposure time or the first gain is not equal to the second gain.

6. The pupil tracking device as claimed in claim 1 , wherein the pupil tracking device is disposed on a head accessory.

7. A pupil tracking device, comprising:

two active light sources configured to alternatively emit light toward an eyeball to respectively form a reflection spot on the eyeball;

an image sensor configured to capture a first image frame containing a brightest region associated with one of the reflection spots on the eyeball and a second image frame containing a brightest region associated with the other one of the reflection spots on the eyeball corresponding to lighting of different active light sources among the two active light sources; and

a processing unit electrically coupled to the image sensor and the two active light sources, and configured to

calculate a differential image between the first image frame and the second image frame, and

identify both of a darkest region and a brightest region in the differential image corresponding to positions of the brightest regions in the first and second image frames as images of the two active light sources projected on the eyeball.

8. The pupil tracking device as claimed in claim 7 , wherein the processing unit is further configured to identify a first pupil region according to the first image frame and identify a second pupil region according to the second image frame.

9. The pupil tracking device as claimed in claim 8 , wherein the processing unit is further configured to

identify

a plurality of pixels around a lowest gray level in the first image frame and having gray levels within a gray level range as the first pupil region, and

a plurality of pixels around a lowest gray level in the second image frame and having gray levels within the gray level range as the second pupil region,

calculate

a gravity center or a center of the first pupil region as a first pupil position, and

a gravity center or a center of the second pupil region as a second pupil position.

10. The pupil tracking device as claimed in claim 8 , wherein the processing unit is further configured to define a joined region of the first pupil region and the second pupil region as an outputted pupil region.

11. The pupil tracking device as claimed in claim 7 , wherein

the pupil tracking device comprises four active light sources, and

the processing unit is configured to identify four images of the four active light sources projected on a cornea of the eyeball in the differential image and respectively calculate a positioning coordinate associated with the four images.

12. The pupil tracking device as claimed in claim 11 , wherein the processing unit is further configured to calculate an eyeball center according to the four positioning coordinates.

13. The pupil tracking device as claimed in claim 12 , wherein the processing unit is further configured to

identify an outputted pupil region according to the first image frame and the second image frame, and

define a direction of a line connecting the eyeball center and the outputted pupil region as a direction of line of sight.

14. The pupil tracking device as claimed in claim 7 , wherein the pupil tracking device is disposed on a head accessory.

15. A pupil tracking device, comprising:

an active light source configured to emit light toward an eyeball in a first brightness value at a first time to form a first reflection spot on the eyeball and the same active light source configured to emit light toward the eyeball in a second brightness value at a second time to form a second reflection spot on the eyeball, wherein the first brightness value is different from the second brightness value, and the first time is different from the second time;

an image sensor configured to capture a first brightness image containing a brightest region associated with the first reflection spot on the eyeball corresponding to the first brightness value and a second brightness image containing a brightest region associated with the second reflection spot on the eyeball corresponding to the second brightness value; and

a processing unit electrically coupled to the image sensor and the active light source, and configured to

calculate a differential image by subtracting the second brightness image from the first brightness image, and

identify one of two darkest regions in the differential image having a smaller area and corresponding to a position of the brightest region in the first or second brightness image as an image of the active light source projected on the eyeball, and the other one of the two darkest regions in the differential image having a larger area as a pupil image.

16. The pupil tracking device as claimed in claim 15 , wherein the pupil tracking device is disposed on a head accessory.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 27, 2019
From: PIXART IMAGING INC.
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 051431/0464 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 31, 2016
From: HUANG, YU-HAO; KAO, MING-TSAN; LEE, YI-FANG; HSU, EN-FENG; HSIEH, MENG-HUAN; CHEN, NIEN-TSE
To: PIXART IMAGING INC.
Reel/Frame 039606/0739 →
Priority Claims (1)
TW 101139163 A · Oct 23, 2012 · national
Continuity (2)
Continuation 14054724 · Oct 15, 2013
Related Publication 20160370859A1 · Dec 22, 2016