IP Library › Granted Patent US 9,846,807
Granted Patent B1
US 9,846,807 · App. 14/986,427 · Granted Dec 19, 2017

Detecting eye corners

Inventors: Joseph Mayer (Wayne, NJ); Brian Martin (McMurray, PA)
Assignee: MorphoTrust USA, LLC
G06K9/0061G06K9/00604G06K9/4609G06K9/6203G06K9/6267G06K9/66G06T7/0042G06T7/0081G06K9/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,846,807
App. No.
14/986,427
Granted
Dec 19, 2017
Kind
B1
Abstract

A method and system for detecting eye corners using neural network classifiers is described. After an eye image is received, the eye image may be processed by at least two neural network classifiers including an inner eye corner neural network classifier and an outer eye corner neural network classifier. The neural network classifiers provide periocular information including a distance or coordinates of an eye corner location from a center of an iris of the eye, and an outcome of whether the eye corner is an inner eye corner or an outer eye corner. Output from the various neural network classifiers are combined to generate a decision on the location of eye corners in an eye image.

Claims (82)

1. A computer-implemented method comprising:

detecting an eye image in an image;

classifying, using one or more of a plurality of classifiers coupled to a processor, an inner eye corner in the eye image;

determining, using one or more of the plurality of classifiers coupled to the processor, one or more relative displacements of the inner eye corner from an iris in the eye image based on the classification of the inner eye corner;

determining an average displacement of the inner eye corner from the iris in the eye image based on the relative displacements of the inner eye corner provided by the one or more of the plurality of classifiers;

classifying, using one or more of the plurality of classifiers coupled to the processor, an outer eye corner in the eye image;

determining, using one or more of the plurality of classifiers coupled to the processor, one or more relative displacements of the outer eye corner from the iris in the eye image based on the classification of the outer eye corner;

determining an average displacement of the outer eye corner from the iris in the eye image based on the relative displacements of the outer eye corner provided by the one or more of the plurality of classifiers; and

identifying, using the processor, a location of the inner eye corner in the image based on the determined average displacement of the inner eye corner and a location of the outer eye corner in the image based on the determined average displacement of the outer eye corner.

2. The computer-implemented method of claim 1 , further comprising:

dividing the eye image into a plurality of regions of interest,

wherein each of the regions of interest are processed by the one or more of the plurality of classifiers to classify the inner eye corner or the outer eye corner.

3. The computer-implemented method of claim 1 , wherein identifying, using the processor, the location of the inner eye corner in the image based on the determined average displacement of the inner eye corner and the location of the outer eye corner in the image based on the determined average displacement of the outer eye corner comprises:

determining whether the average displacement of the inner eye corner from the iris in the eye image satisfies a threshold; and

identifying the location of the inner eye corner in the image based on the average displacement of the inner eye corner from the iris in the eye image satisfying the threshold.

4. The computer-implemented method of claim 1 , wherein identifying, using the processor, the location of the inner eye corner in the image based on the determined average displacement of the inner eye corner and the location of the outer eye corner in the image based on the determined average displacement of the outer eye corner comprises:

determining whether the average displacement of the outer eye corner from the iris in the eye image satisfies a threshold;

identifying the location of the outer eye corner in the image based on the average displacement of the outer eye corner from the iris in the eye image satisfying the threshold.

5. The computer-implemented method of claim 1 , further comprising:

receiving the image from an image acquisition device; and

processing the eye image to center the iris of an eye imaged by the eye image.

6. The computer-implemented method of claim 1 , wherein the plurality of classifiers comprises a plurality of neural network classifiers, respectively.

7. A non-transitory computer-readable storage medium comprising instructions, which, when executed by one or more computers, cause the one or more computers to perform actions comprising:

detecting an eye image in an image;

classifying, using one or more of a plurality of classifiers, an inner eye corner in the eye image;

determining one or more relative displacements of the inner eye corner from an iris in the eye image based on the classification of the inner eye corner;

determining an average displacement of the inner eye corner from the iris in the eye image based on the relative displacements of the inner eye corner provided by the one or more of the plurality of classifiers;

classifying, using one or more of the plurality of classifiers, an outer eye corner in the eye image;

determining one or more relative displacements of the outer eye corner from the iris in the eye image based on the classification of the outer eye corner;

determining an average displacement of the outer eye corner from the iris in the eye image based on the relative displacements of the outer eye corner provided by the one or more of the plurality of classifiers; and

identifying a location of the inner eye corner in the image based on the determined average displacement of the inner eye corner and a location of the outer eye corner in the image based on the determined average displacement of the outer eye corner.

8. The non-transitory computer-readable storage medium of claim 7 , wherein the actions further comprise:

dividing the eye image into a plurality of regions of interest,

wherein each of the regions of interest are processed by the one or more of the plurality of classifiers to classify the inner eye corner or the outer eye corner.

9. The non-transitory computer-readable storage medium of claim 8 , wherein identifying the location of the inner eye corner in the image based on the determined average displacement of the inner eye corner and the location of the outer eye corner in the image based on the determined average displacement of the outer eye corner comprises:

determining whether the average displacement of the inner eye corner from the iris in the eye image satisfies a threshold; and

identifying the location of the inner eye corner in the image based on the average displacement of the inner eye corner from the iris in the eye image satisfying the threshold.

10. The non-transitory computer-readable storage medium of claim 8 , wherein identifying, using the processor, the location of the inner eye corner in the image based on the determined average displacement of the inner eye corner and the location of the outer eye corner in the image based on the determined average displacement of the outer eye corner comprises:

determining whether the average displacement of the outer eye corner from the iris in the eye image satisfies a threshold; and

identifying the location of the outer eye corner in the image based on the average displacement of the outer eye corner from the iris in the eye image satisfying the threshold.

11. The non-transitory computer-readable storage medium of claim 10 , wherein the actions further comprise:

receiving the image from an image acquisition device; and

processing the eye image to center the iris of an eye imaged by the eye image.

12. The non-transitory computer-readable storage medium of claim 10 , wherein the plurality of classifiers comprises a plurality of neural network classifiers, respectively.

13. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable and when executed by one or more computers, cause the one or more computers to perform actions comprising:

detecting an eye image in an image;

classifying, using one or more of a plurality of classifiers, an inner eye corner in the eye image;

determining one or more relative displacements of the inner eye corner from an iris in the eye image based on the classification of the inner eye corner;

determining an average displacement of the inner eye corner from the iris in the eye image based on the relative displacements of the inner eye corner provided by the one or more of the plurality of classifiers;

classifying, using one or more of the plurality of classifiers, an outer eye corner in the eye image;

determining one or more relative displacements of the outer eye corner from the iris in the eye image based on the classification of the outer eye corner;

determining an average displacement of the outer eye corner from the iris in the eye image based on the relative displacements of the outer eye corner provided by the one or more of the plurality of classifiers; and

identifying a location of the inner eye corner in the image based on the determined average displacement of the inner eye corner and a location of the outer eye corner in the image based on the determined average displacement of the outer eye corner.

14. The system of claim 13 , wherein the actions further comprise: dividing the eye image into a plurality of regions of interest,

wherein each of the regions of interest are processed by the one or more of the plurality of classifiers to classify the inner eye corner or the outer eye corner.

15. The system of claim 13 , wherein identifying the location of the inner eye corner in the image based on the determined average displacement of the inner eye corner and the location of the outer eye corner in the image based on the determined average displacement of the outer eye corner comprises:

determining whether the average displacement of the inner eye corner from the iris in the eye image satisfies a threshold; and

identifying the location of the inner eye corner in the image based on the average displacement of the inner eye corner from the iris in the eye image satisfying the threshold.

16. The system of claim 13 , wherein identifying, using the processor, the location of the inner eye corner in the image based on the determined average displacement of the inner eye corner and the location of the outer eye corner in the image based on the determined relative displacements average displacement of the outer eye corner comprises:

determining whether the average displacement of the outer eye corner from the iris in the eye image satisfies a threshold; and

identifying the location of the outer eye corner in the image based on the average displacement of the outer eye corner from the iris in the eye image satisfying the threshold.

17. The system of claim 14 , wherein:

the plurality of classifiers comprises a plurality of neural network classifiers, respectively; and

the actions further comprise:

receiving the image from an image acquisition device; and

processing the eye image to center the iris of an eye imaged by the eye image.

18. The computer-implemented method of claim 1 , further comprising:

in response to detecting the eye image in the image, performing, by the processor, one or more of:

downsampling the image;

removing eyelid and eyelash occlusions from the image segmenting the image into a plurality of regions of interest; and

detecting one or more edges in the image.

19. The non-transitory computer-readable storage medium of claim 7 , wherein the actions further comprise:

in response to detecting the eye image in the image, performing, by the processor, one or more of:

downsampling the image;

removing eyelid and eyelash occlusions from the image segmenting the image into a plurality of regions of interest; and

detecting one or more edges in the image.

20. The system of claim 13 , wherein the actions further comprise:

in response to detecting the eye image in the image, performing, by the processor, one or more of:

downsampling the image;

removing eyelid and eyelash occlusions from the image segmenting the image into a plurality of regions of interest; and

detecting one or more edges in the image.

Assignments (2)
CHANGE OF NAME Recorded Dec 23, 2022
From: MORPHOTRUST USA, LLC
To: IDEMIA IDENTITY & SECURITY USA LLC
Reel/Frame 062218/0605 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 3, 2016
From: MAYER, JOSEPH; MARTIN, BRIAN
To: MORPHOTRUST USA, LLC
Reel/Frame 037658/0551 →
Continuity (1)
Provisional Application 62098883 · Dec 31, 2014