IP Library Granted Patent US 10,831,264
Granted Patent B2
US 10,831,264 · App. 15/408,250 · Granted Nov 10, 2020

Eye image combination

Inventor: Adrian Kaehler (Los Angeles, CA)
Assignee: Magic Leap, Inc.
G06F3/013G06F1/163G06F3/011G06K9/0061G06K9/00335G06K9/00604G06K9/00617G06K9/00671G06K9/00926G06K9/036G06K9/2081G06K9/52G06T1/20G06T3/4053G06T7/74G06T11/60G06T13/40G06T19/006G06T2207/20221
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,831,264
App. No.
15/408,250
Granted
Nov 10, 2020
Kind
B2
Abstract

Systems and methods for eye image set selection, eye image collection, and eye image combination are described. Embodiments of the systems and methods for eye image combination can include assigning an eye pose region of a plurality of eye pose regions to each eye image and determining a representation of an eye pose in each eye image. The set of the determined representations can be fused to generate a hybrid eye image, which can in turn be used to generate an iris code associated with a confidence score.

Claims (59)

1. “A head mounted display system comprising:

an image capture device;

a processor programmed to:

for each of a plurality of eye images captured by the image capture device:

assign an eye pose region of a plurality of eye pose regions to each eye image; and

determine a representation of an eye pose in each eye image;

fuse two or more eye images of the plurality of eye images to generate a hybrid eye image;

generate an iris code of the hybrid eye image; and

determine a confidence score associated with the generated iris code, the confidence score based at least partly on a diversity of determined representations of the eye pose in the two or more eye images of the plurality of eye images utilized to generate the hybrid eye image.

2. The head mounted display system of claim 1 , wherein to fuse the two or more eye images of the plurality of eye images, the processor is programmed to select the two or more eye images of the plurality of eye images with different determined representations that pass an image quality threshold, wherein the image quality threshold corresponds to an image quality for utilization of a biometric application.

3. The head mounted display system of claim 1 , wherein to fuse the two or more eye images of the plurality of eye images, the processor is programmed to utilize a super resolution algorithm.

4. The head mounted display system of claim 1 , wherein the processor is further programmed to:

determine that the confidence score does not pass a confidence threshold, wherein the confidence threshold corresponds to a specific access level for an application associated with the head mounted display system; and

cause the application associated with the head mounted display system to terminate execution.

5. The head mounted display system of claim 1 , wherein the processor is further programmed to:

determine that the confidence score passes a confidence threshold, wherein the confidence threshold corresponds to a specific access level for an application associated with the head mounted display system; and

cause the application associated with the head mounted display system to indicate approval.

6. The head mounted display system of claim 5 , wherein the processor is further programmed to modify a level indicator displayed on the head mounted display system to indicate approval by the application, to cause an audio unit of the head mounted display system to emit sound, to cause a display of the head mounted display system to display an approval text, or a combination thereof.

7. The head mounted display system of claim 1 , wherein the processor is further programmed to:

determine biometric data of an eye using the hybrid eye image.

8. “A head mounted display system comprising:

an image capture device; and

a processor programmed to:

for each image of the plurality of eye images captured by the image capture device:

assign an eye pose region of a plurality of eye pose regions to each eye image;

determine a representation of an eye pose in each eye image; and

generate an iris code for each eye image;

merge each generated iris code to generate a hybrid iris code; and

determine a confidence score associated with the hybrid iris code, the confidence score based at least partly on a diversity of determined representations of eye poses in each eye image utilized to generate the hybrid iris code.

9. The head mounted display system of claim 8 , wherein to merge each generated iris code, the processor is programmed to compare each determined iris code to a threshold score, wherein the threshold score corresponds to a quality level of the eye image.

10. The head mounted display system of claim 8 , wherein to merge each generated iris code, the processor is programmed to utilize a filter configured to merge iris codes comprising at least: a median filter or a Bayes filter.

11. The head mounted display system of claim 8 , wherein the processor is further programmed to:

determine that the confidence score does not pass a confidence threshold, wherein the confidence threshold corresponds to a specific access level for an application associated with the head mounted display system; and

cause the application associated with the head mounted display system to terminate execution.

12. The head mounted display system of claim 8 , wherein the processor is further programmed to:

determine that the confidence score passes the confidence threshold; and

cause the application associated with the head mounted display system to indicate approval.

13. The head mounted display system of claim 12 , wherein the processor is further programmed to modify a security bar displayed on the head mounted display system, to cause an audio unit of the head mounted display system to emit sound, to display a textual indication on a display of the head mounted display system, or a combination thereof.

14. A method for obtaining an iris code of an eye, the method comprising:

under control of a processor:

accessing a plurality of eye images;

performing (1) an image fusion operation on the plurality of eye images, (2) an iris code fusion operation on the plurality of eye images, or both (1) and (2),

wherein the image fusion operation comprises:

identifying an eye pose for each eye image in the plurality of eye images;

fusing a first set of two or more eye images of the plurality of eye images to generate a hybrid image; and

generating a hybrid iris code from the hybrid image, and

wherein the iris code fusion operation comprises:

generating an iris code for a second set of two or more eye images of the eye images in the plurality of eye images;

merging the generated iris codes to generate a composite iris code;

determining a confidence score based at least partly on a diversity of eye poses in the first set of two or more eye images or the second set of two or more eye images.

15. The method of claim 14 , wherein the image fusion operation or the iris code fusion operation is performed on only one or more eye images having an image quality metric that passes an image quality threshold.

16. The method of claim 14 , further comprising:

determining that the confidence score does not pass a biometric security confidence threshold, wherein the biometric security confidence threshold corresponds to a specific access level for an application associated with biometric data; and

causing execution of the application to terminate.

17. The method of claim 14 , further comprising utilizing the hybrid iris code or the composite iris code for a biometric application.

18. The method of claim 14 , further comprising correcting for fragile bits.

19. The head mounted display system of claim 1 , wherein the diversity of determined representations comprises a number of pose regions represented by eye images or representations of eye images with different region assignments used to generate the iris code of the hybrid eye image.

20. The head mounted display system of claim 8 , wherein the diversity of determined representations comprises a number of pose regions represented by eye images or representations of eye images with different region assignments used to generate the hybrid iris code.

21. The head mounted display system of claim 14 , wherein the diversity of eye poses in the first set of two or more eye images comprises a number of pose regions represented by eye images or representations of eye images with different region assignments used to generate the hybrid image and wherein the diversity of eye poses in the second set of two or more eye images comprises a number of pose regions represented by eye images or representations of eye images with different region assignments used to generate the composite iris code.

Assignments (4)
SECURITY INTEREST Recorded Oct 28, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073388/0027 →
ASSIGNMENT OF SECURITY INTEREST IN PATENTS Recorded Nov 7, 2019
From: JPMORGAN CHASE BANK, N.A.
To: CITIBANK, N.A.
Reel/Frame 050967/0138 →
PATENT SECURITY AGREEMENT Recorded Aug 22, 2019
From: MAGIC LEAP, INC.; MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC
To: JP MORGAN CHASE BANK, N.A.
Reel/Frame 050138/0287 →
PROPRIETARY INFORMATION AND INVENTIONS AGREEMENT Recorded Jun 26, 2018
From: KAEHLER, ADRIAN
To: MAGIC LEAP, INC.
Reel/Frame 046427/0783 →
Continuity (4)
Provisional Application 62280456 · Jan 19, 2016
Provisional Application 62280515 · Jan 19, 2016
Provisional Application 62280437 · Jan 19, 2016
Related Publication 20170206401A1 · Jul 20, 2017