IP Library › Granted Patent US 12,688,268
Granted Patent B2
US 12,688,268 · App. 18/027,919 · Granted Jul 21, 2026

Pose optimization in biometric authentication systems

Inventors: Martin Haller (Mountain View, CA); Tom Sengelaub (Oakland, CA)
Assignee: Apple Inc.
G06F21/32G06V10/141G06V10/993G06V40/19G06V40/197G06V40/67H04N23/23H04N23/56H04N23/80
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,688,268
App. No.
18/027,919
Granted
Jul 21, 2026
Kind
B2
Abstract

Methods and apparatus for biometric authentication in which a current eye pose is determined and evaluated to determine if the current pose is satisfactory, and in which the eye pose may be improved by the user manually adjusting the device or their pose/gaze direction in response to a signal from the controller, and/or in which the imaging system is mechanically adjusted at the direction of the controller to improve the current view of the eye.

Claims (37)

1 . A system, comprising:

a camera configured to capture images of an eye region of a user; and

a controller comprising one or more processors configured to:

determine and evaluate, based on one or more of the images captured by the camera, a current eye pose of the user, wherein the eye pose indicates a current eye location and gaze direction with respect to the camera;

determine, based on the evaluation of the current eye location and the current gaze direction, that the current eye pose determined from the one or more images captured by the camera is not satisfactory for use in a biometric authentication process for biometric authentication of the user;

signal, responsive to the determination that the current eye pose is not satisfactory for the biometric authentication of the user, the camera to automatically adjust a viewing angle of the camera with respect to the eye region based on the current eye location and the current gaze direction and capture one or more additional images of the eye region of the user using the adjusted viewing angle; and

perform the biometric authentication for the user based on the captured one or more additional images of the eye region of the user captured by the camera using the viewing angle automatically adjusted in response to the signal.

2 . The system as recited in claim 1 , wherein said determine and evaluate a current eye pose of the user, said determine that the current eye pose is not satisfactory for use in a biometric authentication process, and said signal the camera to automatically adjust a viewing angle of the camera with respect to the eye region are iteratively performed until a satisfactory eye pose is detected.

3 . The system as recited in claim 1 , wherein, to determine and evaluate a current eye pose of the user, the controller is configured to analyze one or more images of the eye region captured by the camera to determine if one or more objective criteria with regard to the eye region are achieved in the captured images.

4 . The system as recited in claim 3 , wherein the objective criteria include one or more of gaze direction, exposure, contrast, shadows, edges, undesirable streaks, occlusions, sharpness, uniformity of illumination, and undesired reflections.

5 . The system as recited in claim 1 , wherein the eye region includes one or more of an iris, an eye, a periorbital region, and a portion of the user's face.

6 . The system as recited in claim 1 , wherein, to perform biometric authentication for the user based on one or more additional images of the eye region of the user captured by the camera using the viewing angle automatically adjusted in response to the signal, the controller is configured to:

analyze quality of one or more biometric aspects captured in the one or more additional images according to one or more objective criteria;

select at least one biometric aspect according to the analysis; and

perform the biometric authentication process based at least in part on the selected at least one biometric aspect.

7 . The system as recited in claim 6 , wherein the objective criteria include one or more of exposure, contrast, shadows, edges, undesirable streaks, occluding objects, sharpness, uniformity of illumination, and undesired reflections.

8 . The system as recited in claim 6 , wherein the biometric aspects include one or more of an eye surface, eye veins, eyelids, eyebrows, skin features, nose features, and iris features, wherein the iris features include one or more of colors, patterns, and musculature.

9 . The system as recited in claim 1 , further comprising an illumination source comprising a plurality of light-emitting elements configured to emit light towards the eye region to be imaged by the camera.

10 . The system as recited in claim 9 , wherein the light-emitting elements include light-emitting diodes (LEDs).

11 . The system as recited in claim 9 , wherein the light-emitting elements include infrared (IR) light sources, and wherein the camera is an infrared camera.

12 . The system as recited in claim 1 , wherein the system is a component of a head-mounted device (HMD), a handheld device, or a wall-mounted device.

13 . A method, comprising:

performing, by a controller comprising one or more processors:

determining and evaluating, based on one or more of the images captured by the camera, a current eye pose of a user, wherein the eye pose indicates a current eye location and gaze direction with respect to a camera;

determining, based on the evaluation of the current eye location and the current gaze direction, that the current eye pose determined from the one or more images captured by the camera is not satisfactory for use in a biometric authentication process for biometric authentication of the user;

signaling, responsive to the determination that the current eye pose is not satisfactory for the biometric authentication of the user, the camera to automatically adjust a viewing angle of the camera with respect to the eye region based on the current eye location and the current gaze direction and capture one or more additional images of the eye region of the user using the adjusted viewing angle; and

performing the biometric authentication for the user based on the captured one or more additional images of the eye region of the user captured by the camera using the viewing angle automatically adjusted in response to the signal.

14 . The method as recited in claim 13 , further comprising iteratively performing said determining and evaluating a current eye pose of the user, said determining that the current eye pose is not satisfactory for use in a biometric authentication process, and said signaling the camera to automatically adjust a viewing angle of the camera with respect to the eye region until a satisfactory eye pose is detected.

15 . The method as recited in claim 13 , wherein determining and evaluating a current eye pose of the user comprises analyzing one or more images of the eye region captured by the camera to determine if one or more objective criteria with regard to the eye region are achieved in the captured images.

16 . The method as recited in claim 15 , wherein the objective criteria include one or more of gaze direction, exposure, contrast, shadows, edges, undesirable streaks, occlusions, sharpness, uniformity of illumination, and undesired reflections.

17 . The method as recited in claim 13 , wherein the eye region includes one or more of an iris, an eye, a periorbital region, and a portion of the user's face.

18 . The method as recited in claim 13 , wherein performing biometric authentication for the user based on one or more additional images of the eye region of the user captured by the camera using the viewing angle automatically adjusted in response to the signal comprises:

analyzing quality of one or more biometric aspects captured in the additional one or more images according to one or more objective criteria;

selecting at least one biometric aspect according to the analysis; and

performing the biometric authentication process based at least in part on the selected at least one biometric aspect.

19 . The method as recited in claim 18 , wherein the objective criteria include one or more of exposure, contrast, shadows, edges, undesirable streaks, occluding objects, sharpness, uniformity of illumination, and undesired reflections.

20 . The method as recited in claim 18 , wherein the biometric aspects include one or more of an eye surface, eye veins, eyelids, eyebrows, skin features, nose features, and iris features, wherein the iris features include one or more of colors, patterns, and musculature.

Continuity (2)
Provisional Application 63083761 · Sep 25, 2020
Related Publication 20230367857A1 · Nov 16, 2023
References Cited (137)
US 6289113B1 · McHugh · 2001 [cited by applicant]
US 7068820B2 · Nakaigawa · 2006 [cited by examiner]
US 7155354B1 · Mooney · 2006 [cited by applicant]
US 8433103B2 · Friedman · 2013 [cited by examiner]
US 9004687B2 · Stack · 2015 [cited by examiner]
US 9767358B2 · Xue · 2017 [cited by applicant]
US 9959478B2 · Hanna · 2018 [cited by applicant]
US 10156900B2 · Publicover · 2018 [cited by examiner]
US 10311286B2 · Saripalle et al. · 2019 [cited by applicant]
US 10354158B2 · Zhou · 2019 [cited by applicant]
US 10401158B2 · Gemoth · 2019 [cited by applicant]
US 10890968B2 · Kurlethimar · 2021 [cited by applicant]
US 10970574B2 · Joshi · 2021 [cited by applicant]
US 11210380B2 · Hoyos et al. · 2021 [cited by applicant]
US 11906742B2 · Kaehler et al. · 2024 [cited by applicant]
US 20020136435A1 · Prokoski · 2002 [cited by examiner]
US 20040151347A1 · Wisniewski · 2004 [cited by examiner]
US 20050249385A1 · Kondo · 2005 [cited by applicant]
US 20050271258A1 · Rowe · 2005 [cited by applicant]
US 20080037841A1 · Ogawa · 2008 [cited by examiner]
US 20100149315A1 · Qu · 2010 [cited by applicant]
US 20100316263A1 · Hamza · 2010 [cited by applicant]
US 20110163163A1 · Rowe · 2011 [cited by applicant]
US 20120154536A1 · Stoker et al. · 2012 [cited by applicant]
US 20130227651A1 · Schultz · 2013 [cited by examiner]
US 20130250087A1 · Smith · 2013 [cited by examiner]
US 20140044318A1 · Derakhshani · 2014 [cited by examiner]
US 20140044321A1 · Derakhshani · 2014 [cited by examiner]
US 20140099005A1 · Mogi · 2014 [cited by examiner]
US 20140283113A1 · Hanna · 2014 [cited by applicant]
US 20150262024A1 · Braithwaite · 2015 [cited by applicant]
US 20160019421A1 · Feng · 2016 [cited by applicant]
US 20160087952A1 · Tartz · 2016 [cited by examiner]
US 20160117544A1 · Xue · 2016 [cited by applicant]
US 20160132735A1 · Derakhshani · 2016 [cited by examiner]
US 20160180143A1 · Horesh · 2016 [cited by examiner]
US 20160360970A1 · Tzvieli · 2016 [cited by applicant]
US 20170124392A1 · Gu · 2017 [cited by applicant]
US 20170205875A1 · Kaehler · 2017 [cited by applicant]
US 20170206412A1 · Kaehler · 2017 [cited by applicant]
US 20170308763A1 · Salmelin · 2017 [cited by examiner]
US 20170337441A1 · Mapen · 2017 [cited by applicant]
US 20180025244A1 · Bohl · 2018 [cited by examiner]
US 20180032815A1 · Lee · 2018 [cited by applicant]
US 20180039846A1 · Grubb · 2018 [cited by applicant]
US 20180113508A1 · Berkner-Cieslicki · 2018 [cited by applicant]
US 20180120932A1 · Sengelaub · 2018 [cited by applicant]
US 20180150690A1 · Yin · 2018 [cited by applicant]
US 20180285669A1 · Mapen et al. · 2018 [cited by applicant]
US 20180330490A1 · Kido · 2018 [cited by applicant]
US 20180349721A1 · Agrawal et al. · 2018 [cited by applicant]
US 20180365490A1 · Agrawal · 2018 [cited by applicant]
US 20190044723A1 · Prakash · 2019 [cited by applicant]
US 20190057268A1 · Burge · 2019 [cited by applicant]
US 20190073533A1 · Chen · 2019 [cited by examiner]
US 20190094981A1 · Bradski · 2019 [cited by applicant]
US 20190278987A1 · Eskilsson · 2019 [cited by applicant]
US 20190347483A1 · Krichen · 2019 [cited by applicant]
US 20190387168A1 · Smith · 2019 [cited by applicant]
US 20200012105A1 · Liu · 2020 [cited by applicant]
US 20200127267A1 · Dulle · 2020 [cited by examiner]
US 20200218887A1 · He · 2020 [cited by applicant]
US 20200241635A1 · Cohen · 2020 [cited by examiner]
US 20200258044A1 · Joshi · 2020 [cited by applicant]
US 20200293757A1 · Nilsson · 2020 [cited by applicant]
US 20200311238A1 · Ackerman · 2020 [cited by applicant]
US 20200334478A1 · Hecht · 2020 [cited by applicant]
US 20210081652A1 · Matsunami · 2021 [cited by applicant]
US 20210318558A1 · Tzvieli · 2021 [cited by applicant]
US 20210334581A1 · Gandara · 2021 [cited by applicant]
US 20210365533A1 · Kaplan et al. · 2021 [cited by applicant]
US 20210365535A1 · Casteneda · 2021 [cited by applicant]
US 20220253135A1 · Cohen · 2022 [cited by applicant]
US 20230222197A1 · Derakhshani et al. · 2023 [cited by applicant]
US 20250103693A1 · Maila et al. · 2025 [cited by applicant]
CA 3120035 · 2021 [cited by applicant]
CN 1892676A · 2007 [cited by applicant]
CN 101411606A · 2009 [cited by applicant]
CN 101520838 · 2009 [cited by applicant]
CN 101814129 · 2013 [cited by applicant]
CN 103870819A · 2014 [cited by applicant]
CN 104301633A · 2015 [cited by applicant]
CN 105874472A · 2016 [cited by applicant]
CN 106462236A · 2017 [cited by applicant]
CN 107087431A · 2017 [cited by applicant]
CN 104933344 · 2019 [cited by applicant]
CN 109508695 · 2019 [cited by applicant]
CN 113505357 · 2021 [cited by applicant]
CN 114596622 · 2022 [cited by applicant]
EP 3506196 · 2019 [cited by applicant]
GB 2471192 · 2010 [cited by applicant]
JP H10137225 · 1998 [cited by applicant]
JP 2017047724A · 2017 [cited by applicant]
JP 2017191374 · 2017 [cited by applicant]
JP 7485448B2 · 2020 [cited by applicant]
KR 20180072734A · 2018 [cited by examiner]
KR 20180100890 · 2018 [cited by applicant]
KR 20200127267A · 2020 [cited by examiner]
KR 102483345B1 · 2022 [cited by examiner]
WO 9808439 · 1998 [cited by applicant]
WO 2017031089A1 · 2017 [cited by applicant]
WO 2017052807 · 2017 [cited by applicant]
WO 2017162504 · 2017 [cited by applicant]
WO 2018100241 · 2018 [cited by applicant]
WO 2019118716 · 2019 [cited by applicant]
WO 2020141537 · 2020 [cited by applicant]
WO 2023149829 · 2023 [cited by applicant]
“Sowon Yoon, Kwanghyuk Bae, Kang Ryoung Park, and Jaihie Kim”. “Pan-Tilt-Zoom Based Iris Image Capturing System for Unconstrained User Environments at a Distance.” (S.-W. Lee and S.Z. Li (Eds.): ICB 2007, LNCS 4642, pp.… [cited by examiner]
Invitation to pay additional fees, Partial International Search, date of mailing Dec. 7, 2021, pp. 1-16. [cited by applicant]
Sowon Yoon, et al., “Pan-Tilt-Zoom Based Iris Image Capturing System for Unconstrained User Environments at a Distance”, Aug. 27, 2007, ICIAP: International Conference of Image Analysis and Processing, 17th Internationa… [cited by applicant]
Ken Hughes et al., “Detection of Contact-Lens-Based Iris Biometric Spoofs Using Stereo Imaging”, 46th Hawaii International Conference on System Sciences, Jan. 1, 2013, pp. 1763-1772 [2013]. [cited by applicant]
International Search Report and Written Opinion from PCT/US2021/051612, date of mailing Dec. 23, 2021, pp. 1-13. [cited by applicant]
International Search Report and Written Opinion from PCT/US2021/051613, date of mailing Nov. 23, 2021, pp. 1-12. [cited by applicant]
Hau T Ngo, et al., “Design and implementation of a multispectral iris capture system”, Signals, Systems and Computers, 2009 Conference Record of the Forty-third ASIMLOMAR Conference, IEE, Nov. 1, 2009, pp. 380-384 [2009… [cited by applicant]
Thavalengal Shejin et al., “Proof-of-concept and evaluation of a dual function visible/NIR camera for iris authentication in smartphones”, IEEE Transactions on Consumer Electronics, IEEE Service Center, vol. 61, No. 2, … [cited by applicant]
International Search Report and Written Opinion from PCT/US2021/051615, date of mailing Dec. 9, 2021, all pages. [cited by applicant]
Emine Krichen et al., “Color-Based Iris Verification”, Proceedings of ICB: International Conference on Advances in Biometrics, Aug. 27, 2007, pp. 997-1005 [2007]. [cited by applicant]
International Search Report and Written Opinion from PCT/US2021/051611, date of mailing Jan. 2, 2022, pp. 1-13. [cited by applicant]
U.S. Appl. No. 18/027,910, filed Mar. 22, 2023, Martin Haller, et al. [cited by applicant]
U.S. Appl. No. 18/027,916, filed Mar. 22, 2023, Martin Haller, et al. [cited by applicant]
U.S. Appl. No. 18/027,914, filed Mar. 22, 2023, Martin Haller, et al. [cited by applicant]
U.S. Appl. No. 18/027,912, filed Mar. 22, 2023, Martin Haller, et al. [cited by applicant]
International Preliminary Report from PCT Application No. PCT/US2021/051614, dated Sep. 22, 2021, pp. 1-19. [cited by applicant]
Anil K. Jain, et al., “50 years of biometric research: Accomplishments, challenges, and opportunities,” Pattern Recognition Letters, Elsevier, Amsterdam, NL, vol. 79, Jan. 12, 2016, pp. 80-105. [cited by applicant]
Olade, et al., “A Review of Multimodal Facial Biometric Authentication Methods in Mobile Devices and Their Application in Head Mounted Displays,” 2018 IEEE SmartWorld, SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI, 2018, pp… [cited by applicant]
“Information technology—Biometric data Interchange,” Part 6: Iris image data, BS ISO/IEC 19794-6:2011+A1:2015, Incorporating corrigenda Apr. 2012 and Sep. 2013, bsi, pp. 1-78. [cited by applicant]
Galbally Javier et al.,. “A review of iris anti=spoofing,” 2016 4th International Conference on Biometrics and Forensics (IWBF), IEEE, Mar. 3, 2016, XP032890563, pp. 1-6. [cited by applicant]
Venkatesh, et al., “A New Multi-spectral Iris Acquisition Sensor for Biometric Verification and Presentation Attack Detection,” IEEE, Jan. 2019, pp. 47-54. [cited by applicant]
Examiner Report from Application No. 21801270.6-1218, dated May 14, 2025, pp. 1-8. [cited by applicant]
Lee Eui Chul, “Fake iris detection method using Purkinje images based on gaze position,” Optical Engineering, vol. 47, No. 61, Jun. 2008, XP093275560, p. 067204. [cited by applicant]
U.S. Appl. No. 19/449,142, filed Jan. 14, 2026, Sanjiv Das, et al. [cited by applicant]
U.S. Appl. No. 19/449,270, filed Jan. 14, 2026, Sanjiv Das, et al. [cited by applicant]
Tao, et al., “Biometric Authentication System on Mobile Personal Devices,” In IEEE Transactions Instrumentation and Measurement, vol. 59, No. 4, pp. 763-773, Apr. 2010. [cited by applicant]
Okabe, et al., “Smart Device-Based Multimodal Biometric Authentication with the Function for Environment Recognition,” 2015 Third International Symposium on Computing and Networking (CANDAR), 2015, pp. 495-498. [cited by applicant]
Hemery, et al., “Face Authentication for Banking,” First International Conference on Advances in Computer-Human Interaction, 2008, pp. 137-142. [cited by applicant]
Smith-Creasey, et al., “Context Awareness for Improved Continuous Face Authetication on Mobile Devices,” 2018 IEEE 16th Intl Conf on Dependable, Autonomic and secure Computing, pp. 644-652, 2018. [cited by applicant]
U.S. Appl. No. 19/349,858, filed Oct. 3, 2025, Martin Haller, et al. [cited by applicant]