IP Library Granted Patent US 12,288,414
Granted Patent B2
US 12,288,414 · App. 17/577,553 · Granted Apr 29, 2025

Systems and methods for performing fingerprint based user authentication using imagery captured using mobile devices

Inventors: Asem Othman (Worcester, MA); Richard Tyson (Oxfordshire, GB); Aryana Tavanai (Oxford, GB); Yiqun Xue (Quincy, MA); Andrew Simpson (London, GB)
Assignee: VERIDIUM IP LIMITED
G06V40/1371G06F21/32G06V10/32G06V30/194G06V40/11G06V40/13G06V40/1353G06V40/1359G06V40/1388G06V40/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,288,414
App. No.
17/577,553
Granted
Apr 29, 2025
Kind
B2
Abstract

Technologies are presented herein in support of a system and method for performing fingerprint recognition. Embodiments of the present invention concern a system and method for capturing a user's biometric features and generating an identifier characterizing the user's biometric features using a mobile device such as a smartphone. The biometric identifier is generated using imagery captured of a plurality of fingers of a user for the purposes of authenticating/identifying the user according to the captured biometrics and determining the user's liveness. The present disclosure also describes additional techniques for preventing erroneous authentication caused by spoofing. In some examples, the anti-spoofing techniques may include capturing one or more images of a user's fingers and analyzing the captured images for indications of liveness.

Claims (45)

1. A method for performing fingerprint recognition, the method comprising:

capturing, by a mobile device having a camera, a storage medium, instructions stored on the storage medium, and a processor configured by executing the instructions, images depicting a plurality of fingers of a subject;

detecting, with the processor using a finger detection algorithm, one or more fingers depicted in at least one of the images;

assessing, with the processor using a convolutional neural network (CNN), a quality score of the at least one image;

processing the at least one image, with the processor using a segmentation algorithm, to identify a respective fingertip segment for at least one finger depicted therein based on the assessed quality score;

measuring, with the processor from the identified respective fingertip segment, one or more features of the at least one finger, wherein the measured feature is a frequency of fingerprint ridges, and wherein the at least one image is scaled based on the frequency of fingerprint ridges and one or more of a prescribed reference frequency and a target resolution;

scaling, the at least one image based on the measured feature; and

storing a biometric identifier, wherein the biometric identifier comprises at least a portion of the scaled image depicting the respective fingertip segment.

2. The method of claim 1 , wherein measuring the frequency comprises: calculating the inverse of a median distance between ridges in a local area of the respective fingertip segment, wherein the distance is measured in a direction perpendicular to a ridge orientation of the at least one finger.

3. The method of claim 1 , wherein the target resolution is a resolution of images stored in a legacy database and that are used for matching against the biometric identifier.

4. The method of claim 1 , wherein the images are captured at a distance from the fingers and thereby depict three-dimensional fingers, and wherein the step of generating the biometric identifier further comprises:

converting an image of the respective fingertip segment to an image of a two-dimensional surface of the fingertip segment using a convolutional neural network (CNN), wherein the CNN is trained on pairs of images that each include a first image of a respective finger captured at a distance using a camera and a second image depicting a two-dimensional surface of the respective finger captured using a contact scanner.

5. A method for performing fingerprint recognition, the method comprising:

capturing, by a mobile device having a camera, a storage medium, instructions stored on the storage medium, and a processor configured by executing the instructions, images depicting a plurality of fingers of a subject;

detecting, with the processor using, a finger detection algorithm, the plurality of fingers depicted in one or more of the images;

processing, with the processor, at least one image using a segmentation algorithm, to identify a respective fingertip segment for one or more fingers depicted in the at least one image;

assessing, with the processor using a convolutional neural network (CNN), a quality score of the at least one image;

extracting, with the processor from the identified respective fingertip segment for the one or more fingers, features of the one or more fingers based on the assessed quality score;

detecting, using a CNN trained to detect minutia points, minutia points within at least one image of the captured images;

determining respective quality scores for the detected minutia points;

selecting a subset of the minutia points for inclusion in the biometric identifier according to the respective quality scores;

generating, with the processor, a biometric identifier including the extracted features and the subset of minutia points; and

storing the generated biometric identifier in the storage medium.

6. The method of claim 5 , wherein the minutia points including fingerprint features.

7. The method of claim 6 , further comprising:

enhancing, the respective fingertip segment.

8. The method of claim 6 , further comprising:

scaling, the respective fingertip segment.

9. A method for performing fingerprint recognition, the method comprising:

capturing, by a mobile device having a camera, a storage medium, instructions stored on the storage medium, and a processor configured by executing the instructions, images depicting a plurality of fingers of a subject;

detecting, with the processor using, a finger detection algorithm, the plurality of fingers depicted in one or more of the images;

processing, with the processor, at least one image using a segmentation algorithm, to identify a respective fingertip segment for one or more fingers depicted in the at least one image;

assessing, with the processor using a convolutional neural network (CNN), a quality score of the at least one image;

extracting, with the processor from the identified respective fingertip segment for the one or more fingers, features of the one or more fingers based on the assessed quality score;

detecting a set of minutia points using a minutia extraction algorithm;

calculating respective quality scores for the minutia points in the set; and

selecting at least a subset of the minutia points in the set for inclusion in the biometric identifier according to the respective quality scores;

generating, with the processor, a biometric identifier including the extracted features and the subset of minutia points; and

storing the generated biometric identifier in the storage medium.

10. The method of claim 9 , wherein the minutia extraction algorithm is a CNN.

11. The method of claim 10 , further comprising:

detecting a background of the at least one image and the respective fingertip segment; and

filtering detected minutia points that correspond to the background from the set.

12. The method of claim 9 , further comprising:

using a plurality of the captured images captured at different respective angles to generate a 3D map of one or more fingers among the plurality of fingers.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 28, 2022
From: OTHMAN, ASEM; TYSON, RICHARD; TAVANAI, ARYANA; XUE, YIQUN; SIMPSON, ANDREW
To: VERIDIUM IP LIMITED
Reel/Frame 059409/0766 →
Continuity (9)
Continuation 16440640 · Jun 13, 2019
Continuation In Part 15835527 · Dec 8, 2017
Continuation In Part 15704561 · Sep 14, 2017
Continuation In Part 15212335 · Jul 18, 2016
Continuation 14988833 · Jan 6, 2016
Continuation In Part 14819639 · Aug 6, 2015
Provisional Application 62112961 · Feb 6, 2015
Provisional Application 62431629 · Dec 8, 2016
Related Publication 20220215686A1 · Jul 7, 2022
References Cited (161)
US 5335288A · Faulkner · 1994 [cited by applicant]
US 5659626A · Ort et al. · 1997 [cited by applicant]
US 6292576B1 · Brownlee · 2001 [cited by applicant]
US 7936907B2 · Maurer · 2011 [cited by applicant]
US 8548206B2 · Sahin et al. · 2013 [cited by applicant]
US 8649575B2 · Nagar et al. · 2014 [cited by applicant]
US 8724861B1 · Sun · 2014 [cited by applicant]
US 9003196B2 · Hoyos et al. · 2015 [cited by applicant]
US 9112858B2 · Sahin et al. · 2015 [cited by applicant]
US 9202100B2 · Wolfer et al. · 2015 [cited by applicant]
US 9202102B1 · Sahin et al. · 2015 [cited by applicant]
US 9208492B2 · Hoyos · 2015 [cited by applicant]
US 9361507B1 · Hoyos · 2016 [cited by examiner]
US 9424458B1 · Mather · 2016 [cited by examiner]
US 9741107B2 · Xu · 2017 [cited by examiner]
US 9785323B2 · Lu et al. · 2017 [cited by applicant]
US 9785823B2 · Mather et al. · 2017 [cited by applicant]
US 10339362B2 · Othman · 2019 [cited by examiner]
US 10521643B2 · Mather · 2019 [cited by examiner]
US 11188734B2 · Mather et al. · 2021 [cited by applicant]
US 11263432B2 · Othman · 2022 [cited by examiner]
US 20020133725A1 · Roy et al. · 2002 [cited by applicant]
US 20040144841A1 · Tsukamoto et al. · 2004 [cited by applicant]
US 20050249388A1 · Linares · 2005 [cited by examiner]
US 20060153432A1 · Lo et al. · 2006 [cited by applicant]
US 20070041622A1 · Salva Calcagno · 2007 [cited by applicant]
US 20070047783A1 · Kim et al. · 2007 [cited by applicant]
US 20070112701A1 · Chellapilla et al. · 2007 [cited by applicant]
US 20080031495A1 · Saijo et al. · 2008 [cited by applicant]
US 20080056539A1 · Sweeney et al. · 2008 [cited by applicant]
US 20080063244A1 · Tanaka et al. · 2008 [cited by applicant]
US 20090190857A1 · Epshtein · 2009 [cited by applicant]
US 20090245585A1 · Manabe et al. · 2009 [cited by applicant]
US 20090267893A1 · Kato et al. · 2009 [cited by applicant]
US 20090310831A1 · Zhang et al. · 2009 [cited by applicant]
US 20100165090A1 · Sweeney et al. · 2010 [cited by applicant]
US 20100312763A1 · Peirce · 2010 [cited by applicant]
US 20100322300A1 · Li et al. · 2010 [cited by applicant]
US 20110150303A1 · Dinerstein et al. · 2011 [cited by applicant]
US 20110196240A1 · Mitani · 2011 [cited by applicant]
US 20120086794A1 · Burcham et al. · 2012 [cited by applicant]
US 20120250947A1 · Abramovich et al. · 2012 [cited by applicant]
US 20120281890A1 · Kamakura et al. · 2012 [cited by applicant]
US 20120326963A1 · Minnen · 2012 [cited by applicant]
US 20130067545A1 · Hanes · 2013 [cited by applicant]
US 20130101186A1 · Walch et al. · 2013 [cited by applicant]
US 20130177221A1 · Yalla et al. · 2013 [cited by applicant]
US 20130259331A1 · Maurer et al. · 2013 [cited by applicant]
US 20130287271A1 · Harper · 2013 [cited by applicant]
US 20130322705A1 · Wong · 2013 [cited by applicant]
US 20140018519A1 · Jacobs · 2014 [cited by applicant]
US 20140233810A1 · Spence et al. · 2014 [cited by applicant]
US 20140294262A1 · Schuckers et al. · 2014 [cited by applicant]
US 20140321718A1 · Wabgaonkar et al. · 2014 [cited by applicant]
US 20140337221A1 · Hoyos · 2014 [cited by applicant]
US 20140337930A1 · Hoyos et al. · 2014 [cited by applicant]
US 20150055836A1 · Moleki et al. · 2015 [cited by applicant]
US 20150074615A1 · Han et al. · 2015 [cited by applicant]
US 20150098013A1 · Gray et al. · 2015 [cited by applicant]
US 20150146943A1 · Son et al. · 2015 [cited by applicant]
US 20150195288A1 · Hoyos et al. · 2015 [cited by applicant]
US 20150199819A1 · Suzuki · 2015 [cited by applicant]
US 20150220771A1 · Thiebot et al. · 2015 [cited by applicant]
US 20150227774A1 · Batch et al. · 2015 [cited by applicant]
US 20150309663A1 · Seo et al. · 2015 [cited by applicant]
US 20150310302A1 · Xie et al. · 2015 [cited by applicant]
US 20160140381A1 · Weiner et al. · 2016 [cited by applicant]
US 20160180142A1 · Riddle et al. · 2016 [cited by applicant]
US 20160217313A1 · Cuti · 2016 [cited by examiner]
US 20160188950A1 · Liu et al. · 2016 [cited by applicant]
US 20160321496A1 · Mather · 2016 [cited by applicant]
US 20180157899A1 · Xu · 2018 [cited by examiner]
US 20180350106A1 · Kasilya Sudarsan · 2018 [cited by examiner]
CN 1937955 · 2007 [cited by applicant]
CN 201207195Y · 2009 [cited by applicant]
CN 101773394 · 2010 [cited by applicant]
CN 101825986 · 2010 [cited by applicant]
CN 102045162 · 2011 [cited by applicant]
CN 102782724A · 2012 [cited by applicant]
CN 103827895 · 2014 [cited by applicant]
CN 103988503A · 2014 [cited by applicant]
CN 104036273 · 2014 [cited by applicant]
CN 104123537A · 2014 [cited by applicant]
CN 104580143A · 2015 [cited by applicant]
CN 105354560A · 2016 [cited by applicant]
EP 1139301 · 2001 [cited by applicant]
EP 2011057 · 2010 [cited by applicant]
EP 2615532 · 2013 [cited by applicant]
EP 2639731 · 2013 [cited by applicant]
GB 2403038A · 2004 [cited by applicant]
GB 2458007A · 2009 [cited by applicant]
JP H07506917A · 1995 [cited by applicant]
JP 2001273498 · 2001 [cited by applicant]
JP 2003178306 · 2003 [cited by applicant]
JP 200672553A · 2006 [cited by applicant]
JP 2006172258 · 2006 [cited by applicant]
JP 2006215975A · 2006 [cited by applicant]
JP 2009199611 · 2009 [cited by applicant]
JP 2009265809 · 2009 [cited by applicant]
JP 2009544108A · 2009 [cited by applicant]
JP 2011048602A · 2011 [cited by applicant]
JP 2011182386A · 2011 [cited by applicant]
JP 201288918A · 2012 [cited by applicant]
JP 2013143082 · 2013 [cited by applicant]
JP 2013196046 · 2013 [cited by applicant]
JP 2014225145A · 2014 [cited by applicant]
KR 100795187 · 2008 [cited by applicant]
KR 1020150003501 · 2015 [cited by applicant]
KR 101626837B1 · 2016 [cited by applicant]
WO 2012170349A2 · 2012 [cited by applicant]
WO 2013135299A1 · 2013 [cited by applicant]
WO 2014143070 · 2014 [cited by applicant]
WO 2016153209A1 · 2016 [cited by applicant]
Carasso, Alternative Methods of Latent Fingerprint Enhancement and Metrics for Comparing Them, nist. (Year: 2013). [cited by examiner]
Office Action in corresponding Indian Application No. 202148029966, mailed May 24, 2022, 7 pages. [cited by applicant]
Office Action in corresponding Korean Application No. 10-2017-7025019, mailed Dec. 20, 2022, 44 pages. [cited by applicant]
International Search Report and Written Opinion for International Patent Application No. PCT/US2017/065240 mailed on Apr. 25, 2018. 9 pages. [cited by applicant]
No Author. “Onyx Mobile Touchless Fingerprint Biometrics.” Diamon Fortress Technologies. Birmingham Alabama. Oct. 16, 2013. http://c1940652.r52.cf0.rackcdn.com/52582f90f45279397a00070e/Onyx_White_Paper. pdf 9 pages. [cited by applicant]
Elferink, W.O. “Fingerprint Acquisition With a Smartphone Camera.” Aug. 25, 2014 (Aug. 25, 2014), pp. 1-56, XP55288674. Retrieved from the Internet at http://essay.utwente.nl/65923/1/BachelorverslagWoutOudeElferink-Norm… [cited by applicant]
International Search Report and Written Opinion for International Patent Application No. PCT/IB2016/000569 mailed on Oct. 4, 2016. 23 pages. [cited by applicant]
Jiang et al. “Exploring global and local decisions for multimodal biometrics verification.” IEEE Transactions on Signal Processing 52.10 (2004): 3059-3072. [cited by applicant]
Communications Pursuant to Article 94(3) EPC in corresponding European Application No. 16723153.9, mailed May 2, 2019. 8 pages. [cited by applicant]
Nogueira Rodrigo Frassetto et al: “Fingerprint Liveness Detection Using Convolutional Neural Networks”, IEEE Transactions on Information Forensics and Security, IEEE, Piscataway, NJ, US, vol. 11, No. 6, Jun. 1, 2016 (Ju… [cited by applicant]
Marasco Emanuela et al: “Robust and interoperable fingerprint spoof detection via convolutional neural networks”, 2016 IEEE Symposium on Technologies for Homeland Security (HST), IEEE, May 10, 2016 (May 10, 2016), pp. 1… [cited by applicant]
Communications Pursuant to Article 94(3) EPC in corresponding European Application No. 17877953.4, mailed Jun. 18, 2020, 14 pages. [cited by applicant]
Wout Oude Elferink: “Fingerprint Acquisition With a Smartphone Camera”, Aug. 25, 2014 (Aug. 25, 2014), pp. 1-58, XP5528867 4, Retrieved from the Internet: URL:http://essay.utwente.nl/65923/1 /Bachelor verslag Wout Oude … [cited by applicant]
Chinese Office Action dated Jun. 24, 2020 corresponding to Chinese Patent Application No. 201680020217.6; 28 pages. [cited by applicant]
Chinese Search Report dated Jun. 16, 2020 corresponding to Chinese Patent Application No. 201680020217.6; 6 pages. [cited by applicant]
Shi Jin-shuai, Zhang You-guang, Lin Guo-jun, “Based on statistical characteristics of gray-scale fingerprint image segmentation algorithm,” Control & Automation, Issue 29, 2010, pp. 197-198. Accessed at: http://caod.ori… [cited by applicant]
Baig Asim et al: “Cascaded multimodal biometric recognition framework”,IET Biometrics, IEEE, Michael 1 Faraday House, Six Hills Way, Stevenage, Herts, SGI 2AY, UK, vol. 3, No. I, Mar. 1, 2014 (Mar. 1, 2014), pp. 16-28. [cited by applicant]
Vatsa Mayank et al: “Context Switching Algorithm for Selective Multibiometric Fusion”, Dec. 16, 2009 (Dec. 16, 2009), International Conference on Financial Cryptography and Data Security; Lecture Notes in Computer Scien… [cited by applicant]
Communications Pursuant to Article 94(3) EPC in corresponding European Application No. 16723153.9, mailed Jun. 5, 2020, 12 pages. [cited by applicant]
First Examination Report dated Dec. 31, 2020 corresponding to Indian Patent Application No. 201847027653; 8 pages. [cited by applicant]
Chris Stein; Claudia Nickel; Christoph Busch, Fingerphoto recognition with smartphone cameras,2012 BIOSIG—Proceedings of the International Conference of Biometrics Special Interest Group (BIOSIG), IEEE,2012, URL; https:… [cited by applicant]
Japanese Patent Office Action Report dated Mar. 22, 2021 corresponding to Japanese Patent Application No. 2020-007244; 65 pages. [cited by applicant]
Butler, J. G., “Automated Fingertip Detection”, Jun. 2012, [online], [retrieved from the Internet on Apr. 7, 2021], <URL: https://scholarsarchive.byu.edu/cgi/viewcontent.cgi?article=4163&context=etd>. [cited by applicant]
Australian Patent Office Action Report dated Apr. 9, 2021 corresponding to Australian Patent Application No. 2016214084; 6 pages. [cited by applicant]
HUANG Fei, et al., “Research on Hand Bone Extraction Method based on ASM”, Computer Engineering and Applications, pp. 164-168 and 219. 2016. [cited by applicant]
Man Jiangyue, “The Realization Principle and Prospect Analysis of Biometric Recognition Technology” Intelligent Building & City Information, pp. 64-66. 2014. [cited by applicant]
European Examination Report dated Oct. 4, 2021 corresponding to European Patent Application No. 16 723 153.9. 6 pages. [cited by applicant]
Office Action dated Nov. 18, 2021 corresponding to Japanese Patent Application No. 2019-529997, 15 pages. [cited by applicant]
Arunalatha G, M. Ezhilarasan, “Fingerprint Liveness detection using probabality density function”, 2016 International Conference on Communication and Signal Processing (ICCSP), IEEE, April 6. 2016. pp. 199 to 204, https… [cited by applicant]
European Examination Report dated Nov. 19, 2021 corresponding to European Patent Application No. 2017370720, 3 pages. [cited by applicant]
Office Action in corresponding Korean Application No. 10-2019-7015805, mailed Oct. 11, 2022, 15 pages. [cited by applicant]
Examination Report in corresponding European Application No. 17877953.4, mailed Oct. 17, 2022, 7 pages. [cited by applicant]
Extended European Search Report in corresponding European Patent Application No. 17877953.4, mailed Sep. 21, 2020, 11 pages. [cited by applicant]
Office Action in corresponding Canadian Patent Application No. 2,976,049, mailed Feb. 15, 2022, 4 pages. [cited by applicant]
Machine translation of Office Action in Mexican Patent Application No. MX/E/2022/086923, mailed Dec. 8, 2023, translation generated Mar. 7, 2023. [cited by applicant]
Office Action in corresponding Japanese Application No. 2019-529997, mailed Aug. 23, 2022, 12 pages. [cited by applicant]
Office Action in corresponding Chinese Application No. 201780082847.0, mailed Nov. 24, 2022, 34 pages. [cited by applicant]
European Examination Report in corresponding European Patent Application No. 16723153.9, mailed Jun. 23, 2023, 7 pages. [cited by applicant]
Machine translation of Office Action in corresponding Brazilian Application No. BR112019011205-7, mailed Apr. 11, 2023, 8 pages. [cited by applicant]
Office Action in corresponding Korean Patent Application No. 10-2023-7025660, mailed Apr. 22, 2024, 16 pages. English translation attached. [cited by applicant]
Australian Examination Report in corresponding Australian Patent Application No. 2017370720, mailed Nov. 19, 2021, 3 pages. [cited by applicant]
Chinese Patent Office Action dated May 21, 2021 corresponding to Chinese Patent Application No. 201680020217.6; 30 pages. [cited by applicant]
Imamverdiyev et al. “Biometric cryptosystem based on discretized fingerprint texture descriptors.” Expert Systems with Applications 40.5 (2013): 1888-1901. 14 pages. [cited by applicant]
Jiang et al. “A Direct Fingerprint Minutiae Extraction Approach Based on Convolutional Neural Networks.” 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, 2016, pp. 571-578. 8 pages. [cited by applicant]
Kerdvibulvech et al. “Markerless guitarist fingertip detection using a bayesian classifier and a template matching for supporting guitarists.” Proceedings of the 10th ACM/IEEE Virtual Reality International Conference, V… [cited by applicant]
Ong et al. “A boosted classifier tree for hand shape detection.” Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings.. IEEE, 2004. 6 pages. [cited by applicant]
Viola et al. “Robust Real-time Objection Detection, Second International Workshop on Statistical and Computational Theories of Vision—Modeling, Learning, Computing, and Sampling.” Jul. 13, 2001. 25 pages. [cited by applicant]
Peng “ China Doctoral Dissertation Full-text Database: Information Science and Technology Series, Issue 01, 2015” Research on Key Issues of Multi-modal Biometric Verification Based on Finger, Jan. 15, 2015 (137 pages in… [cited by applicant]