IP Library › Granted Patent US 12,263,694
Granted Patent B2
US 12,263,694 · App. 18/193,732 · Granted Apr 1, 2025

Evaluating three-dimensional security features on document images

Inventors: Stuart Wells (Saratoga, CA); Attila Balogh (Vienna, AT); Anshuman Vikram Singh (Vienna, AT); Thomas Krump (Buchkirken, AT); Daryl Huff (Saratoga, CA)
Assignee: Jumio Corporation
B42D25/333G06T5/50G06T7/13G06V30/414G06T2207/20221
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,263,694
App. No.
18/193,732
Granted
Apr 1, 2025
Kind
B2
Abstract

The disclosure includes a system and method for obtaining at least one image of a document under test, wherein valid instances of the document under test include at least one three-dimensional security feature; obtaining a first image snippet from the document under test, the first image snippet corresponding to at least a first portion of the three-dimensional security feature present in the valid instances of the document; deriving first dimensional data associated with the first image snippet; analyzing the first dimensional data to determine one or more of: whether the first dimensional data is consistent with a presence of the three-dimensional security feature, and whether the first dimensional data is consistent with second dimensional data; and modifying a likelihood that the document under test is accepted as valid, or rejected as invalid, based on the analysis of the first dimensional data.

Claims (64)

1. A method comprising:

obtaining, using one or more processors, at least one image of a document, wherein valid instances of the document include at least a first three-dimensional security feature;

obtaining, using the one or more processors, a first image snippet from the document, the first image snippet corresponding to at least a first portion of the first three-dimensional security feature present in the valid instances of the document;

deriving, using the one or more processors, first dimensional data associated with the first image snippet;

analyzing, using the one or more processors, the first dimensional data to determine one or more of:

whether the first dimensional data is consistent with a presence of the first three-dimensional security feature, and

whether the first dimensional data is consistent with second dimensional data, wherein the second dimensional data corresponds to one of: a second portion of the first three-dimensional security feature in the valid instances of the document and at least a portion of a second three-dimensional security feature in the valid instances of the document; and

modifying, using one or more processors, a likelihood that the document under test is accepted as valid, or rejected as invalid, based on the analysis of the first dimensional data.

2. The method of claim 1 , the method further comprising:

obtaining a second image snippet from the image of the document; and

deriving the second dimensional data associated with the second image snippet,

wherein the first three-dimensional security feature includes a set of tactile text,

wherein the first dimensional data and first image snippet correspond to the first portion of the first three-dimensional security feature by representing a first character in the set of tactile text, and

wherein the second dimensional data and second image snippet correspond to the second portion of the first three-dimensional security feature by representing a second character in the set of tactile text.

3. The method of claim 1 , wherein deriving the first dimensional data includes:

determining, for each pixel in a set of pixels within the first image snippet, an intensity;

assigning a first subset of pixels that satisfy an intensity threshold to a first color;

assigning a second subset of pixels that fail to satisfy the intensity threshold to a second color; and

generating, as the first dimensional data, a first and second color version of the first image snippet.

4. The method of claim 1 , wherein determining whether the first dimensional data is consistent with the presence of the first three-dimensional security feature includes:

determining whether the first dimensional data is indicative of specular reflection; and

responsive to determining that the first dimensional data is not indicative of specular reflection, determining that first dimensional data is inconsistent with the presence of the first three-dimensional security feature.

5. The method of claim 1 , wherein deriving the first dimensional data includes:

identifying color variation within the first image snippet, wherein a color variation in portions of the first three-dimensional security feature known to have a common color is determined to be indicative of three-dimensionality.

6. The method of claim 1 , wherein deriving the first dimensional data includes:

applying nearest neighbor analysis to the first image snippet.

7. The method of claim 1 , wherein deriving the first dimensional data includes:

generating a depth map from stereoscopic images of the document, the depth map indicating whether portion of the document associated with the first image snippet is three dimensional.

8. The method of claim 1 , wherein determining whether the first dimensional data is consistent with the second dimensional data is based at least in part on determining a common light source.

9. The method of claim 8 , wherein determining the common light source is based on one or more of specular reflection and ray tracing.

10. The method of claim 1 , wherein determining whether the first dimensional data is consistent with second dimensional data is determined subsequent to determining that the first dimensional data is consistent with the presence of the first three-dimensional security feature.

11. A system comprising:

a processor; and

a memory, the memory storing instructions that, when executed by the processor, cause the system to:

obtain at least one image of a document, wherein valid instances of the document include at least a first three-dimensional security feature;

obtain a first image snippet from the document-under test, the first image snippet corresponding to at least a first portion of the first three-dimensional security feature present in the valid instances of the document;

derive first dimensional data associated with the first image snippet;

analyze the first dimensional data to determine one or more of:

whether the first dimensional data is consistent with a presence of the first three-dimensional security feature, and

whether the first dimensional data is consistent with second dimensional data, wherein the second dimensional data corresponds to one of: a second portion of the first three-dimensional security feature in the valid instances of the document and at least a portion of a second first three-dimensional security feature in the valid instances of the document; and

modify a likelihood that the document is accepted as valid, or rejected as invalid, based on the analysis of the first dimensional data.

12. The system of claim 11 , includes instructions that, when executed, cause the system to:

obtain a second image snippet from the image of the document; and

derive the second dimensional data associated with the second image snippet,

wherein the first three-dimensional security feature includes a set of tactile text,

wherein the first dimensional data and first image snippet correspond to the first portion of the first three-dimensional security feature by representing a first character in the set of tactile text, and

wherein the second dimensional data and second image snippet correspond to the second portion of the first three-dimensional security feature by representing a second character in the set of tactile text.

13. The system of claim 11 , wherein deriving the first dimensional data includes:

determining, for each pixel in a set of pixels within the first image snippet, an intensity;

assigning a first subset of pixels that satisfy an intensity threshold to a first color;

assigning a second subset of pixels that fail to satisfy the intensity threshold to a second color; and

generating, as the first dimensional data, a first and second color version of the first image snippet.

14. The system of claim 11 , wherein determining whether the first dimensional data is consistent with the presence of the first three-dimensional security feature includes:

determining whether the first dimensional data is indicative of specular reflection; and

responsive to determining that the first dimensional data is not indicative of specular reflection, determining that first dimensional data is inconsistent with the presence of the first three-dimensional security feature.

15. The system of claim 11 , wherein deriving the first dimensional data includes:

identifying color variation within the first image snippet, wherein a color variation in portions of the first three-dimensional security feature known to have a common color is determined to be indicative of three-dimensionality.

16. The system of claim 11 , wherein deriving the first dimensional data includes:

applying nearest neighbor analysis to the first image snippet.

17. The system of claim 11 , wherein deriving the first dimensional data includes:

generating a depth map from stereoscopic images of the document, the depth map indicating whether portion of the document associated with the first image snippet is three dimensional.

18. The system of claim 11 , wherein determining whether the first dimensional data is consistent with the second dimensional data is based at least in part on determining a common light source.

19. The system of claim 18 , wherein determining the common light source is based on one or more of specular reflection and ray tracing.

20. The system of claim 11 , wherein determining whether the first dimensional data is consistent with second dimensional data is determined subsequent to determining that the first dimensional data is consistent with the presence of the first three-dimensional security feature.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 6, 2023
From: WELLS, STUART; BALOGH, ATTILA; SINGH, ANSHUMAN VIKRAM; KRUMP, THOMAS; HUFF, DARYL
To: JUMIO CORPORATION
Reel/Frame 063245/0691 →
Continuity (4)
Continuation In Part 18148542 · Dec 30, 2022
Continuation In Part 18148544 · Dec 30, 2022
Continuation In Part 18148536 · Dec 30, 2022
Related Publication 20240217256A1 · Jul 4, 2024
References Cited (109)
US 4897814A · Clark · 1990 [cited by applicant]
US 5544255A · Smithies et al. · 1996 [cited by applicant]
US 5668897A · Stolfo · 1997 [cited by applicant]
US 5748780A · Stolfo · 1998 [cited by applicant]
US 6363162B1 · Moed et al. · 2002 [cited by applicant]
US 7040539B1 · Stover · 2006 [cited by applicant]
US 7831531B1 · Baluja et al. · 2010 [cited by applicant]
US 8326761B1 · Hecht et al. · 2012 [cited by applicant]
US 8352494B1 · Badoiu · 2013 [cited by applicant]
US 8886648B1 · Procopio et al. · 2014 [cited by applicant]
US 9135517B1 · Adams · 2015 [cited by applicant]
US 11416562B1 · Gruhl et al. · 2022 [cited by applicant]
US 11593439B1 · Avadhani et al. · 2023 [cited by applicant]
US 11715102B2 · Edwards et al. · 2023 [cited by applicant]
US 20030172066A1 · Cooper et al. · 2003 [cited by applicant]
US 20060041506A1 · Mason et al. · 2006 [cited by applicant]
US 20060124726A1 · Kotovich et al. · 2006 [cited by applicant]
US 20070078846A1 · Gulli et al. · 2007 [cited by applicant]
US 20070086628A1 · Fuchs et al. · 2007 [cited by applicant]
US 20070116328A1 · Sablak · 2007 [cited by applicant]
US 20080065630A1 · Luo et al. · 2008 [cited by applicant]
US 20080149713A1 · Brundage · 2008 [cited by applicant]
US 20090261158A1 · Lawson · 2009 [cited by applicant]
US 20100027896A1 · Geva et al. · 2010 [cited by applicant]
US 20110057040A1 · Jones · 2011 [cited by applicant]
US 20140002872A1 · Cook · 2014 [cited by applicant]
US 20150100590A1 · Robinson et al. · 2015 [cited by applicant]
US 20150242592A1 · Weiss et al. · 2015 [cited by applicant]
US 20160098399A1 · Casperson · 2016 [cited by applicant]
US 20160210450A1 · Su · 2016 [cited by applicant]
US 20170161375A1 · Stoica et al. · 2017 [cited by applicant]
US 20170193285A1 · Negi · 2017 [cited by applicant]
US 20170337449A1 · Hamada et al. · 2017 [cited by applicant]
US 20180075090A1 · Knight et al. · 2018 [cited by applicant]
US 20180186164A1 · Wu · 2018 [cited by applicant]
US 20180204113A1 · Galron et al. · 2018 [cited by applicant]
US 20180293461A1 · Le et al. · 2018 [cited by applicant]
US 20180300296A1 · Ziraknejad et al. · 2018 [cited by applicant]
US 20180373859A1 · Ganong · 2018 [cited by applicant]
US 20190035431A1 · Attorre · 2019 [cited by applicant]
US 20190272549A1 · Mossoba et al. · 2019 [cited by applicant]
US 20190278986A1 · Nepomniachtchi · 2019 [cited by examiner]
US 20200184201A1 · Kaehler · 2020 [cited by applicant]
US 20200304650A1 · Roach · 2020 [cited by applicant]
US 20210124919A1 · Balakrishnan · 2021 [cited by applicant]
US 20210320801A1 · Wyss · 2021 [cited by applicant]
US 20220028086A1 · Woodard et al. · 2022 [cited by applicant]
US 20220058660A1 · Ivanov · 2022 [cited by applicant]
US 20220385880A1 · Nims · 2022 [cited by applicant]
US 20230017185A1 · Cheong et al. · 2023 [cited by applicant]
US 20230113148A1 · Zlotnick · 2023 [cited by examiner]
US 20230129350A1 · Bryan et al. · 2023 [cited by applicant]
MY 192715A · 2022 [cited by applicant]
Shokat, Sana, et al. “Analysis and Evaluation of Braille to Text Conversion Methods.” Mobile Information Systems, vol. 2020, 2020, pp. 1-14. [cited by applicant]
Shokat, Sana, et al. “Characterization of English Braille Patterns Using Automated Tools and RICA Based Feature Extraction Methods.” Sensors (Basel, Switzerland), vol. 22, No. 5, 2022, p. 1836. [cited by applicant]
PCT International Search Report and Written Opinion; Application No. PCT/US23/79821 Jumio Corporation, International filing date of Nov. 15, 2023, date of mailing Apr. 4, 2024, 10 pages. [cited by applicant]
“7 Best Face Morph Apps 2024 (Morph Two Faces Together).” ContentMaverickscom, Content Mavericks, 2024, contentmavericks.com/best-face-morph-app/. 2024 Web. 25 pgs. 2024. [cited by applicant]
“Levenshtein Distance.” Wikipedia, Wikimedia Foundation, Dec. 23, 2023, en.wikipedia.org/wiki/Levenshtein_distance. Web. 6 pgs. 2023. [cited by applicant]
“New Method Detects Deepfake Videos with up to 99% Accuracy.” News, May 3, 2022, news.ucr.edu/articles/2022/05/03/new-method-detects-deepfake-videos-99-accuracy. Web. 3 pgs. 2022. [cited by applicant]
Bassil, Youssef, and Mohammad Alwani. “Context-Sensitive Spelling Correction Using Google Web 1T 5-Gram Information.” Computer and Information Science (Toronto), vol. 5, No. 3, 2012, p. 37. 2012. [cited by applicant]
Bassil, Youssef, and Mohammad Alwani. “OCR Post-Processing Error Correction Algorithm Using Google Online Spelling Suggestion.” ArXiv.org, 2012, pp. arXiv.org, 2012. 2012. [cited by applicant]
Benalcazar, Daniel, et al. “Synthetic ID Card Image Generation for Improving Presentation Attack Detection.” IEEE Transactions on Information Forensics and Security, vol. 18, 2023, pp. 1814-1824. 2023. [cited by applicant]
Casado, Constantino Alvarez, et al. “Real-time Face Alignment: Evaluation Methods, Training Strategies and Implementation Optimization.” Journal of Real-time Image Processing 18.6 (2021): 2239-2267. Web. 2021. [cited by applicant]
Deepswap.AI, “Deepswap—Best Face and Video Edit Tools Online.” DeepSwap AI, www.deepswap.ai/?utm_source=bing&cp_id=441169896&msclkid=eb76125c4d531cf9797e77325a694067. Accessed Dec. 2023. Web. 11 pgs. 2023. [cited by applicant]
DevCodeF1 Editors, Using LLMS for OCR text proofreading: A guide for software developers. Dev Code F1. May 26, 2023, <https://devcodef1.com/news/1007434/IIms-for-ocr-text-proofreading> Web. 3 pgs. 2023. [cited by applicant]
Faceshape. “Face Morphing Simulator—Morph Two Faces Together.” Face Morphing Simulator—Morph Two Faces Together, www.faceshape.com/face-morph. (2022) Web. 2 pgs. 2022. [cited by applicant]
Fadilpaić, Sead. “Deepfake Fraud Attacks Are Hitting More and More Businesses.” TechRadar, TechRadar Pro, Feb. 24, 2023, www.techradar.com/news/deepfake-fraud-attacks-are-hitting-more-and-more-businesses. Accessed Jan. … [cited by applicant]
GitHub, Use Llama2 to Improve the Accuracy of Tesseract OCR. GitHub. (n.d.). https://github.com/ Dicklesworthstone/llama2_aided_tesseract> Web. Last updated Aug. 2, 2023. 3 pgs. 2023. [cited by applicant]
Help Net Security. “Detecting Face Morphing: A Simple Guide to Countering Complex Identity Fraud.” Help Net Security, Mar. 16, 2023, www.helpnetsecurity.com/2023/03/20/facial-morphing-technology/. Web. 5 pgs. 2023. [cited by applicant]
Hu, Yifei, et al. “Misspelling Correction with Pre-Trained Contextual Language Model.” ArXiv.org, 2021, pp. arXiv.org, 2021. 2021. [cited by applicant]
Kaspersky, Secure Futures Editors. “What Does the Rise of Deepfakes Means for the Future of Cybersecurity?” Daily English USA Usakasperskycomblog, Kaspersky Secure Futures Editors, usa.kaspersky.com/blog/secure-futures-… [cited by applicant]
Kramer, Robin S. S., et al. “Face Morphing Attacks: Investigating Detection with Humans and Computers.” Cognitive Research: Principles and Implications, vol. 4, No. 1, 2019, p. 28. 2019. [cited by applicant]
Kumar, Varun. “14 Best Deepfake Apps and Tools in 2024.” RankRed, Jan. 1, 2024, www.rankred.com/best-deepfake-apps-tools/. Web. 22 pgs. 2024. [cited by applicant]
Lee, Jung-Hun, et al. “Deep Learning-Based Context-Sensitive Spelling Typing Error Correction.” IEEE Access, vol. 8, 2020, pp. 152565-152578. 2020. [cited by applicant]
Liu, Zhaoxiang, et al. “Facial Pose Estimation by Deep Learning from Label Distributions.” (2019). Web. IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), 9 pgs. 2019. [cited by applicant]
Lowphansirkul, Lalita, et al. “WangchanBERTa: Pretraining Transformer-Based Thai Language Models.” ArXiv.org, 2021, pp. arXiv.org, 2021. 2021. [cited by applicant]
Luo, Yinhui et al. “A Review of Homography Estimation: Advances and Challenges.” Electronics 2023 pp. 4977-4977. https://www.mdpi.com/2079-9292/12/24/4977 2023. [cited by applicant]
Lyu, Siwei, “Detecting ‘deepfake’ Videos in the Blink of an Eye.” The Conversation, Sep. 15, 2022, theconversation.com/detecting-deepfake-videos-in-the-blink-of-an-eye-101072. Web. 4 pgs. 2022. [cited by applicant]
Mazaheri, Ghazal et al. “2022 Ieee/Cvf Winter Conference on Applications of Computer Vision (Wacv).” Detection and Localization of Facial Expression Manipulations IEEE 2022 pp. 2773-2783 2022. [cited by applicant]
Nightingale, Sophie J., et al. “Perceptual and Computational Detection of Face Morphing.” Journal of Vision (Charlottesville, Va.), vol. 21, No. 3, 2021, p. 4. 2021. [cited by applicant]
Picsi.Ai, “Create Realistic Face Morphs” Picsi.Ai, InsightFace, 2023, www.picsi.ai/. Web. 8 pgs. 2023. [cited by applicant]
Rosebrock, A. (Nov. 16, 2021). OCR passports with opencv and Tesseract. PylmageSearch. <https://pyimagesearch.com/2021/12/01/ocr-passports-with-opencv-and-tesseract/> Web. 24 pgs. 2021. [cited by applicant]
Sadeghzadeh, Arezoo, et al., “Pose-invariant face recognition based on matching the occlusion free regions aligned by 3D generic model.” IET Computer Vision, Aug. 2020, vol. 14, Issue 5, pp. 177-287. 2020. [cited by applicant]
Schulz, Daniel, et al., “Identify Documents Image Quality Assessment.” European Association for Signal Processing, Proceedings 2022 pp. 1017-1021. 2022. [cited by applicant]
Seibold, C., et al., “Detection of Face Morphing Attacks by Deep Learning.” Digital Forensics and Watermarking. IWDW (2017). Lecture Notes in Computer Science(), vol. 10431. Springer, Cham. https://doi.org/10.1007/978-3… [cited by applicant]
Tesseract—OCR, Improving the Quality of the Output Github.com, Tesseract User Manual v. 5.x, 9 pgs, updated Dec. 5, 2023. Web. 2023. [cited by applicant]
Times, Global. “Tencent Launches Large Language Model ‘Hunyuan’ amid Global Generative Ai Frenzy.” Global Times, Sep. 7, 2023, www.globaltimes.cn/page/202309/1297761.shtml. 6 pgs. 2023. [cited by applicant]
Youtube, YouTube, “Morphing Identity: A real-time face morphing system to transforming face identity.” May 15, 2021, Cybernetic Humanity Studio. https://youtu.be/ahKxwaJ3k_U?si=kp4G4wFa8IAsGNLY 2021. [cited by applicant]
Zhao, Jian et al. “2018 Ieee/Cvf Conference on Computer Vision and Pattern Recognition.” Towards Pose Invariant Face Recognition in the Wild IEEE 2018 pp. 2207-2216. 2018. [cited by applicant]
PCT International Search Report and Written Opinion; Application No. PCT/US23/79511 Jumio Corporation, International filing date of Nov. 13, 2023, date of mailing Mar. 4, 2024, 17 pages. [cited by applicant]
Aslam, Asra, et al., May 15, 2019, Depth-Map Generation using Pixel Matching in Stereoscopic Pair of Images, https://arxiv.org/pdf/1902.03471.pdf, 5 pgs. [cited by applicant]
Elsayed, M et al. “A New Method for Full Reference Image Blur Measure.” International Journal of Simulation: Systems Science and Technology V19 N1 (Feb. 1, 2018): 7.1-7.5 2018 https://doi.org/10.5013/IJSSST.a.19.01.7. [cited by applicant]
Canada Passport Phot Security Features, Canada.ca, Government of Canada (Dec. 20, 2022) https://www.canada.ca/en/immigration-refugees-citizenship/services/canadian-passports/photos.html#/photo, webpage 11 pgs. [cited by applicant]
Edge Detection Using OpenCV, LearnOpenCV.com, (2023) https://learnopencv.com/edge-detection-using-opencv/, webpage. 10 pgs. [cited by applicant]
Passport Phot Specifications., Government of Canada (2015) https://www.canada.ca/content/dam/ircc/migration/ircc/english/pdf/pub/pass-photo-spec-eng.pdf, 5 pgs. [cited by applicant]
Rosebrock, Adrian, “OpenCV Fast Fourier Transform (FFT) for blur detection in images and video streams.” PylmageSearch.com, Jun. 15, 2020, webpage, 21 pgs. https://pyimagesearch.com/2020/06/15/opencv-fast-fourier-transf… [cited by applicant]
“Rosebrock, Adrian, Blur Detection with OpenCV” PylmageSearch.com, Sep. 7, 2015, https://pyimagesearch.com/2015/09/07/blur-detection-with-opencv/, webpage, 13 pgs. [cited by applicant]
Zhang, Erhu, et al. “Forgery Detection for Perforated Number in Security Document by Analysing the Perforated Holes.” The Imaging Science Journal 65.1 (2017): 40-48. Web. [cited by applicant]
Di Guardo, Fabrizio. “Facemask—a Real-Time Face Morphing Tool.” Medium, Level Up Coding, May 2, 2023, levelup.gitconnected.com/facemask-a-real-time-face-morphing-tool-5b343591a237. May 2, 2023. [cited by applicant]
“Face Landmark Detection Guide | Google Ai Edge | Google AI for Developers.” Google, ai.google.dev/edge/mediapipe/solutions/vision/face_landmarker. [cited by applicant]
“Image Segmentation Guide | Google Ai Edge | Google AI for Developers.” Google, ai.google.dev/edge/mediapipe/solutions/vision/image_segmenter. [cited by applicant]
Kirillov, Alexander et al. “Segment Anything.” 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 1, 2023, doi:10.1109/iccv51070.2023.00371. Oct. 1, 2023. [cited by applicant]
“Large Language and Vision Assistant.” LLaVA, llava-vl.github.io/. [cited by applicant]
Liu, Haotian, et al. “Visual Instruction Tuning.” 37th Conference on Neural Information Processing Systems, Dec. 11, 2023, arxiv.org/pdf/2304.08485. Dec. 11, 2023. [cited by applicant]
“Llava 1.6—a Hugging Face Space by Liuhaotian.” LLaVA 1.6—a Hugging Face Space by Liuhaotian, huggingface.co/spaces/liuhaotian/LLaVA-1.6. [cited by applicant]
Wang, Xin, et al. “Attribute-Aware Implicit Modality Alignment for Text Attribute Person Search.” Arxiv. Org, Jun. 6, 2024. Jun. 6, 2024. [cited by applicant]
Wang, Zhe, et al. “Attribute-guided Transformer for Robust Person Re-identification.” IET Computer Vision, vol. 17, No. 8, Jun. 23, 2023, pp. 977-992, doi:10.1049/cvi2.12215. Jun. 23, 2023. [cited by applicant]
PCT International Search Report and Written Opinion; Application No. PCT/US23/86219 Jumio Corporation, International filing date of Dec. 28, 2023, date of mailing May 23, 2024, 4 pages. [cited by applicant]
PCT International Search Report and Written Opinion; Application No. PCT/US2024/048525 Jumio Corporation, International filing date of Sep. 26, 2024, date of mailing Nov. 27, 2024, 10 pages. 2024. [cited by applicant]