IP Library Granted Patent US 12,288,413
Granted Patent B2
US 12,288,413 · App. 17/326,450 · Granted Apr 29, 2025

Camera tampering detection

Inventors: Ali Hassani (Ann Arbor, MI); Hafiz Malik (Canton, MI); Jonathan Diedrich (Carleton, MI)
Assignee: Ford Global Technologies, LLC
G06V40/10G05D1/0055H04N25/63H04N25/67
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,288,413
App. No.
17/326,450
Granted
Apr 29, 2025
Kind
B2
Abstract

A computer, including a processor and a memory, the memory including instructions to be executed by the processor to acquire one or more images from a camera and determine first camera noise values based on the one or more images by determining reactions of camera photo receptors to light. The instructions can include further instructions to compare the first camera noise values with second camera noise values determined based on previously acquired images from the camera and output a tamper determination for the camera based on whether the first camera noise values match, within a tolerance value, the second camera noise values determined based on the previously acquired images from the camera.

Claims (32)

1. A computer, comprising:

a processor; and

a memory, the memory including instructions executable by the processor to:

acquire one or more images from a camera;

determine first camera noise values based on the one or more images by determining reactions of camera photo receptors to light;

compare the first camera noise values with second camera noise values determined based on previously acquired images from the camera by determining covariance matrices based on the one or more images of the camera and the previously acquired images from the camera, diagonalizing the covariance matrices to determine eigenvectors, and determining first and second camera noise values that include interaction noise principal components and temporal noise components of the eigenvectors;

output a tamper determination for the camera based on whether the first camera mean and variance values of the noise values match, within a tolerance value, the second camera mean and variance values of the noise values determined based on the previously acquired images from the camera; and

when the tamper d ion indicates that the first camera noise values do not match the second camera noise values, within a tolerance value, update the second camera noise values based on successful non-biometric access.

2. The computer of claim 1 , the instructions including further instructions to, when the tamper determination indicates that the first camera noise values match the second camera noise values, within a tolerance value, output images from the camera.

3. The computer of claim 1 , the instructions including further instructions to, when the tamper determination indicates that the first camera noise values do not match the second camera noise values, within a tolerance value, do not output images from the camera.

4. The computer of claim 1 , the instructions including further instructions to, when the tamper determination indicates that the first camera noise values match the second camera noise values, output one or more images from the camera for use in one or more of biometric authentication and autonomous vehicle operation.

5. The computer of claim 1 , wherein the one or more images are acquired by periodically defeating flat field compensation in the camera.

6. The computer of claim 1 , the instructions including further instructions to determine the first camera noise values by determining photo-response non-uniformity values {circumflex over (K)} by the equation {circumflex over (K)}=Σ i=1 m W i I i /Σ i=1 m (I i ) 2 where I i is the i-th image acquired by the camera and W i is the noise residual of the i-th image.

7. The computer of claim 1 , the instructions including further instructions to determine the first camera noise values by determining camera dark current.

8. The computer of claim 1 , wherein determining camera tampering includes determining lens aberration by comparing geometry of an object in the one or more images to geometry of the object in one or more previously acquired images from the camera.

9. The computer of claim 1 , wherein determining the first camera noise values includes determining one or more components of a principal component analysis that decomposes image data into an n-dimensional component space, wherein each component corresponds to a line that best fits the image data including noise.

10. The computer of claim 9 , wherein determining the first camera noise values includes determining components of the principal component analysis that correspond to interaction noise corresponding to image artifact introduced into the image by acquisition of the one or more images.

11. The computer of claim 9 , wherein determining the first camera noise values includes determining components of the principal component analysis that correspond to temporal noise corresponding to random image noise related to electronic noise and photon quantum noise included in the one or more images.

12. The computer of claim 1 , wherein the second camera noise is updated based on determining a successful non-biometric access, wherein non-biometric access techniques include a passcode, a key, or a fob.

13. A method comprising:

acquiring one or more images from a camera;

determining first camera noise values based on the one or more images by determining reactions of camera photo receptors to light;

comparing the first camera noise values with second camera noise values determined based on previously acquired images from the camera by determining covariance matrices based on the one or more images of the camera and the previously acquired images from the camera, diagonalizing the covariance matrices to determine eigenvectors, and determining first and second camera noise values that include interaction noise principal components and temporal noise components of the eigenvectors;

output a tamper determination for the camera based on whether the first camera mean and variance values of the noise values match, within a tolerance value, the second camera mean and variance values of the noise values determined based on the previously acquired images from the camera; and

when the tamp nation indicates that the first camera noise values do not match the second camera noise values, within a tolerance value, update the second camera noise values based on successful non-biometric access.

14. The method of claim 13 , further comprising, when the tamper determination indicates that the first camera noise values match the second camera noise values, within a tolerance value, output images from the camera.

15. The method of claim 13 , further comprising, when the tamper determination indicates that the first camera noise values do not match the second camera noise values, within a tolerance value, do not output images from the camera.

16. The method of claim 13 , further comprising, when the tamper determination indicates that the first camera noise values match the second camera noise values, output one or more images from the camera for use in one or more of biometric authentication and autonomous vehicle operation.

17. The method of claim 13 , wherein the one or more images are acquired by periodically defeating flat field compensation in the camera.

18. The method of claim 13 , further comprising determining the first camera noise values by determining photo-response non-uniformity values {circumflex over (K)} by the equation {circumflex over (K)}=Σ i=1 m W i I i /Σ i=1 m (I i ) 2 where I i is the i-th image acquired by the camera and W i is the noise residual of the i-th image.

19. The method of claim 13 , further comprising determining the first camera noise values by determining camera dark current.

20. The method of claim 13 , wherein determining camera tampering includes determining lens aberration by comparing geometry of an object in the one or more images to geometry of the object in one or more previously acquired images from the camera.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 21, 2021
From: MALIK, HAFIZ
To: THE REGENTS OF THE UNIVERSITY OF MICHIGAN
Reel/Frame 056310/0252 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 21, 2021
From: THE REGENTS OF THE UNIVERSITY OF MICHIGAN
To: FORD GLOBAL TECHNOLOGIES, LLC
Reel/Frame 056310/0501 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 21, 2021
From: HASSANI, ALI; DIEDRICH, JONATHAN
To: FORD GLOBAL TECHNOLOGIES, LLC
Reel/Frame 056310/0540 →
Continuity (1)
Related Publication 20220374641A1 · Nov 24, 2022
References Cited (145)
US 6092192A · Kanevsky et al. · 2000 [cited by applicant]
US 6593963B1 · Safai · 2003 [cited by examiner]
US 6879247B2 · Shimomura et al. · 2005 [cited by applicant]
US 6920236B2 · Prokoski · 2005 [cited by applicant]
US 7602947B1 · Lemelson et al. · 2009 [cited by applicant]
US 8160293B1 · Fridrich · 2012 [cited by examiner]
US 8300890B1 · Gaikwad · 2012 [cited by examiner]
US 8374404B2 · Williams et al. · 2013 [cited by applicant]
US 8718335B2 · Mason et al. · 2014 [cited by applicant]
US 8725330B2 · Failing · 2014 [cited by applicant]
US 8988219B2 · McCloskey · 2015 [cited by examiner]
US 9886640B1 · Chen et al. · 2018 [cited by applicant]
US 10169671B2 · Matsimanis et al. · 2019 [cited by applicant]
US 10452894B2 · Zhang et al. · 2019 [cited by applicant]
US 10769415B1 · Mostafa et al. · 2020 [cited by applicant]
US 10868984B1 · Charlton · 2020 [cited by examiner]
US 10990660B2 · Magli · 2021 [cited by examiner]
US 11030470B2 · Han et al. · 2021 [cited by applicant]
US 11113510B1 · Mostafa et al. · 2021 [cited by applicant]
US 11178363B1 · Qian et al. · 2021 [cited by applicant]
US 11373449B1 · Genner · 2022 [cited by applicant]
US 11481883B2 · Wildermuth et al. · 2022 [cited by applicant]
US 20010038717A1 · Brown · 2001 [cited by examiner]
US 20020136435A1 · Prokoski · 2002 [cited by applicant]
US 20060001921A1 · Bailey · 2006 [cited by examiner]
US 20060102843A1 · Bazakos et al. · 2006 [cited by applicant]
US 20060104488A1 · Bazakos et al. · 2006 [cited by applicant]
US 20060261931A1 · Cheng · 2006 [cited by applicant]
US 20080059027A1 · Farmer et al. · 2008 [cited by applicant]
US 20090232367A1 · Shinzaki · 2009 [cited by examiner]
US 20090257671A1 · Fridrich · 2009 [cited by examiner]
US 20100102366A1 · Lee et al. · 2010 [cited by applicant]
US 20100102961A1 · McCloskey · 2010 [cited by examiner]
US 20100134250A1 · Chung et al. · 2010 [cited by applicant]
US 20100141770A1 · Gomi · 2010 [cited by applicant]
US 20100201374A1 · Vasilyev et al. · 2010 [cited by applicant]
US 20100208951A1 · Williams et al. · 2010 [cited by applicant]
US 20120154581A1 · Wang · 2012 [cited by examiner]
US 20120176810A1 · Galbraith et al. · 2012 [cited by applicant]
US 20120189225A1 · Li · 2012 [cited by applicant]
US 20120230536A1 · Fridrich · 2012 [cited by examiner]
US 20120243797A1 · Di Venuto Dayer · 2012 [cited by examiner]
US 20130024123A1 · Ochs et al. · 2013 [cited by applicant]
US 20130077958A1 · Xu et al. · 2013 [cited by applicant]
US 20130311001A1 · Hampiholi · 2013 [cited by applicant]
US 20130342702A1 · Zhang et al. · 2013 [cited by applicant]
US 20130342703A1 · Lin · 2013 [cited by applicant]
US 20140002675A1 · Duparre · 2014 [cited by examiner]
US 20140162598A1 · Villa-Real · 2014 [cited by examiner]
US 20140278529A1 · Matos · 2014 [cited by examiner]
US 20140294262A1 · Schuckers et al. · 2014 [cited by applicant]
US 20140307929A1 · Nechyba et al. · 2014 [cited by applicant]
US 20150110418A1 · Bayram · 2015 [cited by examiner]
US 20150281599A1 · Slaby et al. · 2015 [cited by applicant]
US 20150304612A1 · Richards et al. · 2015 [cited by applicant]
US 20160019420A1 · Feng et al. · 2016 [cited by applicant]
US 20160019421A1 · Feng et al. · 2016 [cited by applicant]
US 20160048736A1 · Chu et al. · 2016 [cited by applicant]
US 20160086018A1 · Lemoff · 2016 [cited by applicant]
US 20160206216A1 · Kirenko · 2016 [cited by applicant]
US 20170091550A1 · Feng et al. · 2017 [cited by applicant]
US 20170104916A1 · Mueller · 2017 [cited by applicant]
US 20170242134A1 · Matsuoka · 2017 [cited by examiner]
US 20180039845A1 · Chen et al. · 2018 [cited by applicant]
US 20180048474A1 · Landrock · 2018 [cited by examiner]
US 20180240216A1 · Dirik · 2018 [cited by examiner]
US 20190057268A1 · Burge et al. · 2019 [cited by applicant]
US 20190057502A1 · Wang et al. · 2019 [cited by applicant]
US 20190102873A1 · Wang et al. · 2019 [cited by applicant]
US 20190228248A1 · Han et al. · 2019 [cited by applicant]
US 20200053297A1 · Tokizaki et al. · 2020 [cited by applicant]
US 20200134342A1 · Parupati et al. · 2020 [cited by applicant]
US 20200218772A1 · Zhang et al. · 2020 [cited by applicant]
US 20200250448A1 · Joshi et al. · 2020 [cited by applicant]
US 20200273157A1 · Shao · 2020 [cited by examiner]
US 20200334450A1 · Shen et al. · 2020 [cited by applicant]
US 20200342245A1 · Lubin et al. · 2020 [cited by applicant]
US 20200402223A1 · Ghosh · 2020 [cited by examiner]
US 20210049391A1 · Zou et al. · 2021 [cited by applicant]
US 20210074138A1 · Micko et al. · 2021 [cited by applicant]
US 20210166045A1 · Kwak et al. · 2021 [cited by applicant]
US 20210168347A1 · Margolin et al. · 2021 [cited by applicant]
US 20210192340A1 · Stonehouse · 2021 [cited by examiner]
US 20210241014A1 · Choiniere · 2021 [cited by applicant]
US 20210256281A1 · Henson et al. · 2021 [cited by applicant]
US 20210264183A1 · Wolf et al. · 2021 [cited by applicant]
US 20220012511A1 · Rowe · 2022 [cited by applicant]
US 20220084223A1 · Norris · 2022 [cited by examiner]
US 20220284228A1 · Li et al. · 2022 [cited by applicant]
US 20220398820A1 · Abd-Almageed et al. · 2022 [cited by applicant]
US 20230005240A1 · Chen et al. · 2023 [cited by applicant]
US 20230056564A1 · Han · 2023 [cited by examiner]
US 20230091865A1 · Chen · 2023 [cited by examiner]
US 20240013572A1 · Yang et al. · 2024 [cited by applicant]
CN 202887210U · 2013 [cited by applicant]
CN 204795370U · 2015 [cited by applicant]
CN 108319986A · 2018 [cited by applicant]
CN 110287672A · 2019 [cited by applicant]
CN 111611977A · 2020 [cited by applicant]
CN 111738065A · 2020 [cited by applicant]
WO WO2014078696A2 · 2014 [cited by examiner]
WO 2020048140A1 · 2020 [cited by applicant]
WO 2021157790A1 · 2021 [cited by applicant]
WO 2022134754A1 · 2022 [cited by applicant]
Fridrich, Jessica. “Digital image forensics.” IEEE Signal Processing Magazine 26.2 (2009): 26-37. [cited by examiner]
Kurosawa, Kenji, et al. “Case studies and further improvements on source camera identification.” Media Watermarking, Security, and Forensics 2013. vol. 8665. SPIE, 2013. [cited by examiner]
San Choi, Kai, Edmund Y. Lam, and Kenneth KY Wong. “Automatic source camera identification using the intrinsic lens radial distortion.” Optics express 14.24 (2006): 11551-11565. [cited by examiner]
López-Alonso, José Manuel, and Javier Alda. “Spatial-temporal characterization of noise in web cameras.” Photonics, Devices, and Systems II. vol. 5036. SPIE, 2003. [cited by examiner]
Akshatha et al. (Akshatha, K. R., et al. “Digital camera identification using PRNU: A feature based approach.” Digital investigation 19 (2016): 69-77. [cited by examiner]
Chen, Mo, Jessica Fridrich, Miroslav Goljan, and Jan Lukas. “Determining image origin and integrity using sensor noise.” IEEE Transactions on information forensics and security 3, No. 1 (2008): 74-90. [cited by examiner]
Hu et al., “Heterogeneous Face Recognition: Recent Advances in Infrared-to-Visible Matching”, 2017 IEEE 12th International Conference on Automatic Face & Gesture Recognition, DOI 10.1109/FG.2017.126. [cited by applicant]
Non-Final Office Action dated Oct. 13, 2023 re U.S. Appl. No. 17/326,511, filed May 21, 2021. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 17/326,511, filed May 21, 2021, as issued by the USPTO Mar. 9, 2023. [cited by applicant]
Final Office Action for U.S. Appl. No. 117/326,490, filed May 21, 2021, as issued by the USPTO May 22, 2023. [cited by applicant]
Friedrich, “Digital Image Forensics”, DOI:10.1109/MSP.2008.931078, IEEE, 2009 (Year: 2009). [cited by applicant]
Non-Final Office Action dated Jun. 21, 2023 re U.S. Appl. No. 17/326,581, filed May 21, 2021. [cited by applicant]
Final Office Action dated Jun. 27, 2023 re U.S. Appl. No. 17/326,511, filed May 21, 2021. [cited by applicant]
Final Office Action for U.S. Appl. No. 17/326,511, filed May 21, 2021, as issued by the USPTO Dec. 29, 2022. [cited by applicant]
Maser et al., “PRNY-based Detection of Finger Vein Presentation Attacks”, DOI:10.1109/IWBF.2019.8739203, 7th IAPR/IEEE International Workshop on Biometrics and Forensics—IWBF2019. [cited by applicant]
Final Office Action dated Sep. 27, 2023, U.S. Appl. No. 17/326,581, filed May 21, 2021. [cited by applicant]
Chen et al. “Determining Image origin and Integrity Using Sensor Noise”, IEEE Transactions on Information Forensics and Security, vol. 3, No. 1, Mar. 2008. [cited by applicant]
Debiasi et al. “PRNU-based Detection of Morphed Face Images”, 2018 International Workshop on Biometrics and Forensics (IWBF), 2018. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 17/326,460 as issued by the USPTO Oct. 19, 2022. [cited by applicant]
Hermosilla et al., “A comparative study of thermal face recognition methods in unconstrained environments”, Pattern Recognition 45 (2012) 2445-2459, doi: 10.1016/j.patcog.2012.01.001. [cited by applicant]
Kakkirala et al., “Thermal Infrared Face Recognition: A Review”, 2017 UKSims-AMSS 19th International Conference on Modelling & Simulation, doi: 10.1109/UKSim.2017.38. [cited by applicant]
Kkristo et al., “An Overview of Thermal Face Recognition Methods”, MIPRO 2018, May 21-25, 2018, Opatija Croatia. [cited by applicant]
Sanchez-Sanchez et al., “Conventional Neural Network Approach for Multispectral Facial Presentation Attack Detection in Automated Border Control Systems”, Entropy 2020, 22, 1296, www.mdpi.com/journal/entropy, doi: 10.33… [cited by applicant]
Spinoulas et al., “Multispectral Biometrics System Framework: Application to Presentation Attach Detection”, arXiv:2006.07489v1, Jun. 12, 2020. [cited by applicant]
Steiner et al., “Reliable Face Anti-Spoofing Using Multispectral SWIR Imaging”, 978-1-5090-1869-7/16, copyright 2016 IEEE. [cited by applicant]
Non-Final Office Action dated Aug. 12, 2022 re U.S. Appl. No. 17/326,511, filed May 21, 2021. [cited by applicant]
Non Office Action for U.S. Appl. No. 17/326,490, filed May 21, 2021, as issued by the USPTO Jan. 23, 2023. [cited by applicant]
Nagendran, N. et al., “Security and Safety With Facial Recognition Feature for Next Generation Automobiles,” International Journal of Recent Technology and Engineering (IJRTE), vol. 7 Issue-4S, Nov. 2018, 6 pages. [cited by applicant]
Scherhag, U. et al., “Detection of Face Morphing Attacks based on PRNU Analysis,” IEEE Transactions on Biometrics, Behavior, and Identity Science (T-BIOM), Oct. 2019, 16 pages. [cited by applicant]
Mohd Norzali, et al., “Internal State Measurement From Facial Stereo Thermal and Visible Sensors Through SVM Classification,” ARPN Journal of Engineering and Applied Sciences, Jan. 2015, 8 pages. [cited by applicant]
Kim, et al. “A Motion and Similarity-Based Fake Detection Method for Biometric Face Recognition Systems,” IEEE Transactions on Consumer Electronics, vol. 57, No. 2, May 2011, 7 pages. [cited by applicant]
Angelopoulou, E., “The Reflectance Spectrum of Human Skin,” University of Pennsylvania, Scholarly Commons, Department of Computer & Information Science, Dec. 20, 1999, 16 pages. [cited by applicant]
Anderson, R, et al., “The Optics of Human Skin,” The Journal of Investigative Dermatology, vol. 77, No. J, 1981, 7 pages. [cited by applicant]
Cooksey, C., et al., “Reflectance Measurements of Human Skin,” National Institute of Standards and Technology, Oct. 27, 2020, 3 pages. [cited by applicant]
Barnes, P. et al., “Spectral Reflectance,” U.S. Department of Commerce, Technology Administration, National Institute of Standards and Technology, Mar. 1998, 164 pages. [cited by applicant]
Mhou, K. et al., “Face Spoof Detection Using Light Reflection in Moderate to Low Lighting,” 2nd Asia-Pacific Conference on Intelligent Robot Systems, 2017, 6 pages. [cited by applicant]
Jacquez, J. et al., “Spectral Reflectance of Human Skin in the Region 0.7-2.6,” http://jap.physiology.org/, Sep. 13, 2016, 3 pages. [cited by applicant]
Cooksey, C. et al., “A collection and statistical analysis of skin reflectance signatures for inherent variability over the 250 nm to 2500 nm spectral range,” National Institute of Standards and Technology, Jun. 4, 2014… [cited by applicant]
Akshatha, K.R., et al., “Digital camera identification using PRNU: A feature based approach,” Science Direct, Digital Investigation, Nov. 3, 2016, 9 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 17/326,581, filed May 21, 2021, as issued by the USPTO Jan. 19, 2024. [cited by applicant]
Notice of Allowance dated Feb. 6, 2024 re U.S. Appl. No. 17/326,511, filed May 21, 2021. [cited by applicant]