Methods and systems for authentication of a physical document
Described herein are computerized methods and systems for authentication of a physical document. An image capture device coupled to a mobile device captures images of a physical document, during which the mobile device adjusts operational parameters of the image capture device, resulting in a sequence of images captured using different capture settings. The mobile device partitions the sequence of images into subsets of images, wherein each subset comprises images with a similar alignment of the physical document and captured using the same capture settings. The mobile device processes the subsets of images to identify a region of interest in each image. The mobile device generates a representation of the identified region of interest using the processed images, generates an authentication score for the document using the representation of the identified region of interest, and determines whether the physical document is authentic based upon the authentication score.
1. A system for authentication of a physical document, the system comprising a mobile computing device coupled to an image capture device, the mobile computing device configured to:
capture, using the image capture device, images of a physical document in a scene, during which the mobile computing device adjusts one or more operational parameters of the image capture device, resulting in a sequence of images captured using different capture settings;
partition the sequence of images into one or more subsets of images, wherein each subset comprises images with a similar alignment of the physical document and captured using the same capture settings;
process the subsets of images to identify a region of interest in each image, the region of interest corresponding to one or more visual features of the physical document;
generate a representation of the identified region of interest using the processed images;
generate an authentication score for the document using the representation of the identified region of interest; and
determine whether the physical document is authentic based upon the authentication score.
2. The system of claim 1 , wherein the one or more operational parameters comprise one or more of shutter speed, ISO speed, gain and offset, aperture, flash intensity, flash duration, or light balance.
3. The system of claim 1 , wherein the physical document is stationary during capture of the images by the mobile computing device.
4. The system of claim 1 , wherein the physical document remains in a stationary position relative to the image capture device during capture of the images by the mobile computing device.
5. The system of claim 1 , wherein prior to capturing a first image of the physical document in the scene, the mobile computing device generates baseline operational parameters of the image capture device based upon one or more imaging conditions associated with the physical document.
6. The system of claim 5 , wherein adjusting one or more operational parameters of the image capture device comprises adjusting the baseline operational parameters between capturing each image in the sequence of images.
7. The system of claim 6 , wherein adjusting the baseline operational parameters between capturing each image comprises receiving operational parameters used for the previous image and using the received operational parameters to adjust the baseline operational parameters as part of a dynamic feedback loop.
8. The system of claim 1 , wherein the mobile computing device preprocesses the sequence of images received from the image capture device prior to partitioning the sequence of images.
9. The system of claim 8 , wherein preprocessing the sequence of images comprises one or more of: assessing video quality metrics for the entire sequence of images, detecting a location of the physical document in each image of the sequence of images, and determining one or more quality metrics for each image in the sequence of images.
10. The system of claim 9 , wherein the video quality metrics comprise a length of the sequence of images, a frames-per-second (FPS) value associated with the sequence of images, and an image resolution associated with the sequence of images.
11. The system of claim 9 , wherein the one or more quality metrics comprise (i) global image quality metrics including one or more of: glare, blur, white balance, or sensor noise characteristics, (ii) local image quality metrics including one or more of: blur, sharpness, text region confidence, character confidence, or edge detection, or (iii) both the global image quality metrics and the local image quality metrics.
12. The system of claim 11 , wherein the sensor noise characteristics comprise one or more of: blooming, readout noise, or custom calibration variations.
13. The system of claim 1 , wherein processing the selected images to identify a region of interest in each image comprises normalizing an image signal of each image.
14. The system of claim 13 , wherein normalizing an image signal of each image comprises amplifying the image signal associated with a region of interest on the physical document and reducing the image signal associated with a background of the physical document.
15. The system of claim 1 , wherein generating a representation of the identified region of interest comprises executing one or more of a robust principal component analysis (PCA) algorithm or a learned alternative mapping on the image to reconstruct the region of interest.
16. The system of claim 1 , wherein generating an authentication score for the document using the reconstructed region of interest comprises executing one or more machine learning classification models using one or more features of the reconstructed region of interest as input to generate a classification value for the document.
17. The system of claim 16 , wherein the one or more machine learning classification models comprise one or more of: deep learning models, Random Forest algorithms, Support Vector Machines, neural networks, or ensembles thereof.
18. The system of claim 16 , wherein the classification value comprises at least one of a probability that the document is authentic, a confidence score metric that indicates whether the document is authentic, or a similarity metric that indicates whether the document is authentic.
19. The system of claim 17 , wherein at least one of the one or more machine learning classification models is a convolutional neural network.
20. The system of claim 17 , wherein the one or more machine learning classification models is an ensemble classifier comprised of a plurality of convolutional neural networks.
21. The system of claim 16 , wherein one or more interpretable methods are used to validate the classification value.
22. The system of claim 21 , wherein the one or more interpretable methods comprise occlusion of at least a portion of the document, perturbation of at least a portion of the document, or analysis of a heatmap of at least a portion of the document.
23. The system of claim 22 , wherein an output of the one or more interpretable methods comprises an identification of the reconstructed region of interest that represents proof of the document being genuine or fraudulent.
24. The system of claim 16 , wherein the one or more machine learning classification models are trained using a plurality of genuine documents, a plurality of fraudulent documents, or both.
25. The system of claim 24 , wherein the classification value generated by the one or more machine learning classification models is a measure of similarity between one or more of the plurality of genuine documents, one or more of the plurality of fraudulent documents, or both.
26. The system of claim 1 , wherein the images of the physical document comprise one of: images of a front side of the physical document or images of a back side of the physical document.
27. A computerized method of authentication of a physical document, the method comprising:
capturing, using an image capture device coupled to a mobile computing device, images of a physical document in a scene, during which the mobile computing device adjusts one or more operational parameters of the image capture device, resulting in a sequence of images captured using different capture settings;
partitioning, by the mobile computing device, the sequence of images into one or more subsets of images, wherein each subset comprises images with a similar alignment of the physical document and captured using the same capture settings;
processing, by the mobile computing device, the subsets of images to identify a region of interest in each image, the region of interest corresponding to one or more visual features of the physical document;
generating, by the mobile computing device, a representation of the identified region of interest using the processed images;
generating, by the mobile computing device, an authentication score for the document using the representation of the identified region of interest; and
determining, by the mobile computing device, whether the physical document is authentic based upon the authentication score.
28. The method of claim 27 , wherein the one or more operational parameters comprise one or more of shutter speed, ISO speed, gain and offset, aperture, flash intensity, flash duration, or light balance.
29. The method of claim 27 , wherein the physical document is stationary during capture of the images by the mobile computing device.
30. The method of claim 27 , wherein the physical document remains in a stationary position relative to the image capture device during capture of the images by the mobile computing device.
31. The method of claim 27 , wherein prior to capturing a first image of the physical document in the scene, the mobile computing device generates baseline operational parameters of the image capture device based upon one or more imaging conditions associated with the physical document.
32. The method of claim 31 , wherein adjusting one or more operational parameters of the image capture device comprises adjusting the baseline operational parameters between capturing each image in the sequence of images.
33. The method of claim 32 , wherein adjusting the baseline operational parameters between capturing each image comprises receiving operational parameters used for the previous image and using the received operational parameters to adjust the baseline operational parameters as part of a dynamic feedback loop.
34. The method of claim 27 , wherein the mobile computing device preprocesses the sequence of images received from the image capture device prior to partitioning the sequence of images.
35. The method of claim 34 , wherein preprocessing the sequence of images comprises one or more of: assessing video quality metrics for the entire sequence of images, detecting a location of the physical document in each image of the sequence of images, and determining one or more quality metrics for each image in the sequence of images.
36. The method of claim 35 , wherein the video quality metrics comprise a length of the sequence of images, a frames-per-second (FPS) value associated with the sequence of images, and an image resolution associated with the sequence of images.
37. The method of claim 35 , wherein the one or more quality metrics comprise (i) global image quality metrics including one or more of: glare, blur, white balance, or sensor noise characteristics, (ii) local image quality metrics including one or more of: blur, sharpness, text region confidence, character confidence, or edge detection, or (iii) both the global image quality metrics and the local image quality metrics.
38. The method of claim 37 , wherein the sensor noise characteristics comprise one or more of: blooming, readout noise, or custom calibration variations.
39. The method of claim 27 , wherein processing the selected images to identify a region of interest in each image comprises normalizing an image signal of each image.
40. The method of claim 39 , wherein normalizing an image signal of each image comprises amplifying the image signal associated with a region of interest on the physical document and reducing the image signal associated with a background of the physical document.
41. The method of claim 27 , wherein generating a representation of the identified region of interest comprises executing one or more of a robust principal component analysis (PCA) algorithm or a learned alternative mapping on the image to reconstruct the region of interest.
42. The method of claim 27 , wherein generating an authentication score for the document using the reconstructed region of interest comprises executing one or more machine learning classification models using one or more features of the reconstructed region of interest as input to generate a classification value for the document.
43. The method of claim 42 , wherein the one or more machine learning classification models comprise one or more of: deep learning models, Random Forest algorithms, Support Vector Machines, neural networks, or ensembles thereof.
44. The method of claim 42 , wherein the classification value comprises at least one of a probability that the document is authentic, a confidence score metric that indicates whether the document is authentic, or a similarity metric that indicates whether the document is authentic.
45. The method of claim 42 , wherein at least one of the one or more machine learning classification models is a convolutional neural network.
46. The method of claim 42 , wherein the one or more machine learning classification models is an ensemble classifier comprised of a plurality of convolutional neural networks.
47. The method of claim 42 , wherein one or more interpretable methods are used to validate the classification value.
48. The method of claim 47 , wherein the one or more interpretable methods comprise occlusion of at least a portion of the document, perturbation of at least a portion of the document, or analysis of a heatmap of at least a portion of the document.
49. The method of claim 48 , wherein an output of the one or more interpretable methods comprises an identification of the reconstructed region of interest that represents proof of the document being genuine or fraudulent.
50. The method of claim 42 , wherein the one or more machine learning classification models are trained using a plurality of genuine documents, a plurality of fraudulent documents, or both.
51. The method of claim 50 , wherein the classification value generated by the one or more machine learning classification models is a measure of similarity between one or more of the plurality of genuine documents, one or more of the plurality of fraudulent documents, or both.
52. The method of claim 27 , wherein the images of the physical document comprise one of: images of a front side of the physical document or images of a back side of the physical document.