Physical document verification in uncontrolled environments
A method for verifying authenticity of a physical document includes receiving an image of a physical document to be authenticated including the physical document and a background. A pre-processed image is produced that includes the physical document separated from the background. The producing includes separating the physical document from the background by semantic segmentation utilizing an artificial neural network trained using an augmented dataset generated by applying geometric transformations over different backgrounds. Features of the pre-processed image are extracted to determine a document type. In response to determining the document type of the physical document, the method includes verifying, utilizing a machine learning classifier, whether the physical document is authentic based on the extracted features relative to expected features for the corresponding document type. An indication of whether the physical document is authentic based on the verifying is generated.
1. A method for verifying authenticity of a physical document, comprising:
receiving an image of a physical document to be authenticated, the image including the physical document and a background;
producing, by a computing device, a pre-processed image, the pre-processed image including the physical document separated from the background, wherein the producing includes:
separating the physical document from the background by semantic segmentation, the semantic segmentation utilizing an artificial neural network trained using an augmented dataset generated by applying geometric transformations over different backgrounds; and
one or more of: a corner detection analysis and a brightness analysis;
extracting, by the computing device, features of the pre-processed image to determine a document type of the physical document, wherein the features are defined based on the document type;
in response to determining the document type of the physical document, verifying, by the computing device, whether the physical document is authentic based on the extracted features relative to expected features for the corresponding document type, the verifying utilizing a machine learning classifier; and
generating, by the computing device, an indication of whether the physical document is authentic based on the verifying.
2. The method of claim 1 , wherein the artificial neural network is trained with a plurality of images of physical documents, a first of the plurality of images of physical documents including a corresponding physical document and background and a second of the plurality of images of physical documents including a background with at least a part of the physical document not being visible.
3. The method of claim 1 , the machine learning classifier for the verifying whether the physical document is authentic is trained with a plurality of images of physical documents.
4. The method of claim 1 , wherein the physical document is one of an identity document, an event ticket, a birth certificate, or an invoice.
5. The method of claim 1 , comprising:
cropping the image.
6. The method of claim 1 , wherein the computing device is a mobile computing device.
7. The method of claim 6 , wherein the mobile computing device is one of a smartphone, a smartwatch, a tablet, a laptop, or the like.
8. The method of claim 6 , wherein the mobile computing device includes an image capturing device and the image of the physical document to be authenticated was captured from the image capturing device.
9. A method for localizing an image, the method comprising:
providing a plurality of physical document images and a plurality of background images, wherein each of the plurality of physical document images includes a physical document and each of the plurality of background images includes a background absent a physical document;
selecting one or more subsets including the plurality of physical document images and the plurality of background images;
for each of the subsets:
generating a simulated image using the physical document images and the background images;
providing the simulated images to a fully convolutional neural network with binary masks for the corresponding set of physical document images and background images as inputs to train the fully convolutional neural network; and
training the fully convolutional network to recognize a portion of the simulated image that includes the physical document image and a portion of the simulated image that includes the background image as expected outputs of processing the simulated image;
wherein the fully convolutional neural network runs on at least one server machine having at least one processor and at least one non-transitory computer readable medium.
10. The method of claim 9 , comprising:
detecting the corners of the physical document image, including:
finding a contour of the physical document image; and
based at least in part on the contour of the physical document image, performing a linear regression on each side of the physical document image to identify intersections between border lines which define corners of the physical document image.
11. The method of claim 10 , comprising:
identifying brightness areas that are outside a threshold brightness; and
in response to identifying brightness areas that are outside the threshold brightness, generating an error message and rejecting the physical document image.
12. A system, comprising:
a computing device coupled to a network, the computing device including a processing device, the processing device configured to execute instructions to:
in response to receiving an image of a physical document to be authenticated, the image including the physical document and a background, produce a pre-processed image, the pre-processed image including the physical document separated from the background, including:
separating the physical document from the background by semantic segmentation, the semantic segmentation utilizing an artificial neural network trained using an augmented dataset generated by applying geometric transformations over different backgrounds; and
one or more of: a corner detection analysis and a brightness analysis;
extract features of the pre-processed image to determine a document type of the physical document, wherein the features are defined based on the document type;
in response to determining the document type of the physical document, verify whether the physical document is authentic based on the extracted features relative to expected features for the corresponding document type, the verifying utilizing a machine learning classifier; and
generate an indication of whether the physical document is authentic based on the verifying.
13. The system of claim 12 , wherein the artificial neural network is trained with a plurality of images of physical documents, a first of the plurality of images of physical documents including a corresponding physical document and background and a second of the plurality of images of physical documents including a background with at least a part of the physical document not being visible.
14. The system of claim 12 , the machine learning classifier that verifies whether the physical document is authentic is trained with a plurality of images of physical documents.
15. The system of claim 12 , wherein the physical document is one of an identity document, an event ticket, a birth certificate, or an invoice.
16. The system of claim 12 , comprising: a mobile computing device.
17. The system of claim 16 , wherein the mobile computing device is one of a smartphone, a smartwatch, a tablet, a laptop, or the like.
18. The method of claim 16 , wherein the mobile computing device includes an image capturing device and the image of the physical document to be authenticated was captured from the image capturing device.