Methods and systems for detecting photograph replacement in a photo identity document

1. A computer-implemented method for detecting identity document photograph replacement, comprising:

receiving image data of a cropped photograph region of an identity document (ID), wherein the ID comprises a photograph, and wherein the cropped photograph region comprises at least the photograph;

predicting, by a multiclass classification model and as a predicted ID category, an ID category based on the image data of the cropped photograph region, wherein the predicted ID category corresponds to a predefined class of a set of predefined classes in the multiclass classification model;

comparing the predicted ID category with an indicated category of the ID; and

in response to a calculated mismatch between the predicted ID category and the indicated category of the ID, determining that the cropped photograph region includes a replaced photograph.

2. The computer-implemented method of claim 1 , wherein the cropped photograph region further comprises a background surrounding the photograph.

3. The computer-implemented method of claim 2 , wherein the photograph has a height of H and a width of W, and the cropped photograph region has a height between H and R H *H and a width between W and R W *W, wherein R H and R W are predetermined ratios that are greater than 1.

4. The computer-implemented method of claim 1 , wherein receiving image data of a cropped photograph region of an ID comprises:

receiving an image of the ID;

pre-processing the image of the ID to generate an aligned image of the ID; and

cropping a photograph region from the aligned image of the ID based on the indicated category of the ID and photograph location information corresponding to the indicated category of the ID.

5. The computer-implemented method of claim 1 , wherein predicting the ID category based on the image data of the cropped photograph region comprises:

inputting the image data to the multiclass classification model;

outputting a score for each predefined class of the set of predefined classes; and

determining the ID category, wherein the ID category corresponds to a predefined class of the set of predefined classes with a highest score.

6. The computer-implemented method of claim 5 , further comprising:

setting a threshold for the highest score; and

in response to the predicted ID category matching the indicated category of the ID and the highest score less than the threshold, determining that the cropped photograph region comprises a replaced photograph.

7. The computer-implemented method of claim 1 , wherein the indicated category of the ID is entered or selected by a user.

8. The computer-implemented method of claim 1 , wherein the set of predefined classes comprises national ID cards, passports, driver licenses from one or more countries.

9. The computer-implemented method of claim 1 , wherein the multiclass classification model is trained by:

providing sample images of N categories of photo IDs;

generating cropped photograph regions based on the sample images of N categories of photo IDs; and

training a classification model, wherein the classification model has N classes based on the cropped photograph regions, and wherein the N classes correspond to N categories of photo IDs.

10. The computer-implemented method of claim 9 , wherein generating cropped photograph regions based on the sample images of N categories of photo IDs comprises:

pre-processing at least one sample image of the sample images of N categories of photo IDs to generate an aligned image of the photo ID; and

cropping, based on a category of the photo ID and photograph location information corresponding to the category of the photo ID, a photograph region from the aligned image of the photo ID.

11. The computer-implemented method of claim 9 , further comprising:

updating the cropped photograph regions from the sample images of N categories of photo IDs that were used to train the classification model by including the image data of the cropped photograph regions.

12. The computer-implemented method of claim 9 , wherein training the classification model based on the cropped photograph regions is based on a convolutional neural network (CNN) algorithm.

13. A non-transitory computer-readable medium storing one or more instructions executable by a computer system to perform operations for detecting identity document photograph replacement, comprising:

receiving image data of a cropped photograph region of an identity document (ID), wherein the ID comprises a photograph, and wherein the cropped photograph region comprises at least the photograph;

predicting, by a multiclass classification model and as a predicted ID category, an ID category based on the image data of the cropped photograph region, wherein the predicted ID category corresponds to a predefined class of a set of predefined classes in the multiclass classification model;

comparing the predicted ID category with an indicated category of the ID; and

in response to a calculated mismatch between the predicted ID category and the indicated category of the ID, determining that the cropped photograph region includes a replaced photograph.

14. The non-transitory computer-readable medium of claim 13 , wherein the cropped photograph region further comprises a background surrounding the photograph.

15. The non-transitory computer-readable medium of claim 14 , wherein the photograph has a height of H and a width of W, and the cropped photograph region has a height between H and R H *H and a width between Wand R W *W, wherein R H and R W are predetermined ratios that are greater than 1.

16. The non-transitory computer-readable medium of claim 13 , wherein receiving image data of a cropped photograph region of an ID comprises:

receiving an image of the ID;

pre-processing the image of the ID to generate an aligned image of the ID; and

cropping a photograph region from the aligned image of the ID based on the indicated category of the ID and photograph location information corresponding to the indicated category of the ID.

17. The non-transitory computer-readable medium of claim 13 , wherein predicting the ID category based on the image data of the cropped photograph region comprises:

inputting the image data to the multiclass classification model;

outputting a score for each predefined class of the set of predefined classes; and

determining the ID category, wherein the ID category corresponds to a predefined class of the set of predefined classes with a highest score.

18. The non-transitory computer-readable medium of claim 17 , further comprising operations for:

setting a threshold for the highest score; and

in response to the predicted ID category matching the indicated category of the ID and the highest score less than the threshold, determining that the cropped photograph region comprises a replaced photograph.

19. The non-transitory computer-readable medium of claim 13 , wherein the indicated category of the ID is entered or selected by a user.

20. The non-transitory computer-readable medium of claim 13 , wherein the set of predefined classes comprises national ID cards, passports, driver licenses from one or more countries.

21. The non-transitory computer-readable medium of claim 13 , wherein the multiclass classification model is trained by:

providing sample images of N categories of photo IDs;

generating cropped photograph regions based on the sample images of N categories of photo IDs; and

training a classification model, wherein the classification model has N classes based on the cropped photograph regions, and wherein the N classes correspond to N categories of photo IDs.

22. The non-transitory computer-readable medium of claim 21 , wherein generating cropped photograph regions based on the sample images of N categories of photo IDs comprises:

pre-processing at least one sample image of the sample images of N categories of photo IDs to generate an aligned image of the photo ID; and

cropping, based on a category of the photo ID and photograph location information corresponding to the category of the photo ID, a photograph region from the aligned image of the photo ID.

23. The non-transitory computer-readable medium of claim 21 , further comprising operations for:

updating the cropped photograph regions from the sample images of N categories of photo IDs that were used to train the classification model by including the image data of the cropped photograph regions.

24. The non-transitory computer-readable medium of claim 21 , wherein training the classification model based on the cropped photograph regions is based on a convolutional neural network (CNN) algorithm.

25. A computer-implemented system for detecting identity document photograph replacement, comprising:

one or more computers; and

one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, cause the one or more computers to perform one or more operations comprising:

receiving image data of a cropped photograph region of an identity document (ID), wherein the ID comprises a photograph, and wherein the cropped photograph region comprises at least the photograph;

predicting, by a multiclass classification model and as a predicted ID category, an ID category based on the image data of the cropped photograph region, wherein the predicted ID category corresponds to a predefined class of a set of predefined classes in the multiclass classification model;

comparing the predicted ID category with an indicated category of the ID; and

in response to a calculated mismatch between the predicted ID category and the indicated category of the ID, determining that the cropped photograph region includes a replaced photograph.