Deep learning tattoo match system based
A computer-implemented method executed by at least one processor for detecting tattoos on a human body is presented. The method includes inputting a plurality of images into a tattoo detector, selecting one or more images of the plurality of images including tattoos, extracting, via a feature extractor, tattoo feature vectors from the tattoos found in the one or more images of the plurality of images including tattoos, applying a deep learning tattoo matching model to determine potential matches between the tattoo feature vectors and preexisting tattoo images stored in a tattoo training database, and generating a similarity score between the tattoo feature vectors and one or more of the preexisting tattoo images stored in the tattoo training database.
1. A computer-implemented method executed by at least one processor for detecting tattoos on a human body, the method comprising:
inputting a plurality of images into a tattoo detector;
selecting one or more images of the plurality of images including tattoos on a human body;
extracting, via a feature extractor, tattoo feature vectors from the tattoos found in the one or more images of the plurality of images including tattoos on the human body;
applying a deep learning tattoo matching model to determine potential matches between the tattoo feature vectors and preexisting tattoo images stored in a tattoo training database, the deep learning tattoo matching model having three paths for each image of the plurality of images input into the tattoo detector, the first path including a Conv(4:1) branch, the second path including a ConvNet branch with two attention mechanisms, and the third path including a Conv(8:1) branch, where each path provides for a single L2 Norm directly before concatenation is performed; and
generating a similarity score between the tattoo feature vectors and one or more of the preexisting tattoo images on the human body stored in the tattoo training database to identity a person by a soft biometric trait.
2. The method of claim 1 , wherein the tattoo training database employs a combination of two training datasets.
3. The method of claim 2 , wherein the first training dataset includes non-permanent tattoos detected from multiple frames of a video stream.
4. The method of claim 3 , wherein each non-permanent tattoo image in the first training dataset has multiple image versions, with each image version captured at a different angle.
5. The method of claim 4 , wherein the second training dataset includes permanent tattoos detected from video capturing devices.
6. The method of claim 5 , wherein an identifier is assigned to each non-permanent tattoo instance and each permanent tattoo instance.
7. The method of claim 5 , wherein locations of each non-permanent tattoo instance and each permanent tattoo instance are labeled in different frames of the video stream.
8. A system for detecting tattoos on a human body, the system comprising:
a memory; and
a processor in communication with the memory, wherein the processor runs program code to:
input a plurality of images into a tattoo detector;
select one or more images of the plurality of images including tattoos on a human body;
extract, via a feature extractor, tattoo feature vectors from the tattoos found in the one or more images of the plurality of images including tattoos on the human body;
apply a deep learning tattoo matching model to determine potential matches between the tattoo feature vectors and preexisting tattoo images stored in a tattoo training database, the deep learning tattoo matching model having three path for each image of the plurality of images input into the tattoo detector, the first path including a Conv(4:1) branch, the second path including a ConvNet branch with two attention mechanisms, and the third path including a Conv(8:1) branch, where each path provides for a single L2 Norm directly before concatenation is performed; and
generate a similarity score between the tattoo feature vectors and one or more of the preexisting tattoo images on the human body stored in the tattoo training database to identify a person by a soft biometric trait.
9. The system of claim 8 , wherein the tattoo training database employs a combination of two training datasets.
10. The system of claim 9 , wherein the first training dataset includes non-permanent tattoos detected from multiple frames of a video stream.
11. The system of claim 10 , wherein each non-permanent tattoo image in the first training dataset has multiple image versions, with each image version captured at a different angle.
12. The system of claim 11 , wherein the second training dataset includes permanent tattoos detected from video capturing devices.
13. The system of claim 12 , wherein an identifier is assigned to each non-permanent tattoo instance and each permanent tattoo instance.
14. The system of claim 12 , wherein locations of each non-permanent tattoo instance and each permanent tattoo instance are labeled in different frames of the video stream.
15. A non-transitory computer-readable storage medium comprising a computer-readable program for detecting tattoos on a human body, wherein the computer-readable program when executed on a computer causes the computer to perform the steps of:
inputting a plurality of images into a tattoo detector;
selecting one or more images of the plurality of images including tattoos on a human body;
extracting, via a feature extractor, tattoo feature vectors from the tattoos found in the one or more images of the plurality of images including tattoos on the human body;
applying a deep learning tattoo matching model to determine potential matches between the tattoo feature vectors and preexisting tattoo images stored in a tattoo training database, the deep earning tattoo matching model having three paths for each image of the plurality of images input into the tattoo detector, the first path including a Conv(4:1) branch, the second path including a ConvNet branch with two attention mechanisms, and the third path including a Conv(8:1) branch, where each path provides for a single L2 Norm directly before concatenation is performed; and
generating a similarity score between the tattoo feature vectors and one or more of the preexisting tattoo images on the human body stored in the tattoo training database to identify a person by a soft biometric trait.
16. The non-transitory computer-readable storage medium of claim 15 , wherein the tattoo training database employs a combination of two training datasets.
17. The non-transitory computer-readable storage medium of claim 16 , wherein the first training dataset includes non-permanent tattoos detected from multiple frames of a video stream.
18. The non-transitory computer-readable storage medium of claim 17 , wherein each non-permanent tattoo image in the first training dataset has multiple image versions, with each image version captured at a different angle.
19. The non-transitory computer-readable storage medium of claim 18 , wherein the second training dataset includes permanent tattoos detected from video capturing devices.
20. The non-transitory computer-readable storage medium of claim 19 , wherein an identifier is assigned to each non-permanent tattoo instance and each permanent tattoo instance.