Systems and methods for detecting image recapture
Systems, computer-implemented methods, and non-transitory machine-readable storage media are provided for detecting recapture attacks of images. One method comprises extracting one or more features from an image captured by a device; applying the one or more features as input to a trained machine learning model, wherein the trained machine learning model outputs a first score based on the extracted features; obtaining metadata of the image; performing a statistical analysis of the metadata of the image; generating a second score based on the statistical analysis of the metadata of the image; and generating a probability that the image is a recapture of an original image based on the first score and the second score.
1. A system for detecting recapture attacks of images, the system comprising:
a hardware processor; and
a system encoded with instructions executable by the hardware processor to perform operations comprising:
extracting one or more features from an image;
generating a first score based on the extracted features;
obtaining metadata of the image;
performing a statistical analysis of the metadata of the image;
generating a second score based on the statistical analysis of the metadata of the image; and
generating a probability that the image is a recapture of an original image based on the first score and the second score.
2. The system of claim 1 , wherein generating a probability that the image is a recapture of an original image based on the first score and the second score comprises:
providing the first score and the second score to an image meta model, wherein the image meta model generates the probability that the image is a recapture of an original image based on the first score and the second score.
3. The system of claim 1 , wherein the image is one of multiple images in a video captured by the device, and wherein the operations further comprise:
performing the operations of claim 1 for a second image of the multiple images in the video to generate a second probability that the second image is a recapture of a second original image;
providing the probability that the image is a recapture of an original image and the second probability that the second image is a recapture of a second original image to a video meta model, wherein the video meta model generates a probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image and the second probability that the second image is a recapture of a second original image.
4. The system of claim 3 , wherein the operations further comprise:
generating a depth map of a scene based on the first image and the second image;
providing the depth map to a depth model, wherein the depth model generates a depth score based on the depth map; and
generating a probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image, the second probability that the second image is a recapture of a second original image, and the depth score; and
providing the depth score to the video meta model, wherein the video meta model generates a probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image, the second probability that the second image is a recapture of a second original image, and the depth score.
5. The system of claim 3 , wherein the operations further comprise:
determining a first trajectory taken by the device during capture of the video based on the video;
obtaining metadata of the second image;
determining a second trajectory taken by the device during capture of the video based on sensor data included within the metadata of the image and the metadata of the second image;
generating a similarity score based on a comparison of the first trajectory with the second trajectory; and
providing the similarity score to the video meta model, wherein the video meta model generates the probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image, the second probability that the second image is a recapture of a second original image, and the similarity score.
6. The system of claim 1 , wherein the image is one of multiple images in a video captured by the device, and wherein the operations further comprise:
performing the operations of claim 1 for a second image of the multiple images in the video to generate a second probability that the second image is a recapture of a second original image; and
generating a probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image and the second probability that the second image is a recapture of a second original image.
7. The system of claim 6 , wherein the operations further comprise:
generating a depth map of a scene based on the first image and the second image;
providing the depth map to a depth model, wherein the depth model generates a depth score based on the depth map; and
generating the probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image, the second probability that the second image is a recapture of a second original image, the similarity score, and the depth score.
8. A non-transitory machine-readable storage medium encoded with instructions executable by a hardware processor of a computing component for detecting recapture attacks of images, the machine-readable storage medium comprising instructions to configure the hardware processor to perform operations comprising:
extracting one or more features from an image;
generating a first score based on the extracted features;
obtaining metadata of the image;
performing a statistical analysis of the metadata of the image;
generating a second score based on the statistical analysis of the metadata of the image; and
generating a probability that the image is a recapture of an original image based on the first score and the second score.
9. The non-transitory machine-readable storage medium of claim 8 , wherein generating a probability that the image is a recapture of an original image based on the first score and the second score comprises:
providing the first score and the second score to an image meta model, wherein the image meta model generates the probability that the image is a recapture of an original image based on the first score and the second score.
10. The non-transitory machine-readable storage medium of claim 8 , wherein the image is one of multiple images in a video captured by the device, and wherein the operations further comprise:
performing the operations of claim 8 for a second image of the multiple images in the video to generate a second probability that the second image is a recapture of a second original image;
providing the probability that the image is a recapture of an original image and the second probability that the second image is a recapture of a second original image to a video meta model, wherein the video meta model generates a probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image and the second probability that the second image is a recapture of a second original image.
11. The non-transitory machine-readable storage medium of claim 10 , wherein the operations further comprise:
generating a depth map of a scene based on the first image and the second image;
providing the depth map to a depth model, wherein the depth model generates a depth score based on the depth map; and
generating a probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image, the second probability that the second image is a recapture of a second original image, and the depth score; and
providing the depth score to the video meta model, wherein the video meta model generates a probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image, the second probability that the second image is a recapture of a second original image, and the depth score.
12. The non-transitory machine-readable storage medium of claim 10 , wherein the operations further comprise:
determining a first trajectory taken by the device during capture of the video based on the video;
obtaining metadata of the second image;
determining a second trajectory taken by the device during capture of the video based on sensor data included within the metadata of the image and the metadata of the second image;
generating a similarity score based on a comparison of the first trajectory with the second trajectory; and
providing the similarity score to the video meta model, wherein the video meta model generates the probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image, the second probability that the second image is a recapture of a second original image, and the similarity score.
13. The non-transitory machine-readable storage medium of claim 8 , wherein the image is one of multiple images in a video captured by the device, and wherein the operations further comprise:
performing the operations of claim 8 for a second image of the multiple images in the video to generate a second probability that the second image is a recapture of a second original image; and
generating a probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image and the second probability that the second image is a recapture of a second original image.
14. The non-transitory machine-readable storage medium of claim 13 , wherein the operations further comprise:
generating a depth map of a scene based on the first image and the second image;
provide the depth map to a depth model, wherein the depth model generates a depth score based on the depth map; and
generating the probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image, the second probability that the second image is a recapture of a second original image, the similarity score, and the depth score.
15. A computer-implemented method for detecting recapture attacks of images, the method comprising:
extracting one or more features from an image;
generating a first score based on the extracted features;
obtaining metadata of the image;
performing a statistical analysis of the metadata of the image;
generating a second score based on the statistical analysis of the metadata of the image; and
generating a probability that the image is a recapture of an original image based on the first score and the second score.
16. The computer-implemented method of claim 15 , wherein generating a probability that the image is a recapture of an original image based on the first score and the second score comprises:
providing the first score and the second score to an image meta model, wherein the image meta model generates the probability that the image is a recapture of an original image based on the first score and the second score.
17. The computer-implemented method of claim 15 , wherein the image is one of multiple images in a video captured by the device, the method further comprising:
performing the method of claim 15 for a second image of the multiple images in the video to generate a second probability that the second image is a recapture of a second original image;
providing the probability that the image is a recapture of an original image and the second probability that the second image is a recapture of a second original image to a video meta model, wherein the video meta model generates a probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image and the second probability that the second image is a recapture of a second original image.
18. The computer-implemented method of claim 17 , further comprising:
generating a depth map of a scene based on the first image and the second image;
providing the depth map to a depth model, wherein the depth model generates a depth score based on the depth map; and
generating a probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image, the second probability that the second image is a recapture of a second original image, and the depth score; and
providing the depth score to the video meta model, wherein the video meta model generates a probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image, the second probability that the second image is a recapture of a second original image, and the depth score.
19. The computer-implemented method of claim 17 , further comprising:
determining a first trajectory taken by the device during capture of the video based on the video;
obtaining metadata of the second image;
determining a second trajectory taken by the device during capture of the video based on sensor data included within the metadata of the image and the metadata of the second image;
generating a similarity score based on a comparison of the first trajectory with the second trajectory; and
providing the similarity score to the video meta model, wherein the video meta model generates the probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image, the second probability that the second image is a recapture of a second original image, and the similarity score.
20. The computer-implemented method of claim 15 , wherein the image is one of multiple images in a video captured by the device, the method further comprising:
performing the method of claim 15 for a second image of the multiple images in the video to generate a second probability that the second image is a recapture of a second original image;
generating a depth map of a scene based on the first image and the second image;
providing the depth map to a depth model, wherein the depth model generates a depth score based on the depth map; and
generating the probability that the video is a recapture of an original video based on the probability that the image is a recapture of an original image, the second probability that the second image is a recapture of a second original image, the similarity score, and the depth score.