SYSTEMS AND METHODS FOR DETECTING IMAGE RECAPTURE
Systems, computer-implemented methods, and non-transitory machine-readable storage media are provided for detecting recapture attacks of images. One method comprises extracting one or more features from an image captured by a device; applying the one or more features as input to a trained machine learning model, wherein the trained machine learning model outputs a first score based on the extracted features; obtaining metadata of the image; performing a statistical analysis of the metadata of the image; generating a second score based on the statistical analysis of the metadata of the image; and generating a probability that the image is a recapture of an original image based on the first score and the second score.
1 . A system for detecting image recapture, the system comprising:
a hardware processor;
an image capture sensor;
one or more other sensors; and
a system encoded with instructions executable by the hardware processor to perform operations comprising:
extracting one or more features from an image captured by the image capture sensor; and
generating a probability that an image is a recapture of an original image based on the extracted one or more features and an analysis of metadata of the image
2 . The system of claim 1 , wherein
generating a probability that an image is a recapture of an original image based on the extracted one or more features and an analysis of metadata of the image comprises:
generating a first score based on the extracted one or more features;
generating a second score based on metadata of the image; and
generating a probability that the image is a recapture of an original image based on the first score and the second score.
3 . The system of claim 2 , wherein a portion of the instructions comprises a recapture detection application, the recapture detection application comprising:
a metadata analysis component configured to perform a first analysis that comprises comparing the metadata with fingerprints of known image manipulation software;
a content analysis component configured to perform a second analysis of the extracted one or more features; and
wherein the operations further comprise:
generating the first score based on the second analysis.
4 . The system of claim 3 , wherein performing the second analysis comprises identifying one or more recapture characteristics.
5 . The system of claim 3 , wherein performing the second analysis comprises performing a discrete cosine transform of the image.
6 . The system of claim 3 , wherein performing the second analysis comprises applying the extracted one or more features as input to a trained machine learning model, wherein the trained machine learning model outputs the first score based on the extracted one or more features.
7 . The system of claim 3 , wherein performing the second analysis comprises at least one of:
detecting that a surface is flat or two-dimensional;
detecting pixel color profiles and edges;
detecting muted color distributions;
detecting aliasing; or
detecting Moiré pattern.
8 .- 21 . (canceled)
22 . The system of claim 3 , wherein performing the second analysis comprises:
performing simultaneous localization and mapping on the image;
generating a depth map;
extracting features from the depth map that expose sings of a rebroadcast; and
estimating trajectory of the image capture sensor in space based on relative movement of points in successive frames based on the performed simultaneous localization and mapping.
23 . The system of claim 3 , wherein performing the first analysis comprises identifying one or more potential indicators of recapture.
24 . The system of claim 3 , wherein performing the first analysis comprises at least one of:
determining whether a focal distance used by an image capture device to capture the image aligns with visual content of the image;
determining whether intrinsic and distortion coefficients of the image capture device accurately undistort the image; or
determining whether position and orientation sensors of the image capture device imply that the device was pointing in a direction consistent with visual content of the image.
25 . The system of claim 3 , wherein performing the first analysis comprises:
blacklisting the image in response to determining that the metadata indicates that the image has been derived from or has been manipulated by known manipulative software.
26 . A method for detecting image recapture, the method comprising:
extracting one or more features from an image captured by an image capture sensor; and
generating a probability that an image is a recapture of tan original image based on the extract one or more features and an analysis of metadata of the image.
27 . The method of claim 26 , wherein generating a probability that an image is a recapture of an original image based on the extracted one or more features and an analysis of metadata of the image comprises:
generating a first score based on the extracted one or more features;
generating a second score based on performing a first analysis that comprises comparing the metadata with fingerprints of known image manipulation software; and
generating a probability that the image is a recapture of an original image based on the first score and the second score.
28 . The method of claim 26 , further comprising:
performing a second analysis of the extracted one or more features; and
generating the first score based on the second analysis.
29 . The method of claim 28 , wherein performing the second analysis comprises identifying one or more recapture characteristics.
30 . The method of claim 28 , wherein performing the second analysis comprises performing a discrete cosine transform of the image.
31 . The method of claim 28 , wherein performing the second analysis comprises applying the extracted one or more features as input to a trained machine learning model, wherein the trained machine learning model outputs the first score based on the extracted one or more features.
32 . The method of claim 28 , wherein performing the second analysis comprises at least one of:
detecting that a surface is flat or two-dimensional;
detecting pixel color profiles and edges;
detecting muted color distributions;
detecting aliasing; or
detecting Moiré pattern.
33 . The method of claim 28 , wherein performing the first analysis comprises identifying one or more potential indicators of recapture.
34 . The method of claim 28 , wherein performing the first analysis comprises at least one of:
determining whether a focal distance used by an image capture device to capture the image aligns with visual content of the image;
determining whether intrinsic and distortion coefficients of the image capture device accurately undistort the image; or
determining whether position and orientation sensors of the image capture device imply that the device was pointing in a direction consistent with visual content of the image.