Computer vision systems and methods for blind localization of image forgery

1. A computer vision system for localizing image forgery comprising:

a memory; and

a processor in communication with the memory, the processor:

generating a constrained convolution using a plurality of learned rich filters by computing residuals for the plurality of learned rich filters, each residual being a difference between a predicted value for a central pixel defined over a pixel neighborhood and a scaled value of a pixel;

training a neural network with the constrained convolution and a plurality of images of a dataset to learn a low-level representation indicative of a statistical signature of at least one source camera model for each image among the plurality of images, and

localizing an attribute of an image of the dataset by the trained neural network.

2. The system of claim 1 , wherein the processor:

extracts at least one noise residual pattern from each image among the plurality of images via the constrained convolution,

determines a spatial distribution of the extracted at least one noise residual pattern, and

suppresses semantic edges present in each image among the plurality of images by applying a probabilistic regularization,

generating a constrained convolution via a plurality of learned rich filters by computing residuals for the plurality of learned rich filters, each residual being a difference between a predicted value for a central pixel defined over a pixel neighborhood and a scaled value of a pixel;

training a neural network with the constrained convolution and a plurality of images of a dataset to learn a low-level representation indicative of a statistical signature of at least one source camera model for each image among the plurality of images, and

localizing an attribute of an image of the dataset by the trained neural network.

3. The system of claim 2 , wherein the processor trains the neural network with a complete loss function based on a cross-entropy loss function over the dataset, the probabilistic regularization, and a rich filter constraint penalty.

4. The system of claim 1 , wherein the processor localizes the attribute of the image of the dataset by the trained neural network by:

subdividing the image into a plurality of patches,

determining a hundred-dimensional feature vector for each patch, and

segmenting the plurality of patches by applying an expectation maximization algorithm to each patch to fit a two component Gaussian mixture model to each feature vector.

5. The system of claim 1 , wherein the neural network is an 18 layer deep Convolutional Neural Network (CNN).

6. The system of claim 1 , wherein the dataset is a Dresden Image dataset.

7. The system of claim 1 , wherein the localized adversarial perturbation of the image is a splicing manipulation.

8. A method for localizing image forgery by a computer vision system, comprising the steps of:

subdividing the image into a plurality of patches,

determining a hundred-dimensional feature vector for each patch, and

segmenting the plurality of patches by applying an expectation maximization algorithm to each patch to fit a two component Gaussian mixture model to each feature vector.

9. The method of claim 8 , further comprising:

extracting at least one noise residual pattern from each image among the plurality of images via the constrained convolution,

determining a spatial distribution of the extracted at least one noise residual pattern, and

suppressing semantic edges present in each image among the plurality of images by applying a probabilistic regularization.

10. The method of claim 9 , further comprising training the neural network with a complete loss function based on a cross-entropy loss function over the dataset, the probabilistic regularization, and a rich filter constraint penalty.

11. The method of claim 8 , further comprising localizing the attribute of the image of the dataset by the trained neural network by

subdividing the image into a plurality of patches,

determining a hundred-dimensional feature vector for each patch, and

segmenting the plurality of patches by applying an expectation maximization algorithm to each patch to fit a two component Gaussian mixture model to each feature vector.

12. The method of claim 8 , wherein the neural network is an 18 layer deep Convolutional Neural Network (CNN).

13. The method of claim 8 , wherein the localized adversarial perturbation of the image is a splicing manipulation.

14. A non-transitory computer readable medium having instructions stored thereon for localizing image forgery by a computer vision system which, when executed by a processor, causes the processor to carry out the steps of:

generating a constrained convolution via a plurality of learned rich filters by computing residuals for the plurality of learned rich filters, each residual being a difference between a predicted value for a central pixel defined over a pixel neighborhood and a scaled value of a pixel;

training a neural network with the constrained convolution and a plurality of images of a dataset to learn a low-level representation indicative of a statistical signature of at least one source camera model for each image among the plurality of images, and

localizing an attribute of an image of the dataset by the trained neural network.

15. The non-transitory computer readable medium of claim 14 , the processor further carrying out the steps of:

extracting at least one noise residual pattern from each image among the plurality of images via the constrained convolution,

determining a spatial distribution of the extracted at least one noise residual pattern, and

suppressing semantic edges present in each image among the plurality of images by applying a probabilistic regularization.

16. The non-transitory computer readable medium of claim 15 , the processor further carrying out the step of training the neural network with a complete loss function based on a cross-entropy loss function over the dataset, the probabilistic regularization, and a rich filter constraint penalty.

17. The non-transitory computer readable medium of claim 14 , the processor localizing the attribute of the image of the dataset by the trained neural network by carrying out the steps of:

subdividing the image into a plurality of patches,

determining a hundred-dimensional feature vector for each patch, and

segmenting the plurality of patches by applying an expectation maximization algorithm to each patch to fit a two component Gaussian mixture model to each feature vector.

18. The non-transitory computer readable medium of claim 14 , wherein the neural network is an 18 layer deep Convolutional Neural Network (CNN).

19. The non-transitory computer readable medium of claim 14 , wherein the localized adversarial perturbation of the image is a splicing manipulation.