Near-duplicate detection of images for training or validation of machine learning models
A system filters near-duplicate images to generate data for training or validation of a machine learning model. The system receives a set of images and generates feature vectors from the images. The system clusters the feature vectors. For each cluster of feature vectors, the system determines near-duplicate pairs of images. The system may generate a cost matrix representing a linear assignment problem and find near-duplicate pairs of images by solving the linear assignment problem. The system filters images from the set of images based on the near-duplicate pairs of images. The system uses the filtered set of images for training or validation of the machine learning model.
1 . A computer-implemented method for generating training data for a machine learning model, the method comprising:
receiving a set of images for generating training dataset for training the machine learning model, the machine learning model configured to receive an input image and output a prediction for the input image;
for each image from the set of images, generating a feature vector;
clustering the feature vectors to generate a plurality of clusters of feature vectors;
for each cluster of feature vectors:
generating a cost matrix, wherein a non-diagonal element of the cost matrix represents a measure of distance between a pair of feature vectors;
finding near-duplicate pairs of feature vectors based on the cost matrix; and
obtaining near-duplicate pairs of images based on the near-duplicate pairs of feature vectors;
filtering the set of images by removing one or more images from the set of images, wherein the removed images belong to the near-duplicate pairs of images; and
adding images from the filtered set of images to a training dataset for training the machine learning model.
2 . The computer-implemented method of claim 1 , further comprising:
training the machine learning model using the training dataset, the training comprising updating parameters of the machine learning model.
3 . The computer-implemented method of claim 1 , further comprising:
using images from the filtered set of images for validating the machine learning model.
4 . The computer-implemented method of claim 1 , wherein finding near-duplicate pairs of feature vectors based on the cost matrix comprises solving a linear assignment problem based on the cost matrix.
5 . The computer-implemented method of claim 4 , wherein solving the linear assignment problem based on the cost matrix is performed using a Hungarian matching process.
6 . The computer-implemented method of claim 1 , wherein generating the feature vector comprises providing the image as input to a neural network and extracting an embedding from a hidden layer of the neural network as the feature vector of the image.
7 . The computer-implemented method of claim 1 , wherein clustering the feature vectors to generate the plurality of clusters of feature vectors is performed using a density based clustering of the set of images.
8 . The computer-implemented method of claim 1 , wherein diagonal elements of the cost matrix are set to a value representing infinity.
9 . A non-transitory computer readable storage medium storing instructions that when executed by one or more computer processors, cause the one or more computer processors to perform steps comprising:
receiving a set of images for generating training dataset for training a machine learning model, the machine learning model configured to receive an input image and output a prediction for the input image;
for each image from the set of images, generating a feature vector;
clustering the feature vectors to generate a plurality of clusters of feature vectors;
for each cluster of feature vectors:
generating a cost matrix, wherein a non-diagonal element of the cost matrix represents a measure of distance between a pair of feature vectors;
finding near-duplicate pairs of feature vectors based on the cost matrix; and
obtaining near-duplicate pairs of images based on the near-duplicate pairs of feature vectors;
filtering the set of images by removing one or more images from the set of images, wherein the removed images belong to the near-duplicate pairs of images; and
adding images from the filtered set of images to a training dataset for training the machine learning model.
10 . The non-transitory computer readable storage medium of claim 9 , wherein the instructions further cause the one or more computer processors to perform steps comprising:
training the machine learning model using the training dataset, the training comprising updating parameters of the machine learning model.
11 . The non-transitory computer readable storage medium of claim 9 , wherein the instructions further cause the one or more computer processors to perform steps comprising:
using images from the filtered set of images for validating the machine learning model.
12 . The non-transitory computer readable storage medium of claim 9 , wherein finding near-duplicate pairs of feature vectors based on the cost matrix comprises solving a linear assignment problem based on the cost matrix.
13 . The non-transitory computer readable storage medium of claim 12 , wherein solving the linear assignment problem based on the cost matrix is performed using a Hungarian matching process.
14 . The non-transitory computer readable storage medium of claim 9 , wherein generating the feature vector comprises providing the image as input to a neural network and extracting an embedding from a hidden layer of the neural network as the feature vector of the image.
15 . The non-transitory computer readable storage medium of claim 9 , wherein clustering the feature vectors to generate the plurality of clusters of feature vectors is performed using a density based clustering of the set of images.
16 . A computer system comprising:
one or more computer processors; and
a non-transitory computer readable storage medium storing instructions that when executed by the one or more computer processors, cause the one or more computer processors to perform steps comprising:
receiving a set of images for generating training dataset for training a machine learning model, the machine learning model configured to receive an input image and output a prediction for the input image;
for each image from the set of images, generating a feature vector;
clustering the feature vectors to generate a plurality of clusters of feature vectors;
for each cluster of feature vectors:
generating a cost matrix, wherein a non-diagonal element of the cost matrix represents a measure of distance between a pair of feature vectors;
finding near-duplicate pairs of feature vectors based on the cost matrix; and
obtaining near-duplicate pairs of images based on the near-duplicate pairs of feature vectors;
filtering the set of images by removing one or more images from the set of images, wherein the removed images belong to the near-duplicate pairs of images; and
adding images from the filtered set of images to a training dataset for training the machine learning model.
17 . The computer system of claim 16 , wherein the instructions further cause the one or more computer processors to perform steps comprising:
training the machine learning model using the training dataset, the training comprising updating parameters of the machine learning model.
18 . The computer system of claim 16 , wherein the instructions further cause the one or more computer processors to perform steps comprising:
using images from the filtered set of images for validating the machine learning model.
19 . The computer system of claim 16 , wherein finding near-duplicate pairs of feature vectors based on the cost matrix comprises solving a linear assignment problem based on the cost matrix.
20 . The computer system of claim 19 , wherein solving the linear assignment problem based on the cost matrix is performed using a Hungarian matching process.