System and method for classifying images with a combination of nearest-neighbor-based label propagation and kernel principal component analysis
Described is a system for detecting and classifying new patterns of objects and images for applications where labeled data is scarce. In operation, the system trains a neural network with unlabeled images and extracts features with the neural network from both the unlabeled images and a set of labeled images to generate a feature space. Labels are propagated in the feature space using nearest neighbors, allowing for modeling of a per-class simplified distribution. An object in a new test image can then be classified using reconstruction error based on the per-class simplified distributions.
1. A system for classifying images, the system comprising:
one or more processors and associated memory, the memory being a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions, the one or more processors perform operations of:
training a neural network with unlabeled images;
extracting features with the neural network from both the unlabeled images and a set of labeled images to generate a feature space;
propagating labels in the feature space using nearest neighbors;
modeling a per-class simplified distribution in the feature space using kernel principal component analysis (KPCA); and
classifying an object in a new test image using reconstruction error based on the per-class simplified distributions.
2. The system as set forth in claim 1 , wherein in propagating labels in the feature space, a fixed number of labels are iteratively added through a series of iteration steps.
3. The system as set forth in claim 2 , wherein in each iteration step, only one unlabeled data point is added by finding, in the feature space, a point x j that is closest to any of the points x i in a set of labeled points, with the point x j being added to the set of labeled points.
4. The system as set forth in claim 3 , wherein training the neural network is performed with self-supervised learning.
5. The system as set forth in claim 4 , further comprising an operation of uploading the neural network to a mobile platform.
6. The system as set forth in claim 5 , further comprising an operation of causing the mobile platform to perform a maneuver based on classification of an object.
7. The system as set forth in claim 1 , further comprising an operation of uploading the neural network to a mobile platform.
8. The system as set forth in claim 7 , further comprising an operation of causing the mobile platform to perform a maneuver based on classification of an object.
9. The system as set forth in claim 1 , wherein training the neural network is performed with self-supervised learning.
10. A computer program product for classifying images, the computer program product comprising:
a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions by one or more processors, the one or more processors perform operations of:
training a neural network with unlabeled images;
extracting features with the neural network from both the unlabeled images and a set of labeled images to generate a feature space;
propagating labels in the feature space using nearest neighbors;
modeling a per-class simplified distribution in the feature space using kernel principal component analysis (KPCA); and
classifying an object in a new test image using reconstruction error based on the per-class simplified distributions.
11. The computer program product as set forth in claim 10 , wherein in propagating labels in the feature space, a fixed number of labels are iteratively added through a series of iteration steps.
12. The computer program product as set forth in claim 11 , wherein in each iteration step, only one unlabeled data point is added by finding, in the feature space, a point x j that is closest to any of the points x i in a set of labeled points, with the point x j being added to the set of labeled points.
13. The computer program product as set forth in claim 10 , further comprising an operation of uploading the neural network to a mobile platform.
14. The computer program product as set forth in claim 13 , further comprising an operation of causing the mobile platform to perform a maneuver based on classification of an object.
15. The computer program product as set forth in claim 10 , wherein training the neural network is performed with self-supervised learning.
16. A computer implemented method for classifying images, the method comprising an act of:
causing one or more processers to execute instructions encoded on a non-transitory computer-readable medium, such that upon execution, the one or more processors perform operations of:
training a neural network with unlabeled images;
extracting features with the neural network from both the unlabeled images and a set of labeled images to generate a feature space;
propagating labels in the feature space using nearest neighbors;
modeling a per-class simplified distribution in the feature space using kernel principal component analysis (KPCA); and
classifying an object in a new test image using reconstruction error based on the per-class simplified distributions.
17. The method as set forth in claim 16 , wherein in propagating labels in the feature space, a fixed number of labels are iteratively added through a series of iteration steps.
18. The method as set forth in claim 17 , wherein in each iteration step, only one unlabeled data point is added by finding, in the feature space, a point x j that is closest to any of the points x i in a set of labeled points, with the point x j being added to the set of labeled points.
19. The method as set forth in claim 16 , further comprising an operation of uploading the neural network to a mobile platform.
20. The method as set forth in claim 19 , further comprising an operation of causing the mobile platform to perform a maneuver based on classification of an object.
21. The method as set forth in claim 16 , wherein training the neural network is performed with self-supervised learning.