Multiple hypothesis transformation matching for robust verification of object identification
A method for increasing the confidence of a match between a test image and an image stored in a library database in which features are extracted from the test image and compared to features stored in the image database. If a match is determined, one or more transformations are performed on the test image to generate pose-altered images from which features are extracted and matched with pose-altered images in the database. The scores for the subsequent matchings can be aggregated to determine an overall probability of a match between the test image in an image in the library database.
1. A method comprising:
receiving a test image;
extracting features from the test image;
matching the features extracted from the test image with features stored in a library database;
if a match is obtained, performing one or more transformations on the test image to generate one or more pose-altered images;
extracting features from the one or more pose-altered images; and
matching the features extracted from the one or more pose-altered images with features stored in the library database.
2. The method of claim 1 further comprising:
receiving one or more scores from the matching of features extracted from the one or more pose-altered images; and
submitting the one or more scores to a neural network trained to determine the probability of a match between the test image and an image in the library database, based on the one or more scores.
3. The method of claim 2 wherein the step of transforming the test image comprises:
generating a 3D model of the test image; and
generating pose-altered images showing different viewpoints of the image by rotating the 3D model along one or more axes.
4. The method of claim 2 wherein the step of transforming the test image comprises:
fixing the test image to a plane with a known depth; and
generating pose-altered images showing different viewpoints of the image by rotating the test image along one or more of the axes of the plane.
5. The method of claim 2 wherein the step of transforming the test image comprises:
using planar homography by fitting corner endpoints of the test image to different configurations to simulate pose-altered views of the image.
6. The method of claim 2 wherein the step transforming the test image comprises:
training a machine learning model to generate pose-altered images based on the test image; and
inputting the test image of the item to the machine learning model.
7. The method of claim 2 further comprising:
pose correcting the test image to eliminate portions of the image containing views of sides of an item other than a frontal side.
8. The method of claim 1 further comprising:
receiving one or more scores from the matching of features extracted from the one or more pose-altered images;
aggregating the one or more scores; and
calculating a probability of a match between the test image in an image in the library database based on the aggregated scores.
9. The method of claim 1 wherein the pose-altered images include views modified from the one or more 2D test images, the modifying including presenting items depicted in the image at arbitrary angles, rotating the items, translating the items, crumpling the items, modifying the colors of the items, over-exposing the items and under-exposing the items.
10. The method of claim 1 wherein the matching is performed by a trained classifier.
11. A system comprising:
a processor; and
software that, when excluded by the processor, causes the system to:
receive a test image;
extract features from the test image;
match the features extracted from the test image with features stored in a library database;
if a match is obtained, perform one or more transformations on the test image to generate one or more pose-altered images;
extract features from the one or more pose-altered images; and
match the features extracted from the one or more pose-altered images with features stored in the library database.
12. The system of claim 11 wherein the software further causes the system to:
receive one or more scores from the matching of features extracted from the one or more pose-altered images; and
submit the one or more scores to a neural network trained to determine the probability of a match between the test image and an image in the library database, based on the one or more scores.
13. The system of claim 12 , the software further causing the system to:
pose correct the test image to eliminate portions of the image containing views of sides of an item other than a frontal side.
14. The method of claim 11 , the software further causing the system to:
receive one or more scores from the matching of features extracted from the one or more pose-altered images;
aggregate the one or more scores; and
calculate a probability of a match between the test image in an image in the library database based on the aggregated scores.
15. The system of claim 11 wherein the pose-altered images include views modified from the one or more 2D test images, the modifying including presenting items depicted in the image at arbitrary angles, rotating the items, translating the items, crumpling the items, modifying the colors of the items, over-exposing the items and under-exposing the items.
16. The system of claim 11 wherein the matching is performed by a trained classifier.
17. The method of claim 11 wherein the software causes the transformation of the test image by causing the system to:
generate a 3D model of the test image; and
generate pose-altered images showing different viewpoints of the image by rotating the 3D model along one or more axes.
18. The system of claim 11 wherein the software causes the transformation of the test image by causing the system to:
fix the test image to a plane with a known depth; and
generate pose-altered images showing different viewpoints of the image by rotating the test image along one or more of the axes of the plane.
19. The system of claim 11 wherein the software causes the transformation of the test image by causing the system to:
use planar homography by fitting corner endpoints of the test image to different configurations to simulate pose-altered views of the image.
20. The method of claim 11 wherein the software causes the transformation of the test image by causing the system to:
train a machine learning model to generate pose-altered images based on the test image; and
input the test image of the item to the machine learning model.