Automatic handbag recognition using view-dependent CNNs
A system and related methods for identifying characteristics of handbags is described. One method includes receiving one or more images of a handbag, eliminating all but select images from the one or more images of the handbag to obtain a grouping of one or more select images, the select images being those embodying a complete periphery and frontal view of the handbag. For each of the one or more select images, aligning feature-corresponding pixels with an image axis, comparing at least a portion of the one or more select images with a plurality of stored images, and determining characteristics of the handbag based on said comparing.
1. A method for identifying characteristics of a handbag, the method comprising in any order:
receiving one or more images of a handbag;
eliminating all but select images from the one or more images of the handbag to obtain a grouping of one or more select images, the select images being those embodying a complete periphery and frontal view of the handbag;
for each of the one or more select images, aligning feature-corresponding pixels with an image axis;
comparing at least a portion of the one or more select images with a plurality of stored images; and
determining said characteristics of the handbag based on said comparing;
wherein the eliminating all but select images from the one or more images of the handbag comprises:
executing a frontal view classifier, the frontal view classifier is trained by:
collecting a first dataset, the first dataset comprises a plurality of images, each of the plurality of images comprises one of a plurality of handbags,
labelling the plurality images that embody a complete periphery and frontal view as select,
training a first model with the first dataset,
collecting incorrect predictions from the first model, the collection of the incorrect predictions results in a second dataset, and
training a second model with the first and second datasets, and
identifying each of the one or more images that are select.
2. The method of claim 1 , wherein the characteristics comprise a brand identifier and a style identifier.
3. The method of claim 1 , the steps further comprising transforming the one or more images of the handbag or the one or more select images from a colorscale to a grayscale image.
4. The method of claim 1 , wherein said comparing the at least a portion of the one or more select images with a plurality of stored images comprises:
for each of the one or more select images, extracting a plurality of style-identifying features from the handbag contained therein; and
comparing the plurality of style-identifying features with a plurality of stored-image features from each of the plurality of stored images.
5. A computer system comprising:
a processor; and
a non-transitory computer-readable medium configured to store instructions, the instructions when executed by the processor cause the processor to perform steps comprising:
receiving one or more images of a handbag;
eliminating all but select images from the one or more images of the handbag to obtain a grouping of one or more select images, the select images being those embodying a complete periphery and frontal view of the handbag;
for each of the one or more select images, aligning feature-corresponding pixels with an image axis;
comparing at least a portion of the one or more select images with a plurality of stored images; and
determining characteristics of the handbag based on said comparing;
wherein the eliminating all but select images from the one or more images of the handbag comprises:
executing a frontal view classifier, the frontal view classifier is trained by:
collecting a first dataset, the first dataset comprises a plurality of images, each of the plurality of images comprises one of a plurality of handbags,
labelling the plurality images that embody a complete periphery and frontal view as select,
training a first model with the first dataset,
collecting incorrect predictions from the first model, the collection of the incorrect predictions results in a second dataset, and
training a second model with the first and second datasets, and
identifying each of the one or more images that are select.
6. The computer system of claim 5 , wherein the characteristics comprise a brand identifier and a style identifier.
7. The computer system of claim 5 , the steps further comprising transforming the one or more images of the handbag or the one or more select images from a colorscale to a grayscale image.
8. The computer system of claim 5 , wherein said comparing at least a portion of the one or more select images with the plurality of stored images comprises:
for each of the one or more select images, extracting a plurality of style-identifying features from the handbag contained therein; and
comparing the plurality of style-identifying features with a plurality of stored-image features from each of the plurality of stored images.
9. A non-transitory computer-readable medium configured to store instructions, the instructions when executed by one or more computers, cause the one or more computers to perform operations comprising:
receiving one or more images of a handbag;
eliminating all but select images from the one or more images of the handbag to obtain a grouping of one or more select images, the select images being those embodying a complete periphery and frontal view of the handbag;
for each of the one or more select images, aligning feature-corresponding pixels with an image axis;
comparing at least a portion of the one or more select images with a plurality of stored images; and
determining characteristics of the handbag based on said comparing;
wherein the eliminating all but select images from the one or more images of the handbag comprises:
executing a frontal view classifier, the frontal view classifier is trained by:
collecting a first dataset, the first dataset comprises a plurality of images, each of the plurality of images comprises one of a plurality of handbags,
labelling the plurality images that embody a complete periphery and frontal view as select,
training a first model with the first dataset,
collecting incorrect predictions from the first model, the collection of the incorrect predictions results in a second dataset, and
training a second model with the first and second datasets, and
identifying each of the one or more images that are select.
10. The non-transitory computer-readable medium of claim 9 , wherein the characteristics comprise a brand identifier and a style identifier.
11. The non-transitory computer-readable medium of claim 9 , the steps further comprising transforming the one or more images of the handbag or the one or more select images from a colorscale to a grayscale image.
12. The non-transitory computer-readable medium of claim 9 , wherein said comparing the at least a portion of the one or more select images with a plurality of stored images comprises:
for each of the one or more select images, extracting a plurality of style-identifying features from the handbag contained therein; and
comparing the plurality of style-identifying features with a plurality of stored-image features from each of the plurality of stored images.