System and method for identifying misplaced products in a shelf management system

1. A computer-implemented method comprising:

obtaining an image containing multiple objects using a camera;

obtaining identifying information associated with a region of interest in the image;

extracting an object image from the image, the object image containing an individual object located in the region of interest;

extracting features from the object image using a trained feature extractor;

determining a best-fit match between features extracted from the object image and features associated with objects in an object library;

receiving an identifier associated with the best-fit match;

determining that the best-fit match identifier does not match the identifying information associated with the region of interest; and

indicating that the object in the object image is not associated with the other objects in the region of interest;

wherein the method is implemented in software executing on a processor; and

wherein the trained feature extractor is a deep neural network trained to on a dataset comprising multiple views of each object and associated identifying information of each object.

2. The method of claim 1 wherein the method is repeated for each object image detected in the image.

3. The method of claim 1 wherein the object library is built by a method comprising:

obtaining a source image and identifying information for each object;

acquiring multiple images of each object from a plurality of sources;

ranking the acquired images based on a highest confidence in an association between the acquired images and the source image;

selecting a pre-determined number of top-ranked acquired images; and

storing features extracted from the source image and the top-ranked acquired images, and the identifying information associated with the source image and the top-ranked acquired images, in the object library.

4. The method of claim 3 wherein the acquired multiple images for each object include images exhibiting different variations and/or viewpoints for each object.

5. The method of claim 3 wherein the feature extractor is trained on an object ID dataset comprising multiple images of each object and associated identifying information.

6. The method of claim 5 wherein the feature extractor outputs the identifying information, given an image of the object as input.

7. The method of claim 5 wherein the multiple images of each object include images of the object exhibiting pose variations.

8. The method of claim 5 wherein the multiple images of each object include images of the object exhibiting variations in labelling of the object.

9. The method of claim 5 wherein the multiple images of each object include images of the object associated with different identifying information.

10. The method of claim 1 further comprising:

determining that the identifier associated with an object image does not exist in the object library, indicating a new object; and

enrolling features extracted from the object image and the associated identifier in the object library.

11. The method of claim 1 further comprising:

determining that the features extracted from the object image are not a best fit with the features in the object library;

determining that the identifier associated with an object image does exist in the object library;

determining that the object image represents an existing object with new labelling; and

associating features extracted from the object image with the existing identifying information in the object library.

12. The method of claim 1 further comprising:

determining that the best fit match matches identifying information from an adjacent region of interest; and

identifying the individual object as a spread.

13. A system comprising:

a camera, for obtaining images containing a plurality of objects;

a processor, executing software for analyzing the images;

a feature extractor trained to extract features from an object in the image and output identifying information associated with the object; and

an object library containing features extracted from multiple objects within the image, the features associated with identifying information of the object;

wherein the software performs the functions of:

obtaining an image containing multiple objects using the camera;

obtaining identifying information associated with a region of interest in the image;

extracting an object image from the image, the object image containing an individual object located in the region of interest;

determining a best-fit match between features extracted from the object image and features in the object library;

receiving an object identifier associated with the best-fit match; and

determining that the object identifier associated with the best fit match does not match the identifying information associated with the region of interest; and

indicating that the individual object is not associated with other objects in the region of interest;

wherein the feature extractor is a deep neural network trained to on a dataset comprising multiple views of each object and associated identifying information of each object.

14. The system of claim 13 wherein the object library is built by:

obtaining a source image and identifying information for each object;

acquiring multiple images of each object from a plurality of sources;

ranking the acquired images based on a highest confidence in an association between the acquired images and the source image;

selecting a pre-determined number of the top-ranked acquired images; and

storing features extracted from the source image and the top-ranked acquired images, and the identifying information associated with the source image and the top-ranked acquired images, in the object library.

15. The system of claim 14 wherein the acquired multiple images for each object include images exhibiting different variations and/or viewpoints for each object.