Localizing relevant objects in multi-object images

1. A method of localizing relevant objects in multi-object images, the method comprising:

receiving a multi-object image;

detecting a plurality of detected objects within the multi-object image;

generating a primary heatmap for the multi-object image, the primary heatmap having at least one region of interest;

determining a first relevant detected object corresponding to a region of interest in the primary heatmap;

determining an irrelevant detected object not corresponding to a region of interest in the primary heatmap;

determining a class score for each of the first relevant detected object and the irrelevant detected object;

ranking the class scores, wherein the class score for the first relevant detected object exceeds the class score for the irrelevant detected object; and

indicating the first relevant detected object as an output result but not indicating the irrelevant detected object as an output result.

2. The method of claim 1 , further comprising:

identifying that a first region of interest in the primary heatmap does not correspond to a detected object of the plurality of detected objects; and

performing a second object detection process for a region of the multi-object image corresponding to the first region of interest in the primary heatmap, to detect the first relevant detected object.

3. The method of claim 1 , further comprising:

cropping the multi-object image to a bounding box around the first relevant detected object.

4. The method of claim 1 , further comprising:

identifying a plurality of visually similar images containing objects that are visually similar to the first relevant detected object; and

displaying the plurality of visually similar images to a user.

5. The method of claim 1 , further comprising:

determining a second relevant detected object corresponding to a second region of interest of the primary heatmap; and

indicating the second relevant detected object as a second output result.

6. The method of claim 1 , wherein the primary heatmap is generated using a gradient activation map (GAM) process.

7. The method of claim 1 , wherein generating the primary heatmap comprises:

generating a first heatmap for the multi-object image for a first object type;

generating a second heatmap for the multi-object image for a second object type; and

combining a region of interest of the first heatmap with a region of interest of the second heatmap.

8. A system for localizing relevant objects in multi-object images, the system comprising:

a processor; and

a computer-readable medium storing instructions that are operative upon execution by the processor to:

receive a multi-object image;

detect a plurality of detected objects within the multi-object image;

generate a primary heatmap for the multi-object image, the primary heatmap having at least one region of interest;

determine a first relevant detected object corresponding to a region of interest in the primary heatmap;

determine an irrelevant detected object not corresponding to a region of interest in the primary heatmap;

determine a class score for each of the first relevant detected object and the irrelevant detected object;

rank the class scores, wherein the class score for the first relevant detected object exceeds the class score for the irrelevant detected object; and

indicate the first relevant detected object as an output result but not indicate the irrelevant detected object as an output result.

9. The system of claim 8 , wherein the instructions are further operative to:

identify that a first region of interest in the primary heatmap does not correspond to a detected object of the plurality of detected objects; and

perform a second object detection process for a region of the multi-object image corresponding to the first region of interest in the primary heatmap, to detect the first relevant detected object.

10. The system of claim 8 , wherein the instructions are further operative to:

crop the multi-object image to a bounding box around the first relevant detected object.

11. The system of claim 8 , wherein the instructions are further operative to:

identify a plurality of visually similar images containing objects that are visually similar to the first relevant detected object; and

displaying the plurality of visually similar images to a user.

12. The system of claim 8 , wherein the instructions are further operative to:

determine a second relevant detected object corresponding to a second region of interest of the primary heatmap; and

indicate the second relevant detected object as a second output result.

13. The system of claim 8 , wherein the primary heatmap is generated using a gradient activation map (GAM) process.

14. The system of claim 8 , wherein generating the primary heatmap comprises:

generating a first heatmap for the multi-object image for a first object type;

generating a second heatmap for the multi-object image for a second object type; and

combining a region of interest of the first heatmap with a region of interest of the second heatmap.

15. One or more computer storage devices having computer-executable instructions stored thereon, which, on execution by a computer, cause the computer to perform operations comprising:

receiving a multi-object image;

detecting a plurality of detected objects within the multi-object image;

generating a primary heatmap for the multi-object image, the primary heatmap having at least one region of interest;

determining a first relevant detected object corresponding to a region of interest in the primary heatmap;

determining an irrelevant detected object not corresponding to a region of interest in the primary heatmap;

determining a class score for each of the first relevant detected object and the irrelevant detected object;

ranking the class scores, wherein the class score for the first relevant detected object exceeds the class score for the irrelevant detected object; and

indicating the first relevant detected object as an output result but not indicating the irrelevant detected object as an output result.

16. The one or more computer storage devices of claim 15 , wherein the operations further comprise

identifying that a first region of interest in the primary heatmap does not correspond to a detected object of the plurality of detected objects; and

performing a second object detection process for a region of the multi-object image corresponding to the first region of interest in the primary heatmap, to detect the first relevant detected object.

17. The one or more computer storage devices of claim 15 , wherein the operations further comprise

cropping the multi-object image to a bounding box around the first relevant detected object.

18. The one or more computer storage devices of claim 15 , wherein the operations further comprise

identifying a plurality of visually similar images containing objects that are visually similar to the first relevant detected object; and

displaying the plurality of visually similar images to a user.

19. The one or more computer storage devices of claim 15 , wherein the operations further comprise

determining a second relevant detected object corresponding to a second region of interest of the primary heatmap; and

indicating the second relevant detected object as a second output result.

20. The one or more computer storage devices of claim 15 , wherein the primary heatmap is generated using a gradient activation map (GAM) process.