Object re-identification using self-dissimilarity
A method of identifying an object in an image is disclosed. At least one feature map for each of a plurality of cells in the image is determined. A self-dissimilarity between a first feature map associated with a first one of said cells and a second feature map associated with a second cell, is determined. The self-dissimilarity is determined by determining a sum over thresholds of a difference in area between the first feature map and the second feature map. An appearance signature for the object is formed based on the determined self-dissimilarity. A distance between the appearance signature of the object in the image and appearance signatures of each of a plurality of further objects is determined. The object in the image is identified based on the determined distances.
1. A method of identifying an object in an image, the method comprising:
determining at least one feature map for each of a plurality of cells in the image;
determining, via photometric invariant self-dissimilarity matching, a self-dissimilarity between a first feature map associated with a first one of the cells and a second feature map associated with a second cell, wherein the self-dissimilarity is determined by determining a sum over thresholds of a difference in area between the first feature map and the second feature map;
forming an appearance signature for the object based on the determined self-dissimilarity;
determining a distance between the appearance signature of the object in the image and appearance signatures of each of a plurality of further objects; and
identifying the object in the image based on the determined distances.
2. The method according to claim 1 , wherein the sum over thresholds is determined using a thresholded first feature map and a thresholded second feature map.
3. The method according to claim 1 , wherein the sum over thresholds of a difference in area between the first feature map and the second feature map is determined as the sum over thresholds for which the first feature map has a larger area than the second feature map.
4. The method according to claim 1 , wherein the sum over thresholds of a difference in area between the first feature map and second feature map is determined as the sum over thresholds for which the second feature map has a larger area than the first feature map.
5. The method according to claim 1 , wherein the cells are rectangular regions defined at multiple scales and aspect ratios.
6. The method according to claim 1 , wherein the feature maps include intensity, chrominance, hue, saturation, opponent colours, image gradients, Gabor filter responses or texture filters.
7. The method according to claim 1 , further including determining, for at least one cell, a self-dissimilarity between a determined feature map and a canonical feature map.
8. The method according to claim 7 , wherein the canonical feature map has a cumulative distribution corresponding to a step function, with a step at a center of a range of feature values.
9. The method according to claim 1 , wherein forming an appearance signature includes applying a soft threshold to determined signed dissimilarities, wherein the soft threshold is applied to the determined signed dissimilarities using the following soft threshold equation:
{tilde over (s)} i =1−exp(− s i /σ),
where each value s i , i=1, 2, . . . , 2N in the formed appearance signature is replaced with a new value {tilde over (s)} i , and a strength of the soft threshold is determined or predetermined by the parameter σ.
10. The method according to claim 1 , wherein the photometric invariant self-dissimilarity matching is performed such that relative differences between cells are invariant to photometric change.
11. The method according to claim 1 , wherein the first feature map and the second feature map each have a feature distribution that is defined or estimated by at least one of: a normalized histogram of feature values in the respective feature map, Kernal Density Estimation (KDE) based on feature values in the respective feature map, and a Gaussian Mixture Model (GMM) based on pixel values in the respective feature map.
12. The method according to claim 1 , further comprising determining a first feature distribution from the first feature map and a second feature distribution from the second feature map.
13. The method according to claim 12 , further comprising quantifying or determining a difference between the first feature distribution and the second feature distribution.
14. The method according to claim 13 , wherein the quantifying or determining includes using an Earth mover's distance (EMD).
15. The method according to claim 14 , further comprising computing or determining a signed EMD between the first feature map and the second feature map.
16. The method according to claim 15 , wherein the formation of the appearance signature occurs after the signed EMD is computed or determined.
17. The method according to claim 1 , further comprising using a foreground confidence mask to assign to each pixel of the image a value indicating a confidence that the pixel belongs to the object.
18. A system for identifying an object in an image, the system comprising:
a memory for storing data and a computer program;
at least one processor coupled to the memory for executing the computer program, the at least one processor operating to:
determine at least one feature map for each of a plurality of cells in the image;
determine, via photometric invariant self-dissimilarity matching, a self-dissimilarity between a first feature map associated with a first one of the cells and a second feature map associated with a second cell, wherein the self-dissimilarity is determined by determining a sum over thresholds of a difference in area between the first feature map and the second feature map;
form an appearance signature for the object based on the determined self-dissimilarity;
determine a distance between the appearance signature of the object in the image and appearance signatures of each of a plurality of further objects; and
identify the object in the image based on the determined distances.
19. An apparatus for identifying an object in an image, the apparatus comprising:
means for determining at least one feature map for each of a plurality of cells in the image;
means for determining, via photometric invariant self-dissimilarity matching, a self-dissimilarity between a first feature map associated with a first one of the cells and a second feature map associated with a second cell, wherein the self-dissimilarity is determined by determining a sum over thresholds of a difference in area between the first feature map and the second feature map;
means for forming an appearance signature for the object based on the determined self-dissimilarities;
means for determining a distance between the appearance signature of the object in the image and appearance signatures of each of a plurality of further objects; and
means for identifying the object in the image based on the determined distances.
20. A non-transitory computer-readable storage medium storing at least one program that causes a processor to execute a method of identifying an object in an image, the method comprising:
determining at least one feature map for each of a plurality of cells in the image;
determining, via photometric invariant self-dissimilarity matching, a self-dissimilarity between a first feature map associated with a first one of the cells and a second feature map associated with a second cell, wherein the self-dissimilarity is determined by determining a sum over thresholds of a difference in area between the first feature map and the second feature map;
forming an appearance signature for the object based on the determined self-dissimilarity;
determining a distance between the appearance signature of the object in the image and appearance signatures of each of a plurality of further objects; and
identifying the object in the image based on the determined distances.