System and method for identity preservative representation of persons and objects using spatial and appearance attributes

1. A method for processing images of persons or objects to generate an identity preservative feature descriptor learnt for each person or object, the method comprising:

obtaining an image of a person or object;

extracting, without reference to another image, at least one spatial attribute of the person or object from the obtained image, wherein the extracted at least one spatial attribute corresponds to spatial characteristics of the person or object within the obtained image;

extracting at least one appearance feature of the person or object from the obtained image by using a mapping function to translate image pixels into appearance attributes represented by at least one numerical feature; and

using a fusion mechanism to generate informative features by combining the at least one spatial attribute and the at least one appearance feature, the generated informative features being feature descriptors representing the person or the object, to assign the feature descriptors to the obtained image to enable feature descriptors representing the same person or object to be compared to feature descriptors representing different people or objects in a different image given a predefined mathematical pseudo-distance metric according to a least a distance from each other.

2. The method of claim 1 , further comprising using the at least one spatial attribute of the person or object in learning how to extract the at least one appearance feature.

3. The method of claim 2 , wherein the learning applies a gated fusion process.

4. The method of claim 2 , wherein the learning filters out irrelevant information to focus on the appearance attributes that represent only the person or object of interest.

5. The method of claim 1 , wherein the mapping function is learnt using at least one machine learning technique and at least one labelled datapoint.

6. The method of claim 5 , wherein the at least one machine learning technique uses a mathematical pseudo-distance function to maximize a similarity between the identity preservative feature descriptors extracted from different observations of the same person or object.

7. The method of claim 5 , wherein the at least one machine learning technique uses a mathematical pseudo-distance function to minimize a similarity between the identity preservative feature descriptors extracted from observations of different persons or objects.

8. The method of claim 1 , wherein the identity preservative feature descriptors are used for person or object reidentification.

9. The method of claim 1 , wherein the identity preservative feature descriptors are used to track persons or objects by associating observations of the same person or object across different images in a video, to generate a trajectory of a target corresponding to the person or object of interest.

10. The method of claim 1 , further comprising:

receiving the image or a sequence of images of a person or an object as a query image;

generating the identity preservative feature descriptors for the query image; and

comparing the identity preservative feature descriptors for the query image to a gallery of feature descriptors generated for persons or objects with known identities to identify and/or reidentify the person or object in the query image.

11. The method of claim 1 , further comprising calculating a similarity or dissimilarity score using the mathematical pseudo-distance metric between the query image feature descriptors and images in a gallery with known identification.

12. The method of claim 1 , wherein human body pose information and/or a body skeleton is use as the spatial attribute used for a person identification or reidentification.

13. A non-transitory computer readable medium comprising computer executable instructions for processing images of persons or objects to generate an identity preservative feature descriptor learnt for each person or object, comprising instructions for performing the method of claim 1 .

14. An image processing system for processing images of persons or objects to generate an identity preservative feature descriptor learnt for each person or object, the system comprising a processor and memory, the memory storing computer executable instructions that, when implemented by the processor, cause the image processing system to:

obtain an image of a person or object;

extract, without reference to another image, at least one spatial attribute of the person or object from the obtained image, wherein the extracted at least one spatial attribute corresponds to spatial characteristics of the person or object within the obtained image;

extract at least one appearance feature of the person or object from the obtained image by using a mapping function to translate image pixels into appearance attributes represented by at least one numerical feature; and

use a fusion mechanism to generate informative features by combine the at least one spatial attribute and the at least one appearance feature, the generated informative features being feature descriptors representing the person or the object, to assign the feature descriptors to the obtained image to enable feature descriptors representing the same person or object in a different image to be compared to feature descriptors representing different people or objects given a predefined mathematical pseudo-distance metric according to a least a distance from each other.

15. The system of claim 14 , further comprising using the at least one spatial attribute of the person or object in learning how to extract the at least one appearance feature.

16. The system of claim 15 , wherein the learning applies a gated fusion process.

17. The system of claim 15 , wherein the learning filters out irrelevant information to focus on the appearance attributes that represent only the person or object of interest.

18. The system of claim 14 , wherein the mapping function is learnt using at least one machine learning technique and at least one labelled datapoint.

19. The system of claim 18 , wherein the at least one machine learning technique uses a mathematical pseudo-distance function to maximize a similarity between the identity preservative feature descriptors extracted from different observations of the same person or object.

20. The system of claim 18 , wherein the at least one machine learning technique uses a mathematical pseudo-distance function to minimize a similarity between the identity preservative feature descriptors extracted from observations of different persons or objects.

21. The system of claim 14 , wherein the identity preservative feature descriptors are used for person or object reidentification.

22. The system of claim 14 , wherein the identity preservative feature descriptors are used to track persons or objects by associating observations of the same person or object across different images in a video, to generate a trajectory of a target corresponding to the person or object of interest.

23. The system of claim 14 , further comprising:

receiving the image or a sequence of images of a person or an object as a query image;

generating the identity preservative feature descriptors for the query image; and

comparing the identity preservative feature descriptors for the query image to a gallery of feature descriptors generated for persons or objects with known identities to identify and/or reidentify the person or object in the query image.

24. The system of claim 14 , further comprising calculating a similarity or dissimilarity score using the mathematical pseudo-distance metric between the query image feature descriptors and images in a gallery with known identification.

25. The system of claim 14 , wherein human body pose information and/or a body skeleton is use as the spatial attribute used for a person identification or reidentification.