Individual and user group attributes discovery and comparison from social media visual content
View Patent ↗A method and system are provided. The method includes deriving a set of user attributes from an aggregate analysis of images and videos of a user. The deriving step includes recognizing, by a set of visual classifiers, semantic concepts in the images and videos of the user to generate visual classifier scores. The deriving step further includes deriving, by a statistical aggregator, the set of user attributes. The set of user attributes are derived by mapping the visual classifier scores to a taxonomy of semantic categories to be recognized in visual content. The deriving step also includes displaying, by an interactive user interface having a display, attribute profiles for the attributes and comparisons of the attribute profiles.
1. A method, comprising:
deriving a set of user attributes from an aggregate analysis of images and videos of a user by (i) recognizing, by a set of visual classifiers, semantic concepts in the images and videos of the user to generate visual classifier scores; and (ii) deriving, by a statistical aggregator, the set of user attributes by mapping the visual classifier scores to a taxonomy of semantic categories to be recognized in visual content; and
providing an interactive user interface, having at least one user input element and a display element, configured to interactively display attribute profiles for the attributes and comparisons of the attribute profiles.
2. The method of claim 1 , wherein the comparisons of the attribute profiles are provided using a differential analyzer, the differential analyzer using one or more distribution comparison metrics to provide the comparisons of the attribute profiles.
3. The method of claim 2 , wherein distributions of the visual classifier scores between any of single users and groups of users are compared by the differential analyzer to provide the comparisons of the attribute profiles.
4. The method of claim 1 , further comprising training an attribute prediction system using one or more distributions of the visual classifier scores from multiple labeled users.
5. The method of claim 1 , wherein the aggregate analysis is performed over the images and videos from any of, one or more single users and one or more groups of users.
6. The method of claim 1 , wherein the aggregation analysis is performed over any of images and videos taken at, at least one of, different times and different locations.
7. The method of claim 1 , wherein the images and videos are accessed via at least one of a users' social media stream, a phone, a tablet, and a computer gallery.
8. The method of claim 1 , wherein the taxonomy of semantic categories is configured to model diverse topics.
9. The method of claim 1 , wherein the set of visual classifiers are learned from labeled data, wherein labels for the labeled data are provided by at least one of annotation, tagging, and crowd-sourcing.
10. The method of claim 1 , further comprising pre-training the set of visual classifiers on an image dataset of visual classes, independently from any of the user attributes.
11. A non-transitory article of manufacture tangibly embodying a computer readable program which when executed causes a computer to perform the steps of claim 1 .
12. A system, comprising:
an aggregate analyzer for deriving a set of user attributes from an aggregate analysis of images and videos of a user,
wherein said aggregate analyzer includes:
a set of visual classifiers for recognizing semantic concepts in the images and videos of the user to generate visual classifier scores; and
a statistical aggregator for deriving the set of user attributes, wherein the set of user attributes are derived by mapping the visual classifier scores to a taxonomy of semantic categories to be recognized in visual content; and
an interactive user interface having at least one user input element and a display element for interactively displaying attribute profiles for the attributes and comparisons of the attribute profiles.
13. The system of claim 12 , further comprising a differential analyzer for providing the comparisons of the attribute profiles, the differential analyzer using one or more distribution comparison metrics to provide the comparisons of the attribute profiles.
14. The system of claim 13 , wherein distributions of the visual classifier scores between any of single users and groups of users are compared by the differential analyzer to provide the comparisons of the attribute profiles.
15. The system of claim 12 , wherein the aggregate analysis is performed over the images and videos from any of, one or more single users and one or more groups of users.
16. The system of claim 12 , wherein the aggregation analysis is performed over any of images and videos taken at, at least one of, different times and different locations.
17. The system of claim 12 , wherein the images and videos are accessed via at least one of a users' social media stream, a phone, a tablet, and a computer gallery.
18. The system of claim 12 , wherein the taxonomy of semantic categories is configured to model diverse topics.
19. The system of claim 12 , wherein the set of visual classifiers are learned from labeled data, wherein labels for the labeled data are provided by at least one of annotation, tagging, and crowd-sourcing.
20. The system of claim 12 , wherein the set of visual classifiers are pre-trained on an image dataset of visual classes, independently from any of the user attributes.