Methods and systems for ranking images using semantic and aesthetic models

1. A method for extracting one or more images from a storage medium, the method comprising:

determining a semantic class from a query based on a natural language processing technique;

checking an availability of a semantically related aesthetic model for the determined semantic class, wherein the semantically related aesthetic model corresponds to an aesthetic model that is configured to compute an aesthetic score for a first image that is associated with a predetermined semantic class, and wherein the semantically related aesthetic model is unable to compute aesthetic score for a second image that is associated with a semantic class other than the predetermined semantic class;

selecting a first search model or a second search model based on the availability of the semantically related aesthetic model, wherein the first search model comprises a semantic model and the semantically related aesthetic model, wherein the second search model comprises the semantic model and a generic aesthetic model, wherein the first search model is selected when the semantically related aesthetic model is available for the determined semantic class, and wherein the second search model is selected when the semantically related aesthetic model is unavailable for the determined semantic class;

computing a semantic score and an aesthetic score for each of the one or more images based on the selected search model; and

ranking the one or more images based on the semantic score and the aesthetic score.

2. The method of claim 1 , wherein the availability of the semantically related aesthetic model is checked based on semantics of the query.

3. The method of claim 1 , wherein the semantic model and the semantically related aesthetic model are trained on a sample set of images, wherein each image in the sample set of images has an associated semantic tag and an aesthetic tag.

4. The method of claim 1 , wherein the semantic score is computed based on the semantic model.

5. The method of claim 1 , wherein the aesthetic score is computed based on the semantically related aesthetic model and the semantic model.

6. The method of claim 5 , wherein the semantically related aesthetic model computes the aesthetic score based on one or more image features and the determined semantic class.

7. The method of claim 6 , wherein the one or more image features comprise low level features and high level features.

8. The method of claim 7 , wherein the low level features comprises at least one of SIFT features, colors, brightness, contrast, or Local Binary Patterns (LBP).

9. The method of claim 7 , wherein the high level features comprises at least one of Fisher Vectors, Supervector, VLAD, or sparse coding.

10. The method of claim 1 , wherein the aesthetic score is computed based on the generic aesthetic model.

11. The method of claim 10 , wherein the generic aesthetic model computes the aesthetic score for the one or more images based on an image feature associated with each of the one or more images, and wherein the generic aesthetic model computes the aesthetic score independent of the determined semantic class associated with each of the one or more images.

12. The method of claim 1 further comprises computing a combined score based on the aesthetic score and the semantic score, wherein the ranking is performed based on the combined score.

13. The method of claim 1 further comprising training the semantic model, the semantically related aesthetic mode, and the generic aesthetic model using at least one of large margin classifiers, deep neural networks, and support vector machines (SVM).

14. A method for extracting one or more images from a storage medium, the method comprising:

selecting a semantic model based on a semantic class determined from a query;

checking an availability of a semantically related aesthetic model for the determined semantic class, wherein the semantically related aesthetic model corresponds to an aesthetic model that is configured to compute an aesthetic score for a first image that is associated with a predetermined semantic class, and wherein the semantically related aesthetic model is unable to compute aesthetic score for a second image that is associated with a semantic class other than the predetermined semantic class;

selecting one of the semantically related aesthetic model or a generic aesthetic model based on the availability of the semantically related aesthetic model for the determined semantic class, wherein the semantically related aesthetic model is selected if available, and wherein the generic aesthetic model is selected when the semantically related aesthetic model is unavailable for the semantic class;

computing a semantic score and an aesthetic score for each of the one or more images based on the semantic model and the selected aesthetic model, respectively; and

ranking the one or more images based on the semantic score and the aesthetic score.

15. The method of claim 14 , wherein the aesthetic score is computed by the semantically related aesthetic model based on one or more image features associated with the one or more images and the determined semantic class.

16. The method of claim 14 , wherein the aesthetic score is computed by the generic aesthetic model based on one or more image features associated with the one or more images independent of the determined semantic class.

17. The method of claim 14 further comprises computing a combined score based on the aesthetic score and the semantic score, wherein the ranking of the one or more images is based on the combined score.

18. A system for extracting one or more images from a storage medium, the system comprising:

a processor configured to:

determine a semantic class from a query based on a natural language processing technique;

check an availability of a semantically related aesthetic model for the determined semantic class determined from a query, wherein the semantically related aesthetic model corresponds to an aesthetic model that is configured to compute an aesthetic score for a first image that is associated with a predetermined semantic class, and wherein the semantically related aesthetic model is unable to compute aesthetic score for a second image that is associated with a semantic class other than the predetermined semantic class;

select a first search model or a second search model based on the availability of the semantically related aesthetic model, wherein the first search model comprises a semantic model and the semantically related aesthetic model, wherein the second search model comprises the semantic model and a generic aesthetic model, wherein the first search model is selected when the semantically related aesthetic model is available for the determined semantic class, and wherein the second search model is selected when the semantically related aesthetic model is unavailable for the determined semantic class;

compute a semantic score and an aesthetic score for each of the one or more images based on the selected search model; and

rank the one or more images based on the semantic score and the aesthetic score.

19. The system of claim 18 , wherein the first search model computes the semantic score based on the semantic model.

20. The system of claim 18 , wherein the first search model utilizes the semantically related aesthetic model to compute the aesthetic score based on one or more image features associated with each of the one or more images and the determined semantic class.

21. The system of claim 18 , wherein the second search model comprises a semantic model and the generic aesthetic model.

22. The system of claim 21 , wherein the second search model computes the aesthetic score by utilizing the generic aesthetic model based on one or more image features associated with each of the one or more images, wherein the generic aesthetic model computes the aesthetic score independent of the determined semantic class.

23. The system of claim 18 , wherein the is further configured to compute a combined score based on the aesthetic score and the semantic score, wherein the ranking of the one or more images are based on the combined score.

24. The system of claim 18 , wherein the system is implementable on at least one of an elastic cloud computing architecture.

25. A computer program product for use with a computer, the computer program product comprising a non-transitory computer-usable data carrier storing a computer readable program code embodied therein for extracting one or more images from a storage medium, the computer readable program code comprising:

a program instruction means for determining a semantic class from a query based on a natural language processing technique;

a program instruction means for checking an availability of a semantically related aesthetic model for the determined semantic class, wherein the semantically related aesthetic model corresponds to an aesthetic model that is configured to compute an aesthetic score for a first image that is associated with a predetermined semantic class, and wherein the semantically related aesthetic model is unable to compute aesthetic score for a second image that is associated with a semantic class other than the predetermined semantic class;

a program instruction means for selecting a first search model or a second search model based on the availability of the semantically related aesthetic model, wherein the first search model comprises a semantic model and the semantically related aesthetic model, wherein the second search model comprises the semantic model and a generic aesthetic model, wherein the first search model is selected when the semantically related aesthetic model is available for the determined semantic class, and wherein the second search model is selected when the semantically related aesthetic model is unavailable for the determined semantic class;

a program instruction means for computing a semantic score and an aesthetic score for each of the one or more images based on the selected search model; and

a program instruction means for ranking the one or more images based on the semantic score and the aesthetic score.