Techniques for enhancing content memorability of user generated video content
Techniques are described for analyzing a video for memorability, identifying content features of the video that are likely to be memorable, and scoring specific content features within the video for memorability. The techniques can be optionally applied to selected features in the video, thus improving the memorability of the selected features. The features may be organic features of the originally captured video or add-in features provided using an editing tool. The memorability of video features, text features, or both can be improved by analyzing the effects of applying different styles or edits (e.g., sepia tone, image sharpen, image blur, annotation, addition of object) to the content features or to the video in general. Recommendations can then be provided regarding memorability score caused by application of the image styles to the video features.
1. A computer-implemented method for quantifying memorability of video content, the method comprising:
receiving a video that includes a content feature that comprises a video feature that is associated with a text feature;
identifying the video feature and the associated text feature in the received video; and
determining:
a video feature score corresponding to the video feature, the video feature score indicating memorability of the video feature;
a text feature score corresponding to the text feature, the text feature score indicating memorability of the corresponding text feature;
a similarity metric quantifying a semantic similarity between the video feature and the associated text feature; and
a content memorability score that is based on at least the similarity metric the video feature score and the text feature score.
2. The computer-implemented method of claim 1 , further comprising at least one of:
normalizing the similarity metric to a value between 0 and 1 prior to determining the content memorability score;
responsive to determining the video and text feature scores, determining whether the content memorability score is above a threshold; and
presenting the content feature in a user interface that includes a memorability map highlighting a temporal location of the content feature in the received video.
3. The computer-implemented method of claim 1 , further comprising using at least one of the video feature score and the text feature score to determine the content memorability score for the received video.
4. The computer-implemented method of claim 3 , further comprising:
applying an edit to the received video;
determining a revised content memorability score based on the edit; and
presenting the revised content memorability score.
5. The computer-implemented method of claim 1 , further comprising:
editing the identified video feature;
determining a revised video feature score corresponding to the edited video feature; and
determining a revised content memorability score using the revised video feature score.
6. The computer-implemented method of claim 1 , wherein the determining the video feature score includes analyzing the video feature with a deep neural network learning algorithm to identify a semantic meaning of the video feature.
7. The computer-implemented method of claim 1 , further comprising:
editing the text feature;
determining a revised text feature score corresponding to the edited text feature; and
determining a revised content memorability score using the revised text feature score.
8. A computer program product for quantifying memorability of video content, the computer program product comprising a non-transitory computer-readable storage medium containing computer program code that, when executed by one or more processors, performs a process, the process comprising:
receiving a video that includes a content feature that comprises a video feature that is associated with a text feature;
identifying the video feature and the associated text feature in the received video; and
determining:
a video feature score corresponding to the video feature, the video feature score indicating memorability of the video feature;
a text feature score corresponding to the text feature, the text feature score indicating memorability of the corresponding text feature;
a similarity metric quantifying a semantic similarity between the video feature and the associated text feature; and
a content memorability score that is based on at least the similarity metric, the video feature score and the text feature score.
9. The computer program product of claim 8 , the process further comprising at least one of:
normalizing the similarity metric to a value between 0 and 1 prior to determining the content memorability score;
responsive to determining the video and text feature scores, determining whether the content memorability score is above a threshold; and
presenting the content feature in a user interface that includes a memorability map highlighting a temporal location of the content feature in the received video.
10. The computer program product of claim 8 , the process further comprising using at least one of the video feature score and the text feature score to determine the content memorability score for the received video.
11. The computer program product of claim 8 , the process further comprising:
editing the video feature;
determining a revised video feature score corresponding to the edited video feature; and
determining a revised content memorability score using the revised video feature score.
12. The computer program product of claim 8 , wherein the determining the video feature score includes analyzing the video feature with a deep neural network learning algorithm to identify a semantic meaning of the video feature.
13. The computer program product of claim 8 , the process further comprising:
editing the text feature;
determining a revised text feature score corresponding to the edited text feature; and
determining a revised content memorability score using the revised text feature score.
14. A system for quantifying memorability of video content, the system comprising:
a web server configured for receiving a video that includes a content feature that comprises a video feature that is associated with a text feature;
a content feature identifier configured for identifying the video feature and the associated text feature in the received video;
a scoring module configured for determining:
a video feature score corresponding to the video feature, the video feature score indicating memorability of the video feature; and
a text feature score corresponding to the text feature, the text feature score indicating memorability of the corresponding text feature; and
a text and image comparison module configured for determining a similarity metric quantifying a semantic similarity between the video feature and the associated text feature;
wherein the scoring module is further configured for determining a content memorability score that is based on at least the similarity metric, the video feature score and the text feature score.
15. The system of claim 14 , wherein the text and image comparison module is further configured for normalizing the similarity metric to a value between 0 and 1.
16. The system of claim 14 ,
wherein the text and image comparison module is further configured for normalizing the similarity metric to a value between 0 and 1; and
wherein the scoring module is further configured for determining the content memorability score as a function of the normalized similarity metric, the video feature score and the text feature score.
17. The system of claim 14 , wherein the content feature identifier is further configured for analyzing the video feature with a deep neural network learning algorithm to identify a semantic meaning of the video feature.