Enriching online videos by content detection, searching, and information aggregation

1. A method of identifying digital content relevant to an online video, comprising:

identifying, by one or more processing units, promotional content embedded within the online video, the promotional content being generated by a first content producer;

extracting, by the one or more processing units, a feature from the promotional content embedded in the online video, the feature comprising at least one of a textual feature, a visual feature, or an audio feature;

searching, by the one or more processing units, a database to identify additional information based upon the feature, the additional information being generated by a second content producer different than the first content producer;

identifying, by the one or more processing units, candidates from the additional information, the candidates including at least two of a textual feature, a visual feature, or an audio feature;

aggregating, by the one or more processing units, the candidates into an aggregate video; and

outputting, by the one or more processing units, the aggregate video for presentation.

2. The method of claim 1 , wherein outputting the aggregate video for presentation comprises:

presenting the aggregate video and the promotional content sequentially.

3. The method of claim 1 , the visual feature comprising a color histogram.

4. The method of claim 1 , the extracting the feature comprising at least one of:

extracting the textual feature using an OCR text recognition technique; or

extracting the visual feature using a scale-invariant feature transformation.

5. The method of claim 1 , the searching the database to identify the additional information comprising:

executing a multi modal relevance matching algorithm against the database using the feature to determine the additional information.

6. The method of claim 1 , the aggregate video being formatted to be embedded in the online video.

7. The method of claim 1 , the extracting the feature comprising:

parsing the promotional content embedded in the online video into one or more segments, each of the one or more segments comprising a sequence of frames;

for respective segments, extracting one or more keyframes; and

extracting the feature from the one or more keyframes of respective segments.

8. The method of claim 7 , the extracting one or more keyframes comprising at least one of:

selecting a middle frame as a keyframe; or

selecting a frame having a video quality above a threshold as the keyframe.

9. The method of claim 1 , wherein the aggregate video comprises advertising information associated with the promotional content embedded within the online video.

10. A system for identifying digital content relevant to an online video, comprising:

one or more processing units; and

a memory storing:

a parsing component configured to, when executed by the one or more processing units:

select a frame of an online video as a keyframe based upon the frame having a video quality above a threshold;

a feature extraction component configured to, when executed by the one or more processing units:

identify promotional content embedded within the keyframe, the promotional content being generated by a first content producer; and

extract a feature from at least the promotional content embedded within the keyframe, the feature comprising at least one of a textual feature, a visual feature, or an audio feature;

an information extraction component configured to, when executed by the one or more processing units:

search a database to identify additional information based upon the feature, the additional information being generated by a second content provider different than the first content producer;

identify candidates from the additional information determined to be associated with the feature extracted from the promotional content, the candidates including at least two of a textual feature, a visual feature, or an audio feature; and

in response to identifying the candidates, automatically aggregate the candidates into an aggregate video; and

a presentation component configured to, when executed by the one or more processing units:

present the aggregate video.

11. The system of claim 10 , the presentation component further configured to:

present the online video sequentially with the aggregate video.

12. The system of claim 11 , the aggregate video being different than the promotional content.

13. The system of claim 10 , the parsing component further configured to:

parse the online video into one or more segments, a segment comprising a sequence of frames; and for respective segments, extract one or more keyframes.

14. The system of claim 10 , the information extraction component further configured to:

execute a multi modal relevance matching algorithm against the database using the feature to determine the additional information.

15. The system of claim 14 , the multimodal relevance matching algorithm further configured to:

perform a text based search algorithm upon the database using the textual feature to determine a first list of candidate additional information;

perform a visual feature matching algorithm upon the database using the visual feature to determine a second list of candidate additional information; and

perform a linear combination of the first list and second list to generate a set of additional information.

16. A computer-readable storage device comprising processor-executable instructions that, when executed, cause a computing device to perform a method for identifying digital content relevant to an online video, the method comprising:

selecting a frame of the online video as a keyframe based upon the frame having a video quality above a threshold;

identifying promotional content embedded within the keyframe, the promotional content being generated by a first content producer;

extracting a feature from at least the promotional content embedded in the keyframe, the feature comprising at least one of a textual feature, a visual feature, or an audio feature;

searching a database to identify additional information based upon the feature, the additional information being generated by a second content producer different than the first content producer;

identifying candidates from the additional information determined to be associated with the feature extracted from the promotional content, the candidates including at least two of a texture feature, a visual feature, or an audio feature;

in response to identifying the candidates, automatically aggregating the candidates into an aggregate video; and

outputting the aggregate video for presentation.

17. The computer-readable storage device of claim 16 , the method comprising:

presenting the online video sequentially with the aggregate video.