Systems, methods, and apparatus for generating an audio-visual presentation using characteristics of audio, visual and symbolic media objects
View Patent ↗In an embodiment, a method and apparatus for generating a presentation is provided. The method considers characteristics of audio works and visual works when constructing the presentation. In some embodiments, the presentation may be automatically constructed.
1. A method of creating an audio-visual presentation, the method comprising:
identifying a first plurality of media object characteristics associated with a first media object to be presented by a media player device;
determining a congruency while the audio-visual presentation is being presented, the congruency being between the first plurality of media object characteristics and a second plurality of media object characteristics associated with a second media object, the first media object including audio data and the second media object including visual data which are simultaneously presented, the determining of the congruency including accessing a first score from a first correlation matrix that correlates a first media object characteristic of a first descriptor type among the first plurality of media object characteristics to a second media object characteristic of a second descriptor type different from the first descriptor type among the second plurality of media object characteristics, the first score representing a first degree of congruency between the first media object characteristic and the second media object characteristic, the first media object characteristic and the second media object characteristic having dissimilar descriptor types within the first correlation matrix from which the first score is accessed, the determining of the congruency further including accessing a second correlation matrix that includes a second score that represents a second degree of congruency between the first media object and a non-media entity, the determining of the congruency being based on the first score transgressing a threshold value;
identifying the second media object from a plurality of reference media objects based on the determined congruency between the first plurality of media object characteristics and the second plurality of media object characteristics and based on the second degree of congruency between the first media object and the non-media entity; and
using a processor, dynamically generating the audio-visual presentation to synchronously present the first media object and the second media object on the media player device.
2. The method of claim 1 , wherein the audio data represents at least part of an audio track.
3. The method of claim 1 , wherein the first media object includes visual data and the second media object includes audio data.
4. The method of claim 3 , wherein the visual data represents a picture.
5. The method of claim 3 , wherein the visual data represents a video.
6. The method of claim 1 , further comprising:
communicating the audio-visual presentation to the media player device via a network.
7. The method of claim 1 , further comprising:
identifying a first media object category based on the first plurality of media object characteristics, the first media object category being one of a plurality of categories associated with the first media object; and
identifying, based on the first media object category, a second media object category associated with the plurality of reference media objects, the second media object category being based on the second plurality of media object characteristics; wherein
the identifying of the second media object is based on the second media category.
8. The method of claim 7 , wherein the first media object has a first media type; at least one of the first plurality of media object characteristics is specific to the first media type; the second media object has a second media type; and at least one of the second plurality of media object characteristics is specific to the second media type.
9. The method of claim 1 , wherein the first media object includes audio data and the second media object includes visual data, the method further comprising:
determining a plurality of temporal segments of the audio data; and
determining a duration of a temporal segment of the plurality of temporal segments of the audio data; wherein
the identifying of the second media object is based on the duration of the temporal segment.
10. The method of claim 9 , wherein the identifying of the second media object is based on at least one of a visual effect, a template of parameters, a theme, an animation, or a tag.
11. The method of claim 9 , wherein the first plurality of media object characteristics includes one or more mood characteristics of the audio data; and the determining of the plurality of temporal segments is based on the one or more mood characteristics.
12. The method of claim 9 , wherein the first plurality of media object characteristics includes one or more rhythmic patterns of the audio data; and the determining of the plurality of temporal segments is based on the one or more rhythmic patterns.
13. The method of claim 12 , further comprising:
determining a pace of the temporal segment based on the one or more rhythmic patterns; and the identifying of the second media object is based on the pace being within a range of paces.
14. The method of claim 12 , further comprising:
identifying a rhythmic pattern of the one or more rhythmic patterns in the temporal segment; and
presenting the second media object during a presentation of the temporal segment based on the identified rhythmic pattern.
15. The method of claim 1 , wherein the determining of the congruency is based on a measure of similarity between the first plurality of media object characteristics and the second plurality of media object characteristics and based on a threshold of similarity.
16. The method of claim 1 wherein the identifying of the second media object is performed while at least a portion of the first media object is being presented on the media player device.
17. The method of claim 1 , further comprising
processing the plurality of reference media objects to obtain the second plurality of media object characteristics during a presentation of the first media object.
18. A system comprising:
a processor configured by an analysis module to:
identify a first plurality of media object characteristics associated with a first media object to be presented by a media player device, and
determine a congruency while the audio-visual presentation is being presented, the congruency being between the first plurality of media object characteristics and a second plurality of media object characteristics associated with a second media object, the first media object including audio data and the second media object including visual data which are simultaneously presented, the determining of the congruency including accessing a first score from a first correlation matrix that correlates a first media object characteristic of a first descriptor type among the first plurality of media object characteristics to a second media object characteristic of a second descriptor type different from the first descriptor type among the second plurality of media object characteristics, the first score representing a first degree of congruency between the first media object characteristic and the second media object characteristic, the first media object characteristic and the second media object characteristic having dissimilar descriptor types within the first correlation matrix from which the first score is accessed, the determining of the congruency further including accessing a second correlation matrix that includes a second score that represents a second degree of congruency between the first media object and a non-media entity, the determining of the congruency being based on the first score transgressing a threshold value;
a selection module configured to identify the second media object from a plurality of reference media objects based on the determined congruency between the first plurality of media object characteristics and the second plurality of media object characteristics and based on the second degree of congruency between the first media object and the non-media entity; and
a presentation module configured to dynamically generate the audio-visual presentation to synchronously present the first media object and the second media object on the media player device.
19. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
identifying a first plurality of media object characteristics associated with a first media object to be presented by a media player device;
determining a congruency while the audio-visual presentation is being presented, the congruency being between the first plurality of media object characteristics and a second plurality of media object characteristics associated with a second media object, the first media object including audio data and the second media object including visual data which are simultaneously presented, the determining of the congruency including accessing a first score from a first correlation matrix that correlates a first media object characteristic of a first descriptor type among the first plurality of media object characteristics to a second media object characteristic of a second descriptor type different from the first descriptor type among the second plurality of media object characteristics, the first score representing a first degree of congruency between the first media object characteristic and the second media object characteristic, the first media object characteristic and the second media object characteristic having dissimilar descriptor types within the first correlation matrix from which the first score is accessed, the determining of the congruency further including accessing a second correlation matrix that includes a second score that represents a second degree of congruency between the first media object and a non-media entity, the determining of the congruency being based on the first score transgressing a threshold value;
identifying the second media object from a plurality of reference media objects based on the determined congruency between the first plurality of media object characteristics and the second plurality of media object characteristics and based on the second degree of congruency between the first media object and the non-media entity; and
dynamically generating the audio-visual presentation to synchronously present the first media object and the second media object on the media player device.