Systems, methods, and apparatus for generating an audio-visual presentation using characteristics of audio, visual and symbolic media objects
In an embodiment, a method and apparatus for generating a presentation is provided. The method considers characteristics of audio works and visual works when constructing the presentation. In some embodiments, the presentation may be automatically constructed.
1. A method of creating an audio-visual presentation, the method comprising:
identifying a first plurality of media object characteristics associated with a first media object to be presented by a media player device, the first media object including audio data, and the first plurality of media object characteristics including a mood characteristic of the audio data;
determining a congruency between the first plurality of media object characteristics and a second plurality of media object characteristics associated with a second media object that includes video data, the determining of the congruency including accessing a score representative of a degree of congruency between one of the first plurality of media object characteristics and one of the second plurality of media object characteristics, the score being accessed from a correlation matrix, the determining of the congruency being based on the score and a threshold value;
determining a plurality of temporal segments of the audio data based on the mood characteristic;
determining a duration of a temporal segment of the plurality of temporal segments of the audio data;
identifying the second media object from a plurality of reference media objects based on the congruency between the first plurality of media object characteristics and the second plurality of media object characteristics, the identifying of the second media object being further based on the duration of the temporal segment of the audio data; and
using a processor, generating the audio-visual presentation to synchronously present the first media object and the second media object on the media player device.
2. The method of claim 1 , wherein the identifying of the second media object is based on at least one of a visual effect, a template of parameters, a theme, an animation, or a tag.
3. The method of claim 1 , wherein:
the first plurality of media object characteristics includes a rhythmic pattern of the audio data; and
the determining of the plurality of temporal segments is based on the rhythmic pattern.
4. The method of claim 3 , further comprising:
determining a pace of the temporal segment based on the rhythmic pattern; and
wherein:
the identifying of the second media object is based on the pace being within a range of paces.
5. The method of claim 3 , further comprising:
causing presentation of the second media object during a presentation of the temporal segment based on the rhythmic pattern.
6. The method of claim 1 , wherein:
the first plurality of media object characteristics includes chord progressions of the audio data; and
the determining of the plurality of temporal segments is based on the chord progressions.
7. The method of claim 1 , wherein:
the first plurality of media object characteristics includes timbre of the audio data; and
the determining of the plurality of temporal segments is based on the timbre.
8. The method of claim 1 , wherein:
the generating of the audio-visual presentation comprises determining a presentation pace.
9. The method of claim 8 , wherein:
the audio-visual presentation further comprises additional media objects that have video data; and
the determining of the presentation pace is based on the quantity of the additional media objects.
10. The method of claim 8 , wherein:
the presentation pace varies across a duration of the audio-visual presentation.
11. A system comprising:
a memory having instructions embodied thereon; and
one or more processors configured by the instructions to perform operations comprising:
identifying a first plurality of media object characteristics associated with a first media object to be presented by a media player device, the first media object including audio data, and the first plurality of media object characteristics including a mood characteristic of the audio data;
determining a congruency between the first plurality of media object characteristics and a second plurality of media object characteristics associated with a second media object that includes video data, the determining of the congruency including accessing a score representative of a degree of congruency between one of the first plurality of media object characteristics and one of the second plurality of media object characteristics, the score being accessed from a correlation matrix, the determining of the congruency being based on the score and a threshold value;
determining a plurality of temporal segments of the audio data based on the mood characteristic;
determining a duration of a temporal segment of the plurality of temporal segments of the audio data;
identifying the second media object from a plurality of reference media objects based on the congruency between the first plurality of media object characteristics and the second plurality of media object characteristics, the identifying of the second media object being further based on the duration of the temporal segment of the audio data; and
generating an audio-visual presentation to synchronously present the first media object and the second media object on the media player device.
12. The system of claim 11 , wherein:
the first plurality of media object characteristics includes chord progressions of the audio data; and
the determining of the plurality of temporal segments is based on the chord progressions.
13. The system of claim 11 , wherein the identifying of the second media object is based on at least one of a visual effect, a template of parameters, a theme, an animation, or a tag.
14. The system of claim 11 , wherein:
the first plurality of media object characteristics includes a rhythmic pattern of the audio data; and
the determining of the plurality of temporal segments is based on the rhythmic pattern.
15. The system of claim 14 , wherein:
the operations further comprise:
determining a pace of the temporal segment based on the rhythmic pattern; and
the identifying of the second media object is based on the pace being within a range of paces.
16. The system of claim 11 , wherein the operations further comprise:
causing presentation of the second media object during a presentation of the temporal segment based on the rhythmic pattern.
17. The system of claim 11 , wherein:
the first plurality of media object characteristics includes timbre of the audio data; and
the determining of the plurality of temporal segments is based on the timbre.
18. A non-transitory machine-readable medium having instructions embodied thereon which, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
identifying a first plurality of media object characteristics associated with a first media object to be presented by a media player device, the first media object including audio data, and the first plurality of media object characteristics including a mood characteristic of the audio data;
determining a congruency between the first plurality of media object characteristics and a second plurality of media object characteristics associated with a second media object that includes video data, the determining of the congruency including accessing a score representative of a degree of congruency between one of the first plurality of media object characteristics and one of the second plurality of media object characteristics, the score being accessed from a correlation matrix, the determining of the congruency being based on the score and a threshold value;
determining a plurality of temporal segments of the audio data based on the mood characteristic;
determining a duration of a temporal segment of the plurality of temporal segments of the audio data;
identifying the second media object from a plurality of reference media objects based on the congruency between the first plurality of media object characteristics and the second plurality of media object characteristics; and
generating an audio-visual presentation to synchronously present the first media object and the second media object on the media player device.