Augmenting an audio signal via extraction of musical features and obtaining of media fragments
A system and method is described in which a multimedia story is rendered to a consumer in dependence on features extracted from an audio signal representing for example a musical selection of the consumer. Features such as key changes and tempo of the music selection are related to dramatic parameters defined by and associated with story arcs, narrative story rules and film or story structure. In one example a selection of a few music tracks provides input audio signals ( 602 ) from which musical features are extracted ( 604 ), following which a dramatic parameter list and timeline are generated ( 606 ). Media fragments are then obtained ( 608 ), the fragments having story content associated with the dramatic parameters, and the fragments output ( 610 ) with the music selection.
1. A method for augmenting an audio signal comprising acts of:
receiving an audio signal,
extracting features from said audio signal,
generating a time ordered table of dramatic parameters according to the extracted features, wherein the dramatic parameters include mood, changes of pace and incidents,
selecting a story template at least in part in dependence on said table of dramatic parameters, wherein said story template comprises dramatic parameter data related to a narrative story structure,
obtaining media fragments at least in part in dependence on the table of dramatic parameters by matching the dramatic parameters of the selected story template with those of the media fragments, and
outputting said media fragments in tandem with said audio signal.
2. The method according to claim 1 , wherein said features extracted from said audio signal include one or more of tempo, key, volume.
3. The method according to claim 1 , wherein the generation of said table of dramatic parameters comprises retrieving a list of dramatic parameters and associated audio features, comparing and matching the extracted features with the retrieved associated audio features, and inserting an entry comprising the dramatic parameter associated with the audio feature.
4. The method according to claim 1 , wherein said obtaining said media fragments comprises selecting a fragment from a store, said fragment being stored with an associated dramatic parameter which matches the respective entry in the table of dramatic parameters.
5. The method according to claim 1 , wherein said obtaining said media fragments comprises generating a fragment.
6. The method according to claim 4 , and further comprising receiving user input, said user input affecting said obtaining.
7. The method according to claim 1 , wherein said media fragments include video data.
8. The method according to claim 1 , wherein said outputting comprises storing said media fragments and said audio signal.
9. The method according to claim 1 , wherein said outputting comprises rendering said media fragments and said audio signal.
10. The method according to claim 1 , wherein the story template for selection is generated according to logical story structure rules and the dramatic parameter list.
11. The method according to claim 1 , wherein the dramatic parameters are represented by physical mark up language tags.
12. The method according to claim 1 , wherein combinations of extracted features have associated dramatic parameters.
13. A system for augmenting an audio signal comprising:
an input device for receiving an audio signal, and
processing means for extracting features from said received audio signal, and for generating a time ordered table of dramatic parameters associated with said extracted features, and for selecting a story template at least in part in dependence on said table of dramatic parameters, wherein said story template comprises dramatic parameter data related to a narrative story structure, and further for obtaining media fragments at least in part in dependence on said generated table of dramatic parameters by matching the dramatic parameters of the selected story template with those of the media fragments, and
at least one output device for outputting said media fragments in tandem with said audio signal, and
wherein the dramatic parameters include mood, changes of pace and incidents.
14. The system according to claim 13 , further comprising storage for storing said media fragments.
15. The system according to claim 13 , wherein said at least one output device comprises display means on which said media fragments are displayed.
16. The system according to claim 13 , wherein said at least one output device is responsive to instructions associated with said dramatic parameters.
17. The system of claim 13 , comprising a database storing media fragments with associated dramatic parameters.