Automatically selecting thematically representative music
A method for automatically selecting thematically representative music is disclosed. A processor is used for using a theme-related keyword to search a keyword-indexed video repository to retrieve videos associated with the theme-related keyword; analyzing the retrieved videos to select videos with music; and extracting music tracks and features from the selected videos. The method further includes selecting representative music related to the theme from the extracted music tracks using the extracted features; and storing the selected representative music in a processor accessible memory.
1. A system for automatically selecting thematically representative music, comprising:
a processor configured to:
retrieve a set of videos related to a set of themes;
search, based at least in part on a theme-related keyword, a keyword-indexed video repository to retrieve videos associated with the theme-related keyword;
analyze the retrieved videos and select videos including music from the retrieved videos;
extract video features from the selected videos;
compare the extracted video features of the selected videos to video features of the set of videos related to the set of themes to determine at least one of the selected videos that is thematically similar to at least one of the set of videos;
extract a music track of the determined at least one of the selected videos and select the music track as a representative of the corresponding theme; and
cause the selected representative music track to be stored in a processor accessible memory.
2. The system of claim 1 , wherein the set of themes comprises an ethnographic theme, a geographic theme, a demographic theme, an event-based theme, or any combination thereof.
3. The system of claim 1 , wherein each video included in the keyword-indexed video repository is associated with a respective keyword annotation.
4. The system of claim 1 , wherein the processor is configured to extract the music track of the determined at least one of the selected videos using an FFmpeg open source software library.
5. The system of claim 4 , wherein the processor is configured to pass the music track through a feature extraction algorithm configured to reduce sound information of the music track to a 32-element vector of numbers as a descriptive feature of the music track.
6. The system of claim 5 , wherein the processor is configured to determine a tempo, a time signature, a key, and a mode of the music track based at least in part on the 32-element vector of numbers.
7. The system of claim 1 , wherein the processor is configured to extract the video features from the selected videos using a supervised machine learning technique.
8. The system of claim 1 , wherein the processor is configured to extract the video features from the selected videos using an unsupervised machine learning technique.
9. The system of claim 1 , wherein the processor is configured to select the music track the representative of the corresponding theme based at least in part on a feature fusion technique.
10. The system of claim 9 , wherein the processor is configured to concatenate features of the music track and the extracted video features at a feature level, a score level, and a decision level, as the feature fusion technique.
11. A computer-implemented method for automatically selecting thematically representative music, comprising:
accessing, via a processor, a set of videos related to a set of themes;
searching, via the processor, a keyword-indexed video repository based at least in part on a theme-related keyword to retrieve videos associated with the theme-related keyword;
analyzing, via the processor, the retrieved videos and selecting videos including music from the retrieved videos;
extracting, via the processor, video features from the selected videos;
comparing, via the processor, the extracted video features of the selected videos to video features of the set of videos related to the set of themes to determine at least one of the selected videos that is thematically similar to at least one of the set of videos;
extracting, via the processor, a music track of the determined at least one of the selected videos and selecting the music track as a representative of the corresponding theme; and
causing, via the processor, the selected representative music track to be stored in a processor accessible memory.