Streaming music categorization using rhythm, texture and pitch
A method for categorizing streamed music based on a sample set of RTP scores for predetermined tracks. High-level acoustic attributes for tracks are determined by an analyzed extraction of low-level data from the tracks. The high-level acoustic attributes are used to develop computer-derived RTP scores for the tracks based on the sample set, which includes RTPs score for a plurality of possible combinations of a rhythm score (R), a texture score (T), and a pitch score (P) respectively from a R range, a T range, and a P range. At least some of the RTP scores correspond to human-determined RTP scores for predetermined tracks among a plurality of predetermined tracks. Each RTP score corresponds to a category among a plurality of categories. The computer-derived RTP scores are used to determine a category for each track among the plurality of categories. Playlists of the tracks are based on one or more of the categories.
1. A method for categorizing music tracks, comprising:
creating a sample set that includes a RTP score for a plurality of possible combinations of a rhythm score (R), a texture score (T), and a pitch score (P) respectively from a R range, a T range, and a P range, at least some of which RTP scores each correspond to a human-determined RTP score for a predetermined music track among a plurality of predetermined tracks, each RTP score corresponding to a category among a plurality of categories;
extracting low-level data from each track among a plurality of tracks to be RTP scored;
analyzing the low-level data to develop a plurality of high-level acoustic attributes for each track among the plurality of tracks;
analyzing the high-level acoustic attributes to develop computer-derived RTP scores for each track among the plurality of tracks based on the sample set, each computer-derived RTP score corresponding to one RTP score in the sample set;
utilizing the computer-derived RTP scores for each track to determine a corresponding category for each track among the plurality of categories; and
creating a playlist based on tracks corresponding to one or more categories among the plurality of categories.
2. The method of claim 1 , wherein the high-level acoustic attributes used to develop the R score for each track includes two or more of a time signature of each track, a measure of danceable qualities of each track, a measure of one or more of intensity and activity of each track, a speed of each track, and a measure of sound quality in decibels of each track.
3. The method of claim 2 , wherein the time signature is represented by a number within a range and a confidence value corresponding to measured periodicity.
4. The method of claim 2 , wherein the measure of danceable qualities of each track is based on one or more of a tempo, a rhythm stability, a beat strength, and overall regularity.
5. The method of claim 2 , wherein the measure of one or more of intensity and activity of each track is based on a dynamic range, a measure of perceived loudness, an onset rate and a general entropy.
6. The method of claim 1 , wherein the high-level acoustic attributes used to develop the T score for each track includes two or more of a measure of acoustic qualities of each track, a measure of vocals contained within each track, a measure of one or more of intensity and activity of each track, a measure of a sound presence of one or more of a live audience or voices in each track, and a measure of a sound presence of spoken words in each track.
7. The method of claim 6 , wherein the measure of one or more of intensity and activity of each track is based on a dynamic range, a measure of perceived loudness, an onset rate and a general entropy.
8. The method of claim 1 , wherein the high-level acoustic attributes used to develop the P score for each track includes two or more of a key of each track, a modality of each track, a measure of a positive aspect or a negative aspect of each track, and a measure of a sound presence of spoken words in each track.
9. The method of claim 8 , wherein the key of each track is based on pitch class notation.
10. The method of claim 8 , wherein the modality of each track is an indication of whether each track is in a major key or a minor key.
11. The method of claim 1 , wherein the high-level acoustic attributes used to develop the R score for each track includes a defined rhythm by a music service, wherein the high-level acoustic attributes used to develop the P score for each track includes a defined pitch by the music service, wherein the high-level acoustic attributes used to develop the T score for each track includes a defined timbre by the music service.
12. The method of claim 11 , wherein defined rhythm is defined by dividing each track into bars, dividing the bars into beats, and dividing the beats into tatums and utilizing one or more of the tatums, a location of a downbeat, an acceleration/deceleration of components of each track to determine a time signature.
13. The method of claim 12 , wherein the time signature is identified by a number within a range and a confidence value.
14. The method of claim 13 , wherein a low confidence value indicates a lack of periodicity.
15. The method of claim 11 , wherein the defined pitch is defined by a chroma vector that corresponds to each of 12 pitch classes, wherein a value assigned to each pitch class may depend on a relative dominance of every pitch in a chromatic scale.
16. The method of claim 11 , wherein the defined texture is defined by a quality of a musical note or sound by which one type of musical instrument or voice is distinguished from others.
17. The method of claim 1 , wherein creating the playlist includes selecting an intensity level for the computer-derived RTP scores corresponding to the one or more categories, wherein the playlist includes the tracks corresponding to one or more categories having computer-derived RTP scores above the selected intensity level.
18. A method for categorizing streamed music tracks, comprising:
determining high-level acoustic attributes for a music track through an analyzed extraction of low-level data from the track;
analyzing the high-level acoustic attributes to develop a computer-derived RTP score for the track based on a sample set, the computer-derived RTP score corresponding to one RTP score in the sample set, wherein the sample set includes a RTP score for a plurality of possible combinations of a rhythm score (R), a texture score (T), and a pitch score (P) respectively from a R range, a T range, and a P range, wherein at least some of which RTP scores each correspond to a human-determined RTP score for a predetermined track among a plurality of predetermined tracks, and wherein each RTP score corresponding to a category among a plurality of categories;
utilizing the computer-derived RTP score to determine a corresponding category for the track among the plurality of categories; and
including the track in a playlist based on one or more categories among the plurality of categories.
19. The method of claim 18 , wherein the high-level acoustic attributes used to develop the R score for the track includes two or more of a time signature of the track, a measure of danceable qualities of the track, a measure of one or more of intensity and activity of the track, a speed of the track, and a measure of sound quality in decibels of the track.
20. The method of claim 19 , wherein the time signature is represented by a number within a range and a confidence value corresponding to measured periodicity.
21. The method of claim 19 , wherein the measure of danceable qualities of the track is based on one or more of a tempo, a rhythm stability, a beat strength, and overall regularity.
22. The method of claim 19 , wherein the measure of one or more of intensity and activity of the track is based on a dynamic range, a measure of perceived loudness, an onset rate and a general entropy.
23. The method of claim 18 , wherein the high-level acoustic attributes used to develop the T score for the track includes two or more of a measure of acoustic qualities of the track, a measure of vocals contained within the track, a measure of one or more of intensity and activity of the track, a measure of a sound presence of one or more of a live audience or voices in the track, and a measure of a sound presence of spoken words in the track.
24. The method of claim 23 , wherein the measure of one or more of intensity and activity of the track is based on a dynamic range, a measure of perceived loudness, an onset rate and a general entropy.
25. The method of claim 18 , wherein the high-level acoustic attributes used to develop the P score for the track includes two or more of a key of the track, a modality of the track, a measure of a positive aspect or a negative aspect of the track, and a measure of a sound presence of spoken words in the track.
26. The method of claim 25 , wherein the key of the track is based on pitch class notation.
27. The method of claim 25 , wherein the modality of the track is an indication of whether the track is in a major key or a minor key.
28. The method of claim 18 , wherein the determining, the analyzing and the utilizing are performed at a location remote from a user device, wherein the including is performed in response to the user device, and further comprising streaming the playlist to the user device.
29. The method of claim 18 , wherein including the track in the playlist is based on a selected intensity level for the computer-derived RTP scores corresponding to the one or more categories, wherein the playlist includes the track if the corresponding computer-derived RTP score for the track is above the selected intensity level.
30. A method for categorizing music tracks, comprising:
creating a sample set that includes a RTP score for a plurality of possible combinations of a rhythm score (R), a texture score (T), and a pitch score (P) respectively from a R range, a T range, and a P range, at least some of which RTP scores each correspond to a human-determined RTP score for a predetermined music track among a plurality of predetermined music tracks, each RTP score corresponding to a category among a plurality of categories;
extracting low-level data from each music track among a plurality of music tracks to be RTP scored by converting each music track into a plurality of mel-spectrograms, each mel-spectrogram corresponding to a different predetermined period of each music track;
analyzing the plurality of mel-spectrograms with a first trained neural network to generate a vector of audio features for each predetermined period;
analyzing each vector with a second trained neural network to determine computer-derived RTP scores for each music track among the plurality of music tracks based on the sample set, each computer-derived RTP score corresponding to one RTP score in the sample set;
utilizing the computer-derived RTP scores for each music track to determine a corresponding category for each music track among the plurality of categories; and
creating a playlist based on music tracks corresponding to one or more categories among the plurality of categories.
31. The method of claim 30 , wherein the trained neural network is a resultant classification neural network.
32. The method of claim 31 , wherein the resultant classification neural network utilizes recurrent layers.
33. The method of claim 32 , wherein extracting includes extracting desired recurrent layers to generate the vector of audio features for each predetermined period.
34. The method of claim 30 , wherein the first trained neural network is trained with an ontology of audio event classes and a collection of human-labeled sound clips.