Music streaming, playlist creation and streaming architecture
A system and method for making categorized music tracks available to end user applications. The tracks may be categorized based on computer-derived rhythm, texture and pitch (RTP) scores for tracks derived from high-level acoustic attributes, which is based on low level data extracted from the tracks. RTP scores are stored in a universal database common to all of the music publishers so that the same track, once RTP scored, does not need to be re-RTP scored by other music publishers. End user applications access an API server to import collections of tracks published by publishers, to create playlists and initiate music streaming. Each end user application is sponsored by a single music publisher so that only tracks capable of being streamed by the music publisher are available to the sponsored end user application.
1. A method for categorizing music tracks, comprising:
creating a sample set that includes a RTP score for a plurality of possible combinations of a rhythm score (R), a texture score (T), and a pitch score (P) respectively from a R range, a T range, and a P range, at least some of which RTP scores each correspond to a human-determined RTP score for a predetermined music track among a plurality of predetermined tracks, each RTP score corresponding to a category among a plurality of categories;
analyzing a plurality of high-level acoustic attributes corresponding to each track among a plurality of tracks to develop computer-derived RTP scores for each track among the plurality of tracks based on the sample set, each computer-derived RTP score corresponding to one RTP score in the sample set, the high-level acoustic attributes based on low level data extracted from each track among the plurality of tracks to be RTP scored; and
utilizing the computer-derived RTP scores for each track to determine a corresponding category for each track among the plurality of categories.
2. The method of claim 1 , wherein the high-level acoustic attributes used to develop the R score for each track includes two or more of a time signature of each track, a measure of danceable qualities of each track, a measure of one or more of intensity and activity of each track, a speed of each track, and a measure of sound quality in decibels of each track.
3. The method of claim 2 , wherein the time signature is represented by a number within a range and a confidence value corresponding to measured periodicity.
4. The method of claim 2 , wherein the measure of danceable qualities of each track is based on one or more of a tempo, a rhythm stability, a beat strength, and overall regularity.
5. The method of claim 2 , wherein the measure of one or more of intensity and activity of each track is based on a dynamic range, a measure of perceived loudness, an onset rate and a general entropy.
6. The method of claim 1 , wherein the high-level acoustic attributes used to develop the T score for each track includes two or more of a measure of acoustic qualities of each track, a measure of vocals contained within each track, a measure of one or more of intensity and activity of each track, a measure of a sound presence of one or more of a live audience or voices in each track, and a measure of a sound presence of spoken words in each track.
7. The method of claim 6 , wherein the measure of one or more of intensity and activity of each track is based on a dynamic range, a measure of perceived loudness, an onset rate and a general entropy.
8. The method of claim 1 , wherein the high-level acoustic attributes used to develop the P score for each track includes two or more of a key of each track, a modality of each track, a measure of a positive aspect or a negative aspect of each track, and a measure of a sound presence of spoken words in each track.
9. The method of claim 8 , wherein the key of each track is based on pitch class notation.
10. The method of claim 8 , wherein the modality of each track is an indication of whether each track is in a major key or a minor key.
11. The method of claim 1 , wherein the high-level acoustic attributes used to develop the R score for each track includes a defined rhythm by a music service, wherein the high-level acoustic attributes used to develop the P score for each track includes a defined pitch by the music service, wherein the high-level acoustic attributes used to develop the T score for each track includes a defined timbre by the music service.
12. The method of claim 11 , wherein defined rhythm is defined by dividing each track into bars, dividing the bars into beats, and dividing the beats into tatums and utilizing one or more of the tatums, a location of a downbeat, an acceleration/deceleration of components of each track to determine a time signature.
13. The method of claim 12 , wherein the time signature is identified by a number within a range and a confidence value.
14. The method of claim 13 , wherein a low confidence value indicates a lack of periodicity.
15. The method of claim 11 , wherein the defined pitch is defined by a chroma vector that corresponds to each of 12 pitch classes, wherein a value assigned to each pitch class may depend on a relative dominance of every pitch in a chromatic scale.
16. The method of claim 11 , wherein the defined texture is defined by a quality of a musical note or sound by which one type of musical instrument or voice is distinguished from others.
17. A method for categorizing streamed music tracks, comprising:
determining high-level acoustic attributes for a music track based on low-level data extracted from the track;
analyzing the high-level acoustic attributes to develop a computer-derived RTP score for the track based on a sample set, the computer-derived RTP score corresponding to one RTP score in the sample set, wherein the sample set includes a RTP score for a plurality of possible combinations of a rhythm score (R), a texture score (T), and a pitch score (P) respectively from a R range, a T range, and a P range, wherein at least some of which RTP scores each correspond to a human-determined RTP score for a predetermined track among a plurality of predetermined tracks, and wherein each RTP score corresponding to a category among a plurality of categories; and
utilizing the computer-derived RTP score to determine a corresponding category for the track among the plurality of categories.
18. The method of claim 17 , wherein the high-level acoustic attributes used to develop the R score for the track includes two or more of a time signature of the track, a measure of danceable qualities of the track, a measure of one or more of intensity and activity of the track, a speed of the track, and a measure of sound quality in decibels of the track.
19. The method of claim 18 , wherein the time signature is represented by a number within a range and a confidence value corresponding to measured periodicity.
20. The method of claim 18 , wherein the measure of danceable qualities of the track is based on one or more of a tempo, a rhythm stability, a beat strength, and overall regularity.
21. The method of claim 18 , wherein the measure of one or more of intensity and activity of the track is based on a dynamic range, a measure of perceived loudness, an onset rate and a general entropy.
22. The method of claim 17 , wherein the high-level acoustic attributes used to develop the T score for the track includes two or more of a measure of acoustic qualities of the track, a measure of vocals contained within the track, a measure of one or more of intensity and activity of the track, a measure of a sound presence of one or more of a live audience or voices in the track, and a measure of a sound presence of spoken words in the track.
23. The method of claim 22 , wherein the measure of one or more of intensity and activity of the track is based on a dynamic range, a measure of perceived loudness, an onset rate and a general entropy.
24. The method of claim 17 , wherein the high-level acoustic attributes used to develop the P score for the track includes two or more of a key of the track, a modality of the track, a measure of a positive aspect or a negative aspect of the track, and a measure of a sound presence of spoken words in the track.
25. The method of claim 24 , wherein the key of the track is based on pitch class notation.
26. The method of claim 24 , wherein the modality of the track is an indication of whether the track is in a major key or a minor key.
27. The method of claim 17 , wherein the determining, the analyzing and the utilizing are performed at a location remote from a user device, and wherein the including is performed in response to the user device.
28. A method for categorizing music tracks, comprising:
creating a sample set that includes a RTP score for a plurality of possible combinations of a rhythm score (R), a texture score (T), and a pitch score (P) respectively from a R range, a T range, and a P range, at least some of which RTP scores each correspond to a human-determined RTP score for a predetermined music track among a plurality of predetermined music tracks, each RTP score corresponding to a category among a plurality of categories;
extracting low-level data from each music track among a plurality of music tracks to be RTP scored by converting each music track into a plurality of mel-spectrograms, each mel-spectrogram corresponding to a different predetermined period of each music track;
analyzing the plurality of mel-spectrograms with a first trained neural network to generate a vector of audio features for each predetermined period;
analyzing each vector with a second trained neural network to determine computer-derived RTP scores for each music track among the plurality of music tracks based on the sample set, each computer-derived RTP score corresponding to one RTP score in the sample set; and
utilizing the computer-derived RTP scores for each music track to determine a corresponding category for each music track among the plurality of categories.
29. The method of claim 28 , wherein the trained neural network is a resultant classification neural network.
30. The method of claim 29 , wherein the resultant classification neural network utilizes recurrent layers.
31. The method of claim 30 , wherein extracting includes extracting desired recurrent layers to generate the vector of audio features for each predetermined period.
32. The method of claim 28 , wherein the first trained neural network is trained with an ontology of audio event classes and a collection of human-labeled sound clips.