Intuitive music visualization using efficient structural segmentation

1. A method for automatically identifying structures of a music stream, the method comprising:

extracting, from each of a plurality of frames of a waveform corresponding to the music stream, at least one feature that corresponds to a music attribute;

utilizing a signal segmentation algorithm to symbolize the extracted at least one feature of the plurality of frames of the waveform;

comparing a set of symbolized frames of the plurality of frames to other sets of symbolized frames to determine expression patterns of the extracted at least one feature throughout the waveform;

segmenting the waveform based on the determined expression patterns to produce one or more segments of the waveform; and

causing display of a visualization of the waveform that visually indicates the one or more segments of the waveform.

2. The method of claim 1 , wherein the signal segmentation algorithm is VMO, which is utilized to generate a symbolized signal.

3. The method of claim 1 , wherein, for harmonic content of the music stream, the at least one feature is one or more of a constant-Q transformed (CQT) spectra, a chroma, or a timbre.

4. The method of claim 1 , wherein, for rhythmic content of the music stream, the at least one feature is derived from a tempogram.

5. The method of claim 1 , wherein for harmonic content of the music stream, the at least one feature is a timbre that is represented by Mel-frequency cepstral coefficients (MFCCs).

6. The method of claim 1 , wherein VMO is utilized to generate a matrix, wherein the matrix is an SSM.

7. The method of claim 6 , wherein the SSM is a VMO-SSM.

8. The method of claim 1 , further comprising generating a segment structure based on segmenting the waveform, wherein generating the segment structure utilizes one or more of spectral clustering, connectivity-constrained hierarchical clustering, or structure features and segment similarity.

9. The method of claim 2 , wherein the symbolized signal is a VMO structure, which is a data structure capable of symbolizing the waveform by clustering observations in the waveform.

10. The method of claim 1 , wherein the signal segmentation algorithm is used to symbolize the extracted at least one feature of the plurality of frames of the waveform selectively chooses frames or groups of frames for which to calculate a distance, the selectively choosing based on whether common suffices are shared between two frames or two groups of frames, thereby eliminating unnecessary computations.

11. The method of claim 1 , further comprising identifying sets of frames that have similar expression patterns.

12. The method of claim 2 , wherein the VMO structure stores information corresponding to repeating sub-sequences within a time series by way of suffix links.

13. One or more computer storage media storing computer-useable instructions that, when used by a computing device, cause the computing device to perform a method for automatically identifying structures of a music stream, the method comprising:

receiving a waveform that corresponds to the music stream;

extracting at least one feature from each of a plurality of frames of the waveform;

applying a Variable Markov Oracle (VMO) algorithm to index the at least one feature for each of the plurality of frames;

comparing the indexed at least one feature for a set of frames to other sets of frames;

determining one or more segments of the waveform by applying a segmentation algorithm; and

causing display of a visualization of the waveform that visually indicates the one or more segments of the waveform.

14. The one or more computer storage media of claim 13 , further comprising generating a VMO-SSM from the VMO structure.

15. The one or more computer storage media of claim 13 , further comprising generating a connectivity matrix from the VMO-SSM, wherein generating the connectivity matrix comprises median filtering and adding local linkages.

16. The one or more computer storage media of claim 13 , further comprising generating a segment structure, wherein the segment structure comprises an indication of repetitive segments.

17. The one or more computer storage media of claim 13 , wherein the segmentation algorithm comprises one or more of spectral clustering, connectivity-constrained hierarchical clustering, or structure features and segment similarity.

18. The one or more computer storage media of claim 13 , further comprising refining boundaries of the one or more segments of the waveform by applying an iterative boundary adjusting algorithm to the one or more segments of the waveform.

19. A system for automatically identifying structures of a music stream, the system comprising:

one or more processors; and

one or more computer storage media comprising computer-useable instructions for causing the one or more processors to perform operations, the operations comprising:

extracting, from a waveform corresponding to the music stream, at least one feature that corresponds to a music attribute;

utilizing a Variable Markov Oracle (VMO) algorithm to construct, from the at least one feature, a VMO structure comprising a symbolized signal, and

generate a VMO-SSM matrix;

referencing the VMO-SSM matrix to generate a segment structure, the segment structure illustrating a segmentation of the waveform;

causing display of a visualization of the segmentation of the waveform.

20. The system of claim 19 , wherein the segment structure comprises an indication of repetitive segments.