Creating music by concatenative synthesis
Automated creation of new music by listening is disclosed. A method to create new music may comprise listening to a plurality of music, learning from the plurality of music, and performing concatenative synthesis based on the listening and the learning to create the new music. The method may be performed on a computing device having an audio interface, such as a personal computer.
1. A method performed by a computing device to create new music, the method comprising:
the computing device listening to a plurality of music including receiving through an audio interface live or recorded music or reading a music file from a storage medium, the listening including creating an auditory spectrogram of the plurality of music
the computing device learning from the auditory spectrogram of the plurality of music
the computing device performing concatenative synthesis based on sound segments prepared during the listening and the learning to create the new music, the performing concatenative synthesis including
performing beat matching on the sound segments
performing time scaling on the sound segments
performing cross synthesis on the sound segments.
2. The method of claim 1 wherein listening comprises:
performing signal processing on the plurality of music
preparing segmented audio descriptors based on the plurality of music
recognizing patterns in the plurality of music.
3. The method of claim 1 wherein learning comprises:
preparing segmented audio descriptors based on the plurality of music
recognizing patterns in the plurality of music
classifying clusters in the plurality of music.
4. The method of claim 1 wherein listening comprises:
evaluating loudness indentified in the auditory spectrogram of the plurality of music
evaluating timbre of the auditory spectrogram of the plurality of music
performing onset detection of the auditory spectrogram of the plurality of music to create the sound segments
analyzing beat and tempo of the auditory spectrogram of the plurality of music
processing pitch and harmony using the sound segments created in the onset detection.
5. The method of claim 1 wherein creating an auditory spectrogram comprises:
creating an initial audio spectrogram based on the plurality of music
applying a transfer function to the initial audio spectrogram
applying frequency warping to the initial audio spectrogram
applying frequency masking to the initial audio spectrogram
applying temporal masking to the initial audio spectrogram.
6. The method of claim 1 wherein listening includes performing structural analysis.
7. The method of claim 6 wherein performing structural analysis comprises:
performing sound segment analysis on the plurality of music
performing beat analysis on the plurality of music
performing pattern recognition on the plurality of music.
8. The method of claim 1 wherein learning comprises:
performing prediction analysis on the plurality of music
performing downbeat prediction analysis on the plurality of music
performing clustering analysis on the plurality of music.
9. The method of claim 8 wherein performing downbeat prediction comprises:
performing segmentation on the auditory spectrogram
evaluating time lagged features.
10. A method performed by a computing device to create new music, the method comprising:
the computing device listening to a plurality of music including receiving through an audio interface live or recorded music or reading a music file from a storage medium, wherein listening includes
performing signal processing of the plurality of music
preparing segmented audio descriptors for the plurality of music
recognizing patterns in the plurality of music
the computing device learning from the plurality of music, wherein the learning includes
preparing additional segmented audio descriptors for the plurality of music
recognizing additional patterns in the plurality of music
classifying clusters in the plurality of music based on the additional segmented audio descriptors and the additional patterns
the computing device performing concatenative synthesis based on the segmented audio descriptors and sound segments prepared during the listening and the learning to create the new music, wherein the concatenative analysis includes
performing beat matching analysis on the sound segments and the segmented audio descriptors
performing time scaling of the sound segments and the segmented audio descriptors
performing cross synthesis of the sound segments and the segmented audio descriptors.
11. A computing device to create new music, the computing device comprising:
a processor
a memory coupled with the processor
an audio interface coupled with the processor
a storage medium having instructions stored thereon which when executed cause the computing device to perform actions comprising
listening to a plurality of music including receiving through the audio interface live or recorded music or reading a music file from the storage medium, the listening including creating an auditory spectrogram of the plurality of music
learning from the auditory spectrogram of the plurality of music
performing concatenative synthesis based on sound segments prepared during the listening and the learning to create the new music, the concatenative synthesis including
performing beat matching on the sound segments
performing time scaling on the sound segments
performing cross synthesis on the sound segments.
12. The computing device of claim 11 wherein listening comprises:
performing signal processing on the plurality of music
preparing segmented audio descriptors based on the plurality of music
recognizing patterns in the plurality of music.
13. The computing device of claim 11 wherein learning comprises:
preparing segmented audio descriptors based on the plurality of music
recognizing patterns in the plurality of music
classifying clusters in the plurality of music.
14. The computing device of claim 11 wherein listening comprises:
evaluating loudness indentified in the auditory spectrogram of the plurality of music
evaluating timbre of the auditory spectrogram of the plurality of music
performing onset detection of the auditory spectrogram of the plurality of music to create the sound segments
analyzing beat and tempo of the auditory spectrogram of the plurality of music
processing pitch and harmony using the sound segments created in the onset detection.
15. The computing device of claim 11 wherein creating an auditory spectrogram comprises:
creating an initial audio spectrogram based on the plurality of music
applying a transfer function to the initial audio spectrogram
applying frequency warping to the initial audio spectrogram
applying frequency masking to the initial audio spectrogram
applying temporal masking to the initial audio spectrogram.
16. The computing device of claim 11 wherein listening includes performing structural analysis.
17. The computing device of claim 16 wherein performing structural analysis comprises:
performing sound segment analysis on the plurality of music
performing beat analysis on the plurality of music
performing pattern recognition on the plurality of music.
18. The computing device of claim 11 wherein learning comprises:
performing prediction analysis on the plurality of music
performing downbeat prediction analysis on the plurality of music
performing clustering analysis on the plurality of music.
19. The computing device of claim 18 wherein performing downbeat prediction comprises:
performing segmentation on the auditory spectrogram
evaluating time lagged features.
20. A storage medium having instructions stored thereon which when executed by a processor cause the processor to perform actions comprising:
listening to a plurality of music including receiving through an audio interface live or recorded music or reading a music file from the storage medium, the listening including creating an auditory spectrogram of the plurality of music
learning from the auditory spectrogram of the plurality of music
performing concatenative synthesis based on sound segments prepared during the listening and the learning to create new music, the concatenative synthesis including
performing beat matching on the sound segments
performing time scaling on the sound segments
performing cross synthesis on the sound segments.
21. The storage medium of claim 20 wherein listening comprises:
performing signal processing on the plurality of music
preparing segmented audio descriptors based on the plurality of music
recognizing patterns in the plurality of music.
22. The storage medium of claim 20 wherein learning comprises:
preparing segmented audio descriptors based on the plurality of music
recognizing patterns in the plurality of music
classifying clusters in the plurality of music.
23. The storage medium of claim 20 wherein listening comprises:
evaluating loudness indentified in the auditory spectrogram of the plurality of music
evaluating timbre of the auditory spectrogram of the plurality of music
performing onset detection of the auditory spectrogram of the plurality of music to create the sound segments
analyzing beat and tempo of the auditory spectrogram of the plurality of music
processing pitch and harmony using the sound segments created in the onset detection.
24. The storage medium of claim 20 wherein creating an auditory spectrogram comprises:
creating an initial audio spectrogram based on the plurality of music
applying a transfer function to the initial audio spectrogram
applying frequency warping to the initial audio spectrogram
applying frequency masking to the initial audio spectrogram
applying temporal masking to the initial audio spectrogram.
25. The storage medium of claim 20 wherein listening includes performing structural analysis.
26. The storage medium of claim 25 wherein performing structural analysis comprises:
performing sound segment analysis on the plurality of music
performing beat analysis on the plurality of music
performing pattern recognition on the plurality of music.
27. The storage medium of claim 20 wherein learning comprises:
performing prediction analysis on the plurality of music
performing downbeat prediction analysis on the plurality of music
performing clustering analysis on the plurality of music.
28. The storage medium of claim 27 wherein performing downbeat prediction comprises:
performing segmentation on the auditory spectrogram
evaluating time lagged features.