Scalable Similarity-Based Generation Of Compatible Music Mixes
Computer-implemented techniques for identifying a second music clip that is harmonically compatible with a first music clip. The techniques include computing beat-wise pitch interval space vectors from a chromatic saliency map of the first clip. These beat-wise vectors are then combined to form a clip-wise pitch interval space vector. A second clip undergoes a similar process, resulting in a second clip-wise pitch interval space vector. Harmonic compatibility is determined by measuring the distance or similarity between the first and second clip-wise pitch interval space vectors in the pitch interval space.
1 . A method comprising:
computing a set of beat-wise pitch interval space vectors based on a chromatic saliency map for a first music clip;
forming a first clip-wise pitch interval space vector based on the set of beat-wise pitch interval space vectors, the first clip-wise pitch interval space vector formed for the first music clip;
identifying a second music clip that is harmonically compatible with the first music clip based on a distance or a similarity in a pitch interval space between a second clip-wise pitch interval space vector formed for the second music clip and the first clip-wise pitch interval space vector formed for the first music clip; and
wherein the method is performed by one or more computer systems.
2 . The method of claim 1 , further comprising:
generating a beats-per-minute agnostic chroma representation of the chromatic saliency map; and
applying a Fourier Transform to a signal of the beats-per-minute agnostic chroma representation.
3 . The method of claim 1 , wherein the set of beat-wise pitch interval space vectors are each a twelve-dimensional vector comprising six real components and six imaginary components.
4 . The method of claim 1 , further comprising concatenating the set of beat-wise pitch interval space vectors.
5 . The method of claim 1 , further comprising indexing the first music clip by the first clip-wise pitch interval space vector in an index supporting approximate nearest neighbor searches that use clip-wise pitch interval space vectors as query keys.
6 . The method of claim 1 , further comprising:
indexing the second music clip by second clip-wise pitch interval space vector in an index; and
performing an approximate nearest neighbors search of the index using the first clip-wise pitch interval space vector as a query key.
7 . The method of claim 6 , wherein the index is a quantization-based index, a tree-based index, or a graph-based index.
8 . The method of claim 1 , further comprising:
receiving a request for a music clip that is harmonically compatible with the first music clip; and
providing a response to the request, the response indicating the second music clip.
9 . The method of claim 8 , further comprising:
performing all of the computing the set of beat-wise pitch interval space vectors, the forming the first clip-wise pitch interval space vector, and the identifying the second music clip in response to receiving the request.
10 . The method of claim 1 , further comprising:
causing a graphical user interface to be presented that indicates that the first music clip is harmonically compatible with the second music clip.
11 . One or more non-transitory computer-readable media storing instructions configured for:
computing a set of beat-wise pitch interval space vectors based on a chromatic saliency map for a first music clip;
forming a first clip-wise pitch interval space vector based on the set of beat-wise pitch interval space vectors, the first clip-wise pitch interval space vector formed for the first music clip; and
identifying a second music clip that is harmonically compatible with the first music clip based on a distance or a similarity in a pitch interval space between a second clip-wise pitch interval space vector formed for the second music clip and the first clip-wise pitch interval space vector formed for the first music clip.
12 . The one or more non-transitory computer-readable media of claim 11 , the instructions further configured for:
generating a beats-per-minute agnostic chroma representation of the chromatic saliency map; and
applying a Fourier Transform to a signal of the beats-per-minute agnostic chroma representation.
13 . The one or more non-transitory computer-readable media of claim 11 , wherein the set of beat-wise pitch interval space vectors are each a twelve-dimensional vector comprising six real components and six imaginary components.
14 . The one or more non-transitory computer-readable media of claim 11 , the instructions further configured for concatenating the set of beat-wise pitch interval space vectors.
15 . The one or more non-transitory computer-readable media of claim 11 , the instructions further configured for indexing the first music clip by the first clip-wise pitch interval space vector in an index supporting approximate nearest neighbor searches that use clip-wise pitch interval space vectors as query keys.
16 . The one or more non-transitory computer-readable media of claim 11 , the instructions further configured for:
indexing the second music clip by second clip-wise pitch interval space vector in an index; and
performing an approximate nearest neighbors search of the index using the first clip-wise pitch interval space vector as a query key.
17 . The one or more non-transitory computer-readable media of claim 11 , the instructions further configured for:
receiving a request for a music clip that is harmonically compatible with the first music clip; and
providing a response to the request, the response indicating the second music clip.
18 . The one or more non-transitory computer-readable media of claim 11 , the instructions further configured for causing a graphical user interface to be presented that indicates that the first music clip is harmonically compatible with the second music clip.
19 . A system comprising:
one or more computer systems; and
instructions configured to cause the one or more computer systems to perform operations comprising:
computing a set of beat-wise pitch interval space vectors based on a chromatic saliency map for a first music clip;
forming a first clip-wise pitch interval space vector based on the set of beat-wise pitch interval space vectors, the first clip-wise pitch interval space vector formed for the first music clip; and
identifying a second music clip that is harmonically compatible with the first music clip based on a distance or a similarity in a pitch interval space between a second clip-wise pitch interval space vector formed for the second music clip and the first clip-wise pitch interval space vector formed for the first music clip.
20 . The system of claim 19 , the operations further comprising:
generating a beats-per-minute agnostic chroma representation of the chromatic saliency map for the first music clip; and
applying a Fourier Transform to a signal of the beats-per-minute agnostic chroma representation.