IP Library Granted Patent US 11,461,649
Granted Patent B2
US 11,461,649 · App. 16/823,538 · Granted Oct 4, 2022

Searching for music

Inventors: Jongpil Lee (Seoul, KR); Nicholas J. Bryan (Belmont, CA); Justin J. Salamon (San Francisco, CA); Zeyu Jin (San Francisco, CA)
Assignee: Adobe Inc.
G06N3/08G10H1/0008G10H2210/036G10H2210/056G10H2210/076G10L25/51
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,461,649
App. No.
16/823,538
Granted
Oct 4, 2022
Kind
B2
Abstract

In implementations of searching for music, a music search system can receive a music search request that includes a music file including music content. The music search system can also receive a selected musical attribute from a plurality of musical attributes. The music search system includes a music search application that can generate musical features of the music content, where a respective one or more of the musical features correspond to a respective one of the musical attributes. The music search application can then compare the musical features that correspond to the selected musical attribute to audio features of audio files, and determine similar audio files to the music file based on the comparison of the musical features to the audio features of the audio files.

Claims (62)

1. In a digital medium environment for searching for music, a method implemented by a computing device, the method comprising:

receiving a music search request that includes a music file including music content;

receiving a selected musical attribute from a plurality of musical attributes;

generating musical features of the music content, a respective one or more of the musical features corresponding to a respective one of the plurality of musical attributes;

comparing the respective one or more musical features corresponding to the selected musical attribute to audio features of a plurality of audio files; and

determining similar audio files to the music file based on the comparing of the respective one or more musical features to the audio features of the plurality of audio files.

2. The method as described in claim 1 , wherein the plurality of musical attributes include two or more of a genre attribute, a mood attribute, an instrument attribute, and a tempo attribute.

3. The method as described in claim 1 , further comprising providing audio content of the similar audio files for playback by the computing device.

4. The method as described in claim 1 , wherein the generating the musical features of the music content includes generating a feature vector that includes a representation of each of the musical features, the feature vector partitioned into independent subspaces that each represent a respective musical attribute of the plurality of musical attributes.

5. The method as described in claim 4 , wherein the comparing of the respective one or more musical features to the audio features includes determining distances of the feature vector to feature vectors representing the audio features over one of the independent subspaces that corresponds to the selected musical attribute.

6. The method as described in claim 5 , further comprising:

comparing the distances to a distance similarity threshold; and

determining the similar audio files to the music file based on one or more of the distances that are less than the distance similarity threshold.

7. The method as described in claim 5 , further comprising:

ranking the plurality of audio files based on the distances; and

determining the similar audio files to the music file based on the ranking.

8. The method as described in claim 1 , wherein the generating the musical features of the music content includes:

generating a plurality of additional features from respective portions of the music file; and

averaging the plurality of additional features to generate the musical features.

9. The method as described in claim 1 , further comprising:

receiving an additional music file including additional music content as part of the music search request;

generating additional musical features of the additional music content, a respective one or more of the additional musical features also corresponding to a respective one of the plurality of musical attributes;

combining the musical features of the music content with the additional musical features of the additional music content into averaged musical features;

comparing the averaged musical features corresponding to the selected musical attribute to the audio features of the plurality of audio files; and

determining the similar audio files to the music file based on the comparing of the averaged musical features to the audio features.

10. The method as described in claim 1 , further comprising:

generating a representation of frequencies of the music content in the music file;

and generating, with a neural network, a feature vector that includes the musical features of the music content based on the representation of frequencies, wherein the neural network has coefficients determined from triplets of the audio files, each triplet including an anchor file having anchor content, a positive file having positive content that is similar to the anchor content according to at least one of the plurality of musical attributes, and a negative file having negative content that is dissimilar to the anchor content according to the at least one of the plurality of musical attributes.

11. In a digital medium environment for searching for music, a method implemented by a computing device, the method comprising:

receiving a music search request that includes a music file;

receiving a selected musical attribute from a plurality of musical attributes;

determining a feature vector for the music file, the feature vector partitioned to include multiple dimensions corresponding to a respective one of the plurality of the musical attributes;

determining distances between the feature vector and additional feature vectors of a plurality of audio files over the multiple dimensions that correspond to the selected musical attribute; and

determining, based on the distances, one or more of the audio files.

12. The method as described in claim 11 , further comprising:

receiving a degree of desired similarity;

ranking the distances; and

determining the one or more of the audio files based on the degree of desired similarity and the ranking.

13. The method as described in claim 11 , further comprising:

receiving an additional selected musical attribute from the plurality of musical attributes; and

wherein the determining the distances includes determining a first distance over the multiple dimensions corresponding to the selected musical attribute and a second distance over the multiple dimensions corresponding to the additional selected musical attribute.

14. The method as described in claim 11 , wherein the determining the feature vector for the music file includes:

determining a plurality of music feature vectors based on respective portions of the music file; and

combining two or more of the music feature vectors to determine the feature vector for the music file.

15. A music search system implemented by a computing device for searching for music in a digital medium environment, the music search system comprising:

a memory to maintain elements that represent a plurality of musical attributes; and

a processor system to implement a music search application at least partially in hardware of the computing device to:

receive a music search request that includes a music file including music content;

receive a selected musical attribute from the plurality of musical attributes;

generate musical features of the music content, a respective one or more of the musical features corresponding to a respective one of the plurality of musical attributes;

compare the respective one or more musical features corresponding to the selected musical attribute to audio features of a plurality of audio files; and

determine similar audio files to the music file based on the respective one or more musical features compared to the audio features of the plurality of audio files.

16. A music search system as described in claim 15 , wherein the plurality of musical attributes include two or more of a genre attribute, a mood attribute, an instrument attribute, and a tempo attribute.

17. A music search system as described in claim 15 , wherein the music search application is implemented to provide audio content of the similar audio files for playback by the computing device.

18. A music search system as described in claim 15 , wherein, to generate the musical features of the music content, the music search application is implemented to generate a feature vector that includes a representation of each of the musical features, the feature vector partitioned into independent subspaces that each represent a respective musical attribute of the plurality of musical attributes.

19. A music search system as described in claim 18 , wherein, to compare the respective one or more musical features to the audio features, the music search application is implemented to determine distances of the feature vector to feature vectors representing the audio features over one of the independent subspaces that corresponds to the selected musical attribute.

20. A music search system as described in claim 15 , wherein the music search application is implemented to:

receive an additional music file including additional music content as part of the music search request;

generate additional musical features of the additional music content, a respective one or more of the additional musical features also corresponding to a respective one of the plurality of musical attributes;

combine the musical features of the music content with the additional musical features of the additional music content into averaged musical features;

compare the averaged musical features corresponding to the selected musical attribute to the audio features of the plurality of audio files; and

determine the similar audio files to the music file based on the averaged musical features compared to the audio features.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 19, 2020
From: LEE, JONGPIL; BRYAN, NICHOLAS J.; SALAMON, JUSTIN J.; JIN, ZEYU
To: ADOBE INC.
Reel/Frame 052164/0813 →
Continuity (1)
Related Publication 20210294840A1 · Sep 23, 2021