Audio stem identification systems and methods
Methods, systems and computer program products are provided for determining acoustic feature vectors of query and target items in a first vector space, and mapping the acoustic feature vectors to a second vector space having a lower dimension. The distribution of vectors in the second vector space can then be used to identify items from the same songs, and/or items that are complementary. A mapping function is trained using a machine learning algorithm, such that complementary audio items are closer in the second vector space than the first, according to a given distance metric.
1. An audio content item identifier, comprising:
at least one processor; and
at least one memory storing instructions which when executed by the at least one processor cause the at least one processor to:
receive, by a client device, a query corresponding to a query audio content item;
determine a query vector corresponding to the query audio content item;
compare, in a vector space, the query vector and a plurality of target vectors corresponding to a plurality of target audio content items, to determine likelihood values indicating, for each respective target audio content item of the plurality of target audio content items, a probability that the respective target audio content item is a match for the query audio content item, wherein the query audio content item and the target audio content items are audio stems that are configured to be inserted into an audio content item during an audio content item creation process; and
output, to an editor tool, information identifying one of the plurality of target audio content items having a highest of the likelihood values.
2. The audio content item identifier of claim 1 ,
wherein the instructions, when executed by the at least one processor further cause the at least one processor to:
generate, by the editor tool, another audio content item based on the query audio content item and the information; and
play back, by the client device, the another audio content item.
3. The audio content item identifier of claim 2 , wherein the editor tool assembles together the audio stems of the query audio content item and the one of the plurality of target audio content items having the highest of the likelihood values to generate the another audio content item.
4. The audio content item identifier of claim 1 ,
wherein the query includes a first audio type for the query audio content item;
wherein the plurality of target audio content items have at least one second audio type; and
wherein the first audio type is different from the at least one second audio type.
5. The audio content item identifier of claim 4 , wherein the query includes the at least one second audio type for the plurality of target audio content items.
6. The audio content item identifier of claim 4 ,
wherein the first audio type is one of vocals or instrumentals; and
wherein the at least one second audio type is the other of vocals or instrumentals.
7. The audio content item identifier of claim 1 , wherein the query vector and the plurality of target vectors are acoustic feature vectors.
8. The audio content item identifier of claim 7 , wherein acoustic features of the acoustic feature vectors include one or more of: vibration, distortion, presence of a vocoder, energy, valance, signal amplitude, and time-frequency progression.
9. The audio content item identifier of claim 1 , wherein the information identifies two or more of the plurality of target audio content items having the highest of the likelihood values.
10. A method of identifying an audio content item, comprising:
receiving, by a client device, a query corresponding to a query audio content item;
determining a query vector corresponding to the query audio content item;
comparing, in a vector space, the query vector and a plurality of target vectors corresponding to a plurality of target audio content items, to determine likelihood values indicating, for each respective target audio content item of the plurality of target audio content items, a probability that the respective target audio content item is a match for the query audio content item, wherein the query audio content item and the target audio content items are audio stems that are configured to be inserted into an audio content item during an audio content item creation process; and
outputting, to an editor tool, information identifying one of the plurality of target audio content items having a highest of the likelihood values.
11. The method of claim 10 , further comprising:
generating, by the editor tool, another audio content item based on the query audio content item and the information; and
playing back, by the client device, the another audio content item.
12. The method of claim 11 , wherein the editor tool assembles together the audio stems of the query audio content item and the one of the plurality of target audio content items having the highest of the likelihood values to generate the another audio content item.
13. The method of claim 10 ,
wherein the query includes a first audio type for the query audio content item;
wherein the plurality of target audio content items have at least one second audio type;
wherein the first audio type is different from the at least one second audio type; and
wherein the query includes the at least one second audio type for the plurality of target audio content items.
14. The method of claim 13 ,
wherein the first audio type is one of vocals or instrumentals; and
wherein the at least one second audio type is the other of vocals or instrumentals.
15. The method of claim 10 , wherein the query vector and the plurality of target vectors are acoustic feature vectors.
16. The method of claim 15 , wherein acoustic features of the acoustic feature vectors include one or more of: vibration, distortion, presence of a vocoder, energy, valance, signal amplitude, and time-frequency progression.
17. The method of claim 10 , wherein the information identifies two or more of the plurality of target audio content items having the highest of the likelihood values.
18. A non-transitory computer-readable medium having stored thereon one or more sequences of instructions for causing one or more processors to perform:
receiving, by a client device, a query corresponding to a query audio content item;
determining a query vector corresponding to the query audio content item;
comparing, in a vector space, the query vector and a plurality of target vectors corresponding to a plurality of target audio content items, to determine likelihood values indicating, for each respective target audio content item of the plurality of target audio content items, a probability that the respective target audio content item is a match for the query audio content item, wherein the query audio content item and the target audio content items are audio stems that are configured to be inserted into an audio content item during an audio content item creation process; and
outputting, to an editor tool, information identifying one of the plurality of target audio content items having a highest of the likelihood values.