Crowd-sourced technique for pitch track generation
Digital signal processing and machine learning techniques can be employed in a vocal capture and performance social network to computationally generate vocal pitch tracks from a collection of vocal performances captured against a common temporal baseline such as a backing track or an original performance by a popularizing artist. In this way, crowd-sourced pitch tracks may be generated and distributed for use in subsequent karaoke-style vocal audio captures or other applications. Large numbers of performances of a song can be used to generate a pitch track. Computationally determined pitch trackings from individual audio signal encodings of the crowd-sourced vocal performance set are aggregated and processed as an observation sequence of a trained Hidden Markov Model (HMM) or other statistical model to produce an output pitch track.
1. A method comprising:
receiving a plurality of audio signal encodings for respective vocal performances captured in correspondence with a backing track;
preprocessing the plurality of audio signal encodings for respective vocal performances, wherein the preprocessing comprises one or more of: time-aligning the audio signal encodings for respective vocal performances based on latency metadata, and normalizing the audio signal encodings for respective vocal performances;
identifying, from the plurality of audio signal encodings, a subset of audio signal encodings;
processing the subset of audio signal encodings to computationally estimate, for each of the vocal performances corresponding to the subset of audio signal encodings, a time-varying sequence of vocal pitches;
aggregating the time-varying sequences of vocal pitches computationally estimated from the vocal performances; and
based at least in part on the aggregation, supplying a computer-readable encoding of a resultant pitch track for use as either or both of (i) vocal pitch cues and (ii) pitch correction note targets in connection with karaoke-style vocal captures in correspondence with the backing track.
2. The method of claim 1 , wherein the step of identifying further comprises utilizing metadata associated with the plurality of audio signal encodings to identify the subset of audio signal encodings.
3. The method of claim 1 , wherein the step of identifying further comprises extracting one or more audio features from each of the plurality of audio signal encodings to identify the subset of audio signal encodings.
4. The method of claim 3 , further comprising:
identifying the subset of audio signal encodings based on a clustering technique applied to the extracted audio features.
5. The method of claim 3 , further comprising:
identifying the subset of audio signal encodings based on a distance measure calculated from the extracted audio features.
6. The method of claim 1 , wherein the aggregating is based at least in part on the confidence ratings determined as part of the computational estimation of vocal pitch.
7. A computer program product encoded in one or more non-transitory machine-readable media, the computer program product including instructions executable on a processor of a service platform to cause the service platform to:
receive a plurality of audio signal encodings for respective vocal performances captured in correspondence with a backing track;
preprocess the plurality of audio signal encodings for respective vocal performances, wherein the preprocessing comprises one or more of: time-aligning the audio signal encodings for respective vocal performances based on latency metadata, and normalizing the audio signal encodings for respective vocal performances;
identify, from the plurality of audio signal encodings, a subset of audio signal encodings;
process the subset of audio signal encodings to computationally estimate, for each of the vocal performances corresponding to the subset of audio signal encodings, a time-varying sequence of vocal pitches;
aggregate the time-varying sequences of vocal pitches computationally estimated from the vocal performances; and
based at least in part on the aggregation, supply a computer-readable encoding of a resultant pitch track for use as either or both of (i) vocal pitch cues and (ii) pitch correction note targets in connection with karaoke-style vocal captures in correspondence with the backing track.
8. The computer program product of claim 7 , further comprising instructions to cause the service platform to utilize metadata associated with the plurality of audio signal encodings to identify the subset of audio signal encodings.
9. The computer program product of claim 7 , further comprising instructions to cause the service platform to extract one or more audio features from each of the plurality of audio signal encodings to identify the subset of audio signal encodings.
10. The computer program product of claim 9 , further comprising instructions to cause the service platform to identify the subset of audio signal encodings based on a clustering technique applied to the extracted audio features.
11. The computer program product of claim 9 , further comprising instructions to cause the service platform to identify the subset of audio signal encodings based on a distance measure calculated from the extracted audio features.
12. The computer program product of claim 7 , further comprising instructions to cause the service platform to aggregate based at least in part on the confidence ratings determined as part of the computational estimation of vocal pitch.
13. A pitch track generation system comprising:
a content server configured to:
receive a plurality of audio signal encodings for respective vocal performances captured in correspondence with a backing track;
preprocess the plurality of audio signal encodings for respective vocal performances, wherein the preprocessing comprises one or more of: time-aligning the audio signal encodings for respective vocal performances based on latency metadata, and normalizing the audio signal encodings for respective vocal performances;
identify, from the plurality of audio signal encodings, a subset of audio signal encodings;
process the subset of audio signal encodings to computationally estimate, for each of the vocal performances corresponding to the subset of audio signal encodings, a time-varying sequence of vocal pitches;
aggregate the time-varying sequences of vocal pitches computationally estimated from the vocal performances; and
based at least in part on the aggregation, supply a computer-readable encoding of a resultant pitch track for use as either or both of (i) vocal pitch cues and (ii) pitch correction note targets in connection with karaoke-style vocal captures in correspondence with the backing track.
14. The system of claim 13 , wherein the content server is further configured to utilize metadata associated with the plurality of audio signal encodings to identify the subset of audio signal encodings.
15. The system of claim 13 , wherein the content server is further configured to extract one or more audio features from each of the plurality of audio signal encodings to identify the subset of audio signal encodings.
16. The system of claim 15 , wherein the content server is further configured to identify the subset of audio signal encodings based on a clustering technique applied to the extracted audio features.
17. The system of claim 15 , wherein the content server is further configured to identify the subset of audio signal encodings based on a distance measure calculated from the extracted audio features.
18. The system of claim 13 , wherein the content server is further configured to aggregate based at least in part on the confidence ratings determined as part of the computational estimation of vocal pitch.