Crowd-sourced technique for pitch track generation
Digital signal processing and machine learning techniques can be employed in a vocal capture and performance social network to computationally generate vocal pitch tracks from a collection of vocal performances captured against a common temporal baseline such as a backing track or an original performance by a popularizing artist. In this way, crowd-sourced pitch tracks may be generated and distributed for use in subsequent karaoke-style vocal audio captures or other applications. Large numbers of performances of a song can be used to generate a pitch track. Computationally determined pitch trackings from individual audio signal encodings of the crowd-sourced vocal performance set are aggregated and processed as an observation sequence of a trained Hidden Markov Model (HMM) or other statistical model to produce an output pitch track.
1. A method comprising:
receiving a plurality of audio signal encodings for respective vocal performances captured in correspondence with a backing track;
processing the audio signal encodings to computationally estimate, for each of the vocal performances, a time-varying sequence of vocal pitches;
aggregating the time-varying sequences of vocal pitches computationally estimated from the vocal performances based at least in part on confidence ratings determined as part of the computational estimation of vocal pitch; and
based at least in part on the aggregation, supplying a computer-readable encoding of a resultant pitch track for use as either or both of (i) vocal pitch cues and (ii) pitch correction note targets in connection with karaoke-style vocal captures in correspondence with the backing track.
2. The method of claim 1 , further comprising:
crowd-sourcing the received audio signal encodings from a geographically distributed set of network-connected vocal capture devices.
3. The method of claim 1 , further comprising:
time-aligning the received audio signal encodings to account for differing audio pipeline delays at respective vocal capture devices.
4. The method of claim 1 ,
wherein the aggregating includes, on a per-frame basis, a weighted distribution of pitch estimates from respective of the vocal performances.
5. The method of claim 1 , further comprising:
processing the aggregated time-varying sequences of vocal pitches in accordance with a statistically-based, predictive model for vocal pitch transitions typical of a musical style or genre with which the backing track is associated.
6. The method of claim 1 , further comprising:
supplying the resultant pitch track to network-connected vocal capture devices as part of data structure that encodes temporal correspondence of lyrics with the backing track.
7. A computer program product encoded in one or more non-transitory machine-readable media, the computer program product including instructions executable on a processor of a service platform to cause the service platform to:
receive a plurality of audio signal encodings for respective vocal performances captured in correspondence with a backing track;
process the audio signal encodings to computationally estimate, for each of the vocal performances, a time-varying sequence of vocal pitches;
aggregate the time-varying sequences of vocal pitches computationally estimated from the vocal performances based at least in part on confidence ratings determined as part of the computational estimation of vocal pitch; and
based at least in part on the aggregation, supply a computer-readable encoding of a resultant pitch track for use as either or both of (i) vocal pitch cues and (ii) pitch correction note targets in connection with karaoke-style vocal captures in correspondence with the backing track.
8. The computer program product of claim 7 , further comprising instructions executable to:
crowd-source the received audio signal encodings from a geographically distributed set of network-connected vocal capture devices.
9. The computer program product of claim 7 , further comprising instructions executable to:
time-align the received audio signal encodings to account for differing audio pipeline delays at respective vocal capture devices.
10. The computer program product of claim 7 ,
wherein the aggregating includes, on a per-frame basis, a weighted distribution of pitch estimates from respective of the vocal performances.
11. The computer program product of claim 7 , further comprising instructions executable to:
process the aggregated time-varying sequences of vocal pitches in accordance with a statistically-based, predictive model for vocal pitch transitions typical of a musical style or genre with which the backing track is associated.
12. The computer program product of claim 7 , further comprising instructions executable to:
supply the resultant pitch track to network-connected vocal capture devices as part of a data structure that encodes temporal correspondence of lyrics with the backing track.
13. A pitch track generation system comprising:
a content server configured to:
receive from a first set of geographically distributed set of network-connected devices a plurality of audio signal encodings for respective vocal performances captured in correspondence with a backing track;
process the audio signal encodings to computationally estimate, for each of the vocal performances, a time-varying sequence of vocal pitches;
aggregate the time-varying sequences of vocal pitches computationally estimated from the vocal performances based at least in part on confidence ratings determined as part of the computational estimation of vocal pitch; and
based at least in part on the aggregation, supply to a second geographically distributed set of network-connected devices a computer-readable encoding of a resultant pitch track for use as either or both of (i) vocal pitch cues and (ii) pitch correction note targets in connection with karaoke-style vocal captures in correspondence with the backing track.
14. The system of claim 13 , wherein the content server is further configured to:
time-align the received audio signal encodings to account for differing audio pipeline delays at respective vocal capture devices.
15. The system of claim 13 , wherein the aggregating includes, on a per-frame basis, a weighted distribution of pitch estimates from respective of the vocal performances.
16. The system of claim 13 , wherein the content server is further configured to:
process the aggregated time-varying sequences of vocal pitches in accordance with a statistically-based, predictive model for vocal pitch transitions typical of a musical style or genre with which the backing track is associated.
17. The system of claim 13 , wherein the content server is further configured to:
supply the resultant pitch track to the second geographically distributed set of network-connected devices as part of a data structure that encodes temporal correspondence of lyrics with the backing track.