Crowd sourced technique for pitch track generation
Digital signal processing and machine learning techniques can be employed in a vocal capture and performance social network to computationally generate vocal pitch tracks from a collection of vocal performances captured against a common temporal baseline such as a backing track or an original performance by a popularizing artist. In this way, crowd-sourced pitch tracks may be generated and distributed for use in subsequent karaoke-style vocal audio captures or other applications. Large numbers of performances of a song can be used to generate a pitch track. Computationally determined pitch trackings from individual audio signal encodings of the crowd-sourced vocal performance set are aggregated and processed as an observation sequence of a trained Hidden Markov Model (HMM) or other statistical model to produce an output pitch track.
1. A pitch track generation system comprising:
a first geographically distributed set of network-connected devices configured to capture audio signal encodings for respective vocal performances in correspondence with a backing track; and
a service platform configured to receive and process the audio signal encodings to computationally estimate, for each of the vocal performances, a time-varying sequence of vocal pitches and to aggregate the time-varying sequences of vocal pitches in preparation of a crowd-sourced pitch track, the aggregating based at least in part on confidence ratings determined as part of the computational estimation of vocal pitch.
2. The system of claim 1 , further comprising:
a second geographically distributed set of the network-connected devices communicatively coupled to receive the crowd-sourced pitch track for use in correspondence with the backing track as either or both of (i) vocal pitch cues and (ii) pitch correction note targets in connection with karaoke-style vocal captures at respective ones of the network-connected devices.
3. The system of claim 1 ,
wherein the service platform is further configured to time-align the received audio signal encodings to account for differing audio pipeline delays at respective of ones the network-connected devices.
4. The system of claim 1 ,
wherein the aggregating includes determining at the service platform, on a per-frame basis, a weighted distribution of pitch estimates from respective ones of the vocal performances, and wherein the weighting of individual ones of the pitch estimates is based at least in part on confidence ratings determined as part of the computational estimation of vocal pitch.
5. The system of claim 1 ,
wherein the service platform is further configured to process the aggregated time-varying sequences of vocal pitches in accordance with a statistically-based, predictive model for vocal pitch transitions.
6. The system of claim 5 ,
wherein the statistically-based, predictive model is predictive for vocal pitch transitions typical of a musical style or genre with which the backing track is associated.
7. A method of preparing a crowd-sourced pitch track, comprising:
receiving audio signal encodings from a first geographically-distributed set of network-connected devices configured to capture audio signal encodings for respective vocal performances in correspondence with a backing track;
computationally estimating, for each of the vocal performances, a time-varying sequence of vocal pictures; and
aggregating, based at least in part on confidence ratings determined as part of the computational estimation of vocal pitch, the time varying-sequence of vocal pitches in preparation of a crowd-sourced pitch track.
8. The method of claim 7 , further comprising:
supplying the crowd-sourced pitch track to a second geographically distributed set of the network-connected devices communicatively coupled to receive the crowd-sourced pitch track for use in correspondence with the backing track as either or both of (i) vocal pitch cues and (ii) pitch correction note targets in connection with karaoke-style vocal captures at respective ones of the network-connected devices.
9. The method of claim 7 , further comprising time-aligning the received audio signal encodings to account for differing audio pipeline delays at respective of ones the network-connected devices.
10. The method of claim 7 , wherein the aggregating includes determining, on a per-frame basis, a weighted distribution of pitch estimates from respective ones of the vocal performances, and wherein the weighting of individual ones of the pitch estimates is based at least in part on confidence ratings determined as part of the computational estimation of vocal pitch.
11. The method of claim 7 , further comprising processing the aggregated time-varying sequences of vocal pitches in accordance with a statistically-based, predictive model for vocal pitch transitions.
12. The system of claim 11 , wherein the statistically-based, predictive model is predictive for vocal pitch transitions typical of a musical style or genre with which the backing track is associated.