Crowd-sourced device latency estimation for synchronization of recordings in vocal capture applications
Latency on different devices (e.g., devices of differing brand, model, vintage, etc.) can vary significantly and tens of milliseconds can affect human perception of lagging and leading components of a performance. As a result, use of a uniform latency estimate across a wide variety of devices is unlikely to provide good results, and hand-estimating round-trip latency across a wide variety of devices is costly and would constantly need to be updated for new devices. Instead, a system has been developed for crowdsourcing latency estimates.
1. A system comprising:
a network-resident media content server or service platform;
a non-transitory memory;
one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the network-resident media content server or service platform to perform operations comprising:
receiving, for audio signal captures performed at a first subset of a plurality of network-connected computing devices configured for vocal performance capture, computationally-determined temporal offsets between audio features of the respective audio captures and one or more corresponding audio signals,
wherein the first subset of the plurality of network-connected computing devices are of a consistent hardware and software configuration, and
wherein at least some of the plurality of devices differ in hardware or software configuration from those of the first subset;
based on the computationally-determined temporal offsets, characterizing a round-trip latency through audio systems of devices that match the hardware and software configuration of the first subset; and
supplying the characterized round-trip latency to device instances of the first subset including those for which no temporal offset has been explicitly determined based on audio signal capture at the respective device instance for use in connection with subsequent vocal captures thereon.
2. The system of claim 1 , wherein consistency of hardware and software configuration shared by devices of the first subset includes consistency of plural attributes selected from the set of:
hardware model;
firmware version;
operating system version; and
audio subpath(s) used for audio playback and capture.
3. The system of claim 1 ,
wherein audio signal capture-based determinations of round-trip latency are computed based on a first number of network-connected computing device instances of a first type;
wherein a second number of network-connected computing device instances of the first type are supplied with the characterized round-trip latency for use in connection with subsequent vocal captures thereon; and
wherein the second number substantially exceeds the first number by a factor of at least ten (10×).
4. The system of claim 1 , further comprising software executable on a least some of the network-connected computing device instances of a first type to, at each such device instance,
audibly render a backing track; and
capture vocals performed by a user against the backing track for use in the characterization of round-trip latency for the network-connected computing devices of the first type,
wherein the computational determination of temporal offset is between respective audio features of vocal captures and backing track.
5. The system of claim 1 , wherein the computational determination of temporal offset is performed at the network-resident media content server or service platform.
6. The system of claim 1 , further comprising:
software executable on at least some of the network-connected computing device instances of a first type to, at each such device instance, supply a test signal at a respective audio output thereof and to capture a corresponding audio signal at an audio input thereof for use in the characterization of round-trip latency for the network-connected computing devices of the first type.
7. The system of claim 1 , wherein the computing devices are selected from the set of a mobile phone, a personal digital assistant, a laptop or notebook computer, a pad-type computer and a net book.
8. The system of claim 1 , wherein at least some of the computing devices are selected from the set of audiovisual media devices and connected set-top boxes.
9. A system, as in claim 1 , wherein the computational determination of temporal offset is performed at one or more network-connected computing devices of the first subset of the plurality of network-connected computing devices.
10. A method comprising:
crowdsourcing round-trip latency characterizations for a plurality of network-connected computing devices configured for vocal performance capture;
for a first subset of the plurality of network-connected computing devices, the first subset having a consistent hardware and software configuration:
sampling a first number of network-connected computing devices of the first subset to characterize round-trip latency through audio systems of substantially all devices of the first subset, wherein the sampling includes audio signal captures and determinations of temporal offset between audio features of the respective audio signal captures and corresponding audio signals; and
supplying the characterized round-trip latency to a first network-connected computing device of the first subset for use in connection with subsequent vocal captures thereon, wherein the first network-connected computing device is not of the first number of network-connected computing devices.
11. The method of claim 10 ,
wherein the audio signal captures include vocal audio captured karaoke-style at a particular device instance against an audible rendering of a corresponding backing track.
12. The method of claim 10 ,
wherein the audio signal captures include captures, at an audio input of a particular device instance, a test signal supplied at audio output of the particular device instance.
13. The method of claim 10 , wherein the sampling includes:
audibly rendering a backing track at a particular device instance; and
at the device instance, capturing vocals performed by a user against the backing track,
wherein the determination of temporal offset is between respective audio features of the vocal capture and the backing track.
14. The method of claim 10 , wherein the determination of temporal offset is performed at a network-resident media content server or service platform.
15. The method of claim 10 , wherein the sampling includes:
supplying a test signal at a respective audio output of a particular device instance; and
capturing a corresponding audio signal at an audio input of the particular device instance,
wherein the determination of temporal offset is between respective audio features of the test signal as supplied and captured.