IP Library Granted Patent US 11,997,459
Granted Patent B2
US 11,997,459 · App. 17/450,537 · Granted May 28, 2024

Crowd-sourced device latency estimation for synchronization of recordings in vocal capture applications

Inventors: Amanda Chaudhary (San Francisco, CA); David Steinwedel (San Francisco, CA); John Shimmin (San Francisco, CA); Lance Jabr (San Francisco, CA); Randal Leistikow (Denver, CO)
Assignee: Smule, Inc.
H04R29/004G06F3/162G06F3/165G10L25/60H04R29/005H04R2420/07H04S2400/15
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,997,459
App. No.
17/450,537
Granted
May 28, 2024
Kind
B2
Abstract

Latency on different devices (e.g., devices of differing brand, model, vintage, etc.) can vary significantly and tens of milliseconds can affect human perception of lagging and leading components of a performance. As a result, use of a uniform latency estimate across a wide variety of devices is unlikely to provide good results, and hand-estimating round-trip latency across a wide variety of devices is costly and would constantly need to be updated for new devices. Instead, a system has been developed for crowdsourcing latency estimates.

Claims (26)

1. A device, comprising:

one or more processors coupled to at least one non-transitory memory and configured to read instructions from the at least one non-transitory memory to perform steps including:

estimating a round-trip latency through a first audio subsystem of a first computing device based on automatically determined audio feature analysis information of a first audio capture captured by the first computing device; and

supplying the round-trip latency to a second computing device for use in connection with vocal performance captures by the second computing device.

2. The device of claim 1 , wherein the round-trip latency estimate includes an input latency and an output latency through the first audio subsystem of the first computing device.

3. The device of claim 1 , wherein the first and second computing devices are of a consistent hardware and software configuration.

4. The device of claim 3 , wherein the consistent hardware and software configuration includes consistency of at least one of hardware model, firmware version, operating system version, and audio subpath(s) for audio playback and capture.

5. The device of claim 1 , wherein the audio feature analysis information of the first audio capture is automatically determined by an audio feature analysis device.

6. The device of claim 5 , wherein the first audio capture is associated with a first corresponding audio signal, wherein the first corresponding audio signal includes a signal with known temporal features.

7. The device of claim 1 , wherein the first audio capture includes a vocal performance associated with a first corresponding audio signal.

8. The device of claim 7 , wherein the first corresponding audio signal includes a backing track.

9. The device of claim 7 , wherein the round-trip latency is determined based on syllables computationally identified in the vocal performance.

10. The device of claim 1 , wherein the steps further comprise:

estimating the round-trip latency based on a second audio capture captured at a third computing device.

11. The device of claim 10 , wherein the first and third computing devices are of a consistent hardware and software configuration.

12. A method, comprising:

estimating a round-trip latency through a first audio subsystem of a first computing device based on automatically determined audio feature analysis information of a first audio capture captured by the first computing device; and

supplying the round-trip latency to a second computing device for use in connection with vocal performance captures by the second computing device.

13. The method of claim 12 , wherein the round-trip latency estimate includes an input latency and an output latency through the audio subsystem of the first computing device.

14. The method of claim 12 , wherein the first and second computing devices are of a consistent hardware and software configuration.

15. The method of claim 14 , wherein the consistent hardware and software configuration includes consistency of at least one of hardware model, firmware version, operating system version, and audio subpath(s) for audio playback and capture.

16. The method of claim 12 , wherein the audio feature analysis information of the first audio capture is automatically determined by an audio feature analysis device.

17. The method of claim 12 , wherein the first audio capture includes a vocal performance.

18. The method of claim 17 , wherein the round-trip latency is determined based on syllables computationally identified in the vocal performance.

19. The method of claim 12 , wherein the steps further comprise:

estimating the round-trip latency based on a second audio capture captured at a third computing device.

Assignments (5)
SECURITY INTEREST Recorded Dec 30, 2024
From: SMULE, INC.
To: WESTERN ALLIANCE BANK
Reel/Frame 069703/0571 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 15, 2023
From: CHAUDHARY, AMANDA; STEINWEDEL, DAVID; SHIMMIN, JOHN; LEISTIKOW, RANDAL
To: SMULE, INC.
Reel/Frame 065883/0161 →
SECURITY INTEREST Recorded Sep 16, 2022
From: SMULE, INC.
To: WESTERN ALLIANCE BANK
Reel/Frame 061127/0422 →
CONFIDENTIAL INFORMATION AGREEMENT Recorded Mar 4, 2022
From: JABR, LANCE
To: SMULE, INC.
Reel/Frame 059793/0609 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 3, 2022
From: CHAUDHARY, AMANDA; STEINWEDEL, DAVID; SHIMMIN, JOHN; JABR, LANCE; LEISTIKOW, RANDAL
To: SMULE, INC.
Reel/Frame 058873/0599 →
Continuity (6)
Continuation 16403939 · May 6, 2019
Continuation 15178234 · Jun 9, 2016
Continuation In Part 14216136 · Mar 17, 2014
Provisional Application 62173337 · Jun 9, 2015
Provisional Application 61798869 · Mar 15, 2013
Related Publication 20220103958A1 · Mar 31, 2022