IP Library Granted Patent US 7,836,194
Granted Patent B2
US 7,836,194 · App. 11/868,114 · Granted Nov 16, 2010

Method and system for dynamic latency management and drift correction

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,836,194
App. No.
11/868,114
Granted
Nov 16, 2010
Kind
B2
Abstract

Latency in a real-time electronic communication is dynamically managed. A communication delay arising from a receiving data buffer is measured and a latency adjustment necessary to adjust the size of the communication delay to within a predetermined range and an optimal range for a size of the communication delay are determined. Using these parameters, the number of samples for an audio playback data block passing through the receiving data buffer is modified.

Claims (69)

1. A method for dynamic latency management for electronic communications comprising:

receiving first electronic communications over a communication link;

measuring, over a time window, delays associated with the first electronic communications received over the communication link;

using a first operating range that includes a first upper bound and a first lower bound to determine a first latency adjustment for the first electronic communications that defines a number of samples to insert or remove;

performing a first latency adjustment operation for the first electronic communications received over the communication link based on the first latency adjustment determined using the first operating range that includes the first upper bound and the first lower bound;

adjusting, using at least one processor, the first operating range used to determine the first latency adjustment for the first electronic communications received over the communication link based on the delays measured over the time window, the adjusting including determining a second operating range that differs from the first operating range by adjusting at least one of the first upper bound and the first lower bound;

receiving, subsequent to adjusting the first operating range, second electronic communications over the communication link;

determining a delay associated with the second electronic communications received over the communication link;

comparing the determined delay with the second operating range determined by adjusting at least one of the first upper bound and the first lower bound included in the first operating range; and

based on comparison results, determining a second latency adjustment for the second electronic communications that defines a number of samples to insert or remove;

performing a second latency adjustment operation for the second electronic communications received over the communication link based on the second latency adjustment determined using the second operating range that differs from the first operating range.

2. The method of claim 1 wherein measuring, over a time window, delays associated with the first electronic communications received over the communication link includes measuring, over a time window defined based on a time necessary to capture a complete cycle of fluctuations in a receiving buffer associated with the communication link, delays associated with the first electronic communications received over the communication link.

3. The method of claim 1 wherein using the first operating range that includes the first upper bound and the first lower bound to determine the first latency adjustment for the first electronic communications that defines a number of samples to insert or remove comprises:

comparing at least one delay associated with the first electronic communications received over the communication link with the first operating range; and

based on the comparison of the at least one delay with the predetermined first operating range, determining the first latency adjustment for the first electronic communications that defines a number of samples to insert or remove.

4. The method of claim 3 wherein adjusting the first operating range based on the delays measured over the time window includes:

decreasing the first upper bound and the first lower bound conditioned on measuring at least one delay that exceeds a threshold amount during the time window, and

increasing the first upper bound and the first lower bound conditioned on measuring no delays during the time window.

5. The method of claim 1 wherein the first and second electronic communications are real-time audio communications.

6. The method of claim 1 wherein determining the second operating range that differs from the first operating range by adjusting at least one of the first upper bound and the first lower bound comprises determining a second operating range that differs from the first operating range by increasing the first upper bound.

7. The method of claim 1 wherein determining the second operating range that differs from the first operating range by adjusting at least one of the first upper bound and the first lower bound comprises determining a second operating range that differs from the first operating range by decreasing the first upper bound.

8. The method of claim 1 wherein determining the second operating range that differs from the first operating range by adjusting at least one of the first upper bound and the first lower bound comprises determining a second operating range that differs from the first operating range by increasing the first lower bound.

9. The method of claim 1 wherein determining the second operating range that differs from the first operating range by adjusting at least one of the first upper bound and the first lower bound comprises determining a second operating range that differs from the first operating range by decreasing the first lower bound.

10. The method of claim 1 wherein determining the second operating range that differs from the first operating range by adjusting at least one of the first upper bound and the first lower bound comprises determining a second operating range that differs from the first operating range by increasing the first upper bound and the first lower bound.

11. The method of claim 1 wherein determining the second operating range that differs from the first operating range by adjusting at least one of the first upper bound and the first lower bound comprises determining a second operating range that differs from the first operating range by decreasing the first upper bound and the first lower bound.

12. The method of claim 1 wherein determining the second latency adjustment for the second electronic communications that defines a number of samples to insert or remove comprises determining the second latency adjustment by multiplying a number of out-of-bounds samples by an empirically determined convergent factor.

13. The method of claim 1 :

wherein determining the delay associated with the second electronic communications received over the communication link comprises:

measuring, over a first time window, delays associated with the second electronic communications received over the communication link, and

based on the measured delays associated with the second electronic communications, computing an average measured delay in terms of samples;

wherein comparing the determined delay with the second operating range determined by adjusting at least one of the first upper bound and the first lower bound included in the first operating range comprises comparing the average measured delay to a second upper bound and a second lower bound included in the second operating range; and

wherein determining a second latency adjustment for the second electronic communications that defines a number of samples to insert or remove comprises:

when the comparison reveals that the average measured delay is greater than the second upper bound included in the second operating range, computing a number of out-of-bounds samples using the second upper bound and the average measured delay;

when the comparison reveals that the average measured delay is less than the second lower bound included in the second operating range, computing the number of out-of-bounds samples using the second lower bound and the average measured delay;

when the comparison reveals that the average measured delay is within the second operating range, determining that there are zero out-of-bounds samples; and

determining the second latency adjustment for the second electronic communications that defines a number of samples to insert or remove using the number of out-of-bounds samples, a number of playblock blocks in a receiving buffer that receives electronic communications from the communication link, and a convergence factor with an absolute value greater than one.

14. The method of claim 13 wherein measuring, over the time window, delays associated with the first electronic communications received over the communication link comprises measuring, over a second time window defined based on a time necessary to capture a complete cycle of fluctuations in the receiving buffer that receives electronic communications from the communication link, delays associated with the first electronic communications received over the communication link, the second time window being greater than the first time window.

15. At least one computer-readable medium storing one or more computer programs including instructions that, when executed, perform operations comprising:

receiving first electronic communications over a communication link;

measuring, over a time window, delays associated with the first electronic communications received over the communication link;

using a first operating range that includes a first upper bound and a first lower bound to determine a first latency adjustment for the first electronic communications that defines a number of samples to insert or remove;

performing a first latency adjustment operation for the first electronic communications received over the communication link based on the first latency adjustment determined using the first operating range that includes the first upper bound and the first lower bound;

adjusting the first operating range used to determine the first latency adjustment for the first electronic communications received over the communication link based on the delays measured over the time window, the adjusting including determining a second operating range that differs from the first operating range by adjusting at least one of the first upper bound and the first lower bound;

receiving, subsequent to adjusting the first operating range, second electronic communications over the communication link;

determining a delay associated with the second electronic communications received over the communication link;

comparing the determined delay with the second operating range determined by adjusting at least one of the first upper bound and the first lower bound included in the first operating range; and

based on comparison results, determining a second latency adjustment for the second electronic communications that defines a number of samples to insert or remove;

performing a second latency adjustment operation for the second electronic communications received over the communication link based on the second latency adjustment determined using the second operating range that differs from the first operating range.

16. The medium of claim 15 wherein measuring, over a time window, delays associated with the first electronic communications received over the communication link includes measuring, over a time window defined based on a time necessary to capture a complete cycle of fluctuations in a receiving buffer associated with the communication link, delays associated with the first electronic communications received over the communication link.

17. The medium of claim 15 wherein using the first operating range that includes the first upper bound and the first lower bound to determine the first latency adjustment for the first electronic communications that defines a number of samples to insert or remove comprises:

comparing at least one delay associated with the first electronic communications received over the communication link with the first operating range; and

based on the comparison of the at least one delay with the first operating range, determining the first latency adjustment for the first electronic communications that defines a number of samples to insert or remove.

18. The medium of claim 17 wherein adjusting the first operating range based on the delays measured over the time window includes:

decreasing the first upper bound and the first lower bound conditioned on measuring at least one delay that exceeds a threshold amount during the time window, and

increasing the first upper bound and the first lower bound conditioned on measuring no delays during the time window.

19. The medium of claim 15 wherein the first and second electronic communications are real-time audio communications.

20. An electronic system comprising:

at least one computer; and

at least one computer-readable medium coupled to the at least one computer having instructions stored thereon which, when executed by the at least one computer, causes the at least one computer to perform operations comprising:

receiving first electronic communications over a communication link;

measuring, over a time window, delays associated with the first electronic communications received over the communication link;

using a first operating range that includes a first upper bound and a first lower bound to determine a first latency adjustment for the first electronic communications that defines a number of samples to insert or remove;

performing a first latency adjustment operation for the first electronic communications received over the communication link based on the first latency adjustment determined using the first operating range that includes the first upper bound and the first lower bound;

adjusting the first operating range used to determine the first latency adjustment for the first electronic communications received over the communication link based on the delays measured over the time window, the adjusting including determining a second operating range that differs from the first operating range by adjusting at least one of the first upper bound and the first lower bound;

receiving, subsequent to adjusting the first operating range, second electronic communications over the communication link;

determining a delay associated with the second electronic communications received over the communication link;

comparing the determined delay with the second operating range determined by adjusting at least one of the first upper bound and the first lower bound included in the first operating range; and

based on comparison results, determining a second latency adjustment for the second electronic communications that defines a number of samples to insert or remove;

performing a second latency adjustment operation for the second electronic communications received over the communication link based on the second latency adjustment determined using the second operating range that differs from the first operating range.

Assignments (7)
CHANGE OF NAME Recorded Dec 20, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058961/0436 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 3, 2012
From: AOL INC.
To: FACEBOOK, INC.
Reel/Frame 028487/0602 →
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENT RIGHTS Recorded Nov 16, 2010
From: BANK OF AMERICA, N A
To: AOL INC; AOL ADVERTISING INC; GOING INC; LIGHTNINGCAST LLC; MAPQUEST, INC; NETSCAPE COMMUNICATIONS CORPORATION; QUIGO TECHNOLOGIES LLC; SPHERE SOURCE, INC; TACODA LLC; TRUVEO, INC; YEDDA, INC
Reel/Frame 025323/0416 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 31, 2009
From: AOL LLC
To: AOL INC.
Reel/Frame 023723/0645 →
SECURITY AGREEMENT Recorded Dec 14, 2009
From: AOL INC.; AOL ADVERTISING INC.; BEBO, INC.; ICQ LLC; GOING, INC.; LIGHTNINGCAST LLC; MAPQUEST, INC.; NETSCAPE COMMUNICATIONS CORPORATION; QUIGO TECHNOLOGIES LLC; SPHERE SOURCE, INC.; TACODA LLC; TRUVEO, INC.; YEDDA, INC.
To: BANK OF AMERICAN, N.A. AS COLLATERAL AGENT
Reel/Frame 023649/0061 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 16, 2009
From: MANTEGNA, JOHN; WU, SHUWU
To: AMERICA ONLINE, INC.
Reel/Frame 022831/0807 →
CHANGE OF NAME Recorded Jun 16, 2009
From: AMERICA ONLINE, INC.
To: AOL LLC
Reel/Frame 022832/0340 →