Method and system for predicting a latency spike category of audio and video streams to adjust a jitter buffer size accordingly
View Patent ↗Described are the architecture of such a system, algorithms for time synchronization during a multiway conferencing session, methods to fight with network imperfections such as jitter to improve synchronization, methods of introducing buffering delays to create handicaps for players with faster connections, methods which help players with synchronization (such as a synchronized metronome during a music conferencing session), methods for synchronized recording and live delivery of synchronized data to the audience watching the distributed interaction live over the Internet.
1. A computer implemented method for addressing jitter in an audio signal transmitted across a specific network, the method comprising:
receiving the audio signal;
storing data from the audio signal in a jitter buffer;
dynamically adjusting the size of the jitter buffer in order to address jitter in the specific network, the dynamically adjusting including:
analyzing characteristics of at least some latency spikes of the audio signal in the specific network;
determining from the characteristics the nature of the latency spikes and calculating average jitter characteristics;
generating models of the latency spikes based on the analyzed characteristics of the nature of the latency spikes and the calculated average jitter characteristics;
organizing the generated models of the latency spikes into different latency spike categories;
determining that a given latency spike is beginning;
analyzing characteristics of the given latency spike to predict which latency spike category applies to the given latency spike; and
changing the size of the jitter buffer according to the predicted latency spike category of the given latency spike.
2. A computer implemented method as recited in claim 1 , further including synthesizing sound to play when the jitter buffer cannot provide playable audio signal.
3. A computer implemented method as recited in claim 1 , wherein analyzing the characteristics comprises analyzing latency statistics for periods substantially within 1 minute to 5 minutes.
4. A computer implemented method as recited in claim 1 , wherein analyzing the characteristics comprises analyzing latency statistics for one or more various periods.
5. A computer system addressing jitter in an audio signal received via a specific network, the computer system comprising:
means for receiving the audio signal;
means for storing data from the audio signal in a jitter buffer;
means for dynamically adjusting the size of the jitter buffer in order to address jitter in the specific network, the means for dynamically adjusting including:
means for analyzing characteristics of at least some latency spikes of the audio signal in the specific network;
means for determining from the characteristics the nature of the latency spikes and calculating average jitter characteristics;
means for generating models of the latency spikes based on the analyzed characteristics of the nature of the latency spikes and the calculated average jitter characteristics;
means for organizing the generated models of the latency spikes into different latency spike categories;
means for determining that a given latency spike is beginning;
means for analyzing characteristics of the given latency spike to predict which latency spike category applies to the given latency spike; and
means for changing the size of the jitter buffer according to the predicted latency spike category of the given latency spike.
6. A computer system as recited in claim 5 , further including means for synthesizing sound to play when the jitter buffer cannot provide playable audio signal.
7. A computer system as recited in claim 5 , wherein analyzing the characteristics comprises analyzing latency statistics for periods substantially within 1 minute to 5 minutes.
8. A system as recited in claim 6 , wherein analyzing the characteristics comprises analyzing latency statistics for one or more various periods.
9. A computer system for addressing jitter in a specific signal transmitted across a specific network, the computer system comprising:
means for receiving a media signal;
means for storing data from the media signal in a jitter buffer; and
means for dynamically adjusting the jitter buffer in order to address jitter or jitter like effects in the specific network, said means for dynamically adjusting the jitter buffer including:
means for analyzing one or more characteristics of the media signal in the specific network;
means for determining from the characteristics of the media signal the nature of latency spikes associated with the media signal and calculating average jitter characteristics;
means for generating models of the latency spikes based on the analyzed characteristics of the media signal and the calculated average jitter characteristics;
means for organizing the generated models of the latency spikes into different latency spike categories;
means for analyzing at least some characteristics of an incoming latency spike to predict which latency spike category applies to the incoming latency spike; and
means for adjusting the jitter buffer according to analysis of the one or more characteristics of the media signal in the specific network including the prediction of the incoming latency spike category.
10. A computer implemented method for providing a distributed communication session involving synchronization of media signals transmitted across a network by devices coupled to the network, the method comprising:
receiving a given media signal, the given media signal transmitted by a given device associated with a given participant of a given distributed communication session, the given distributed communication session supported by a given network;
processing the given media signal;
storing data from the given media signal;
analyzing the given network;
determining from the analysis of the given network the nature of network latency spikes and calculating average jitter characteristics associated with the given network;
classifying network latency spikes with respect to a plurality of network spike categories according to the nature of the network latency spikes and the calculated average jitter characteristics;
predicting one or more characteristics of the specific network including predicting a category of an incoming network latency spike with respect to the plurality of network spike categories;
in response to the category of the incoming network latency spike, dynamically adjusting how data from the media signal is processed and/or stored; and
generating a localized media signal reflective of the given media signal.
11. A computer implemented method as recited in claim 10 , wherein the given media signal is an audio signal.
12. A computer implemented method as recited in claim 11 ,
wherein storing data from the given media signal includes storing data from the audio signal in a jitter buffer; and
wherein dynamically adjusting how data from the given media signal is processed and/or stored includes dynamically adjusting the size of the jitter buffer in order to address jitter in the given network.
13. A computer implemented method as recited in claim 12 , wherein dynamically adjusting the size of the jitter buffer in order to address jitter in the given network includes:
analyzing latency spikes of the audio signal in the given network;
determining that a given latency spike is beginning;
predicting characteristics of the given latency spike;
determining the category of the given latency spike based on the predicted characteristics of the given latency spike; and
changing the size of the jitter buffer according to the predicted category of the given latency spike.
14. A computer implemented method as recited in claim 10 , wherein the media signal is a video signal.
15. A computer implemented method as recited in claim 10 , further including synthesizing sound to play when a playable audio signal isn't otherwise available.
16. A computer implemented method as recited in claim 10 , wherein analyzing the characteristics comprises analyzing latency statistics for periods of substantially within 1 minute to 5 minutes.
17. A computer implemented method as recited in claim 10 , wherein analyzing the characteristics comprises analyzing latency statistics for one or more various periods.
18. The computer system as recited in claim 9 , further including means for synthesizing sound to play when the jitter buffer cannot provide a playable audio signal.
19. The computer system as recited in claim 9 , further comprising means for determining an average jitter buffer size based on the characteristics of the media signal.
20. The computer system as recited in claim 19 , further comprising means for adjusting the average jitter buffer size with respect to an average network latency.