Granted Patent B2

US 10,015,103 · App. 15/152,793 · Granted Jul 3, 2018

Inventors: Sascha Kümmel (Dresden, DE); Mathias Buhr (Freiberg, DE)

Assignee: GetGo, Inc.

H04L47/54H04L41/0816H04L43/0835

Abstract

Claims (63)

1. A method of handling packet loss during a communication session, the method comprising:

storing audio data received by an electronic device from a remote device during the communication session into a buffer contained within the electronic device, before the received audio data is provided from the buffer, on a first-in first out basis, to output logic of the electronic device;

during the communication session, detecting a current interactivity mode, wherein the current interactivity mode is selected from the group consisting of a) non-interactive, indicating a low level of interactivity between participants in the communication session, and b) interactive, indicating a high level of interactivity between participants in the communication session;

storing an indication of the detected current interactivity mode;

detecting, during the communication session, that at least one packet of audio data transmitted to the electronic device from a remote device has been lost prior to being delivered to the electronic device; and

in response to detecting that at least one packet of audio data transmitted to the electronic device from the remote device has been lost prior to being delivered to the electronic device, a) checking the stored indication of the current interactivity mode, and b) in response to the stored indication of the current interactivity mode indicating that the current interactivity mode is non-interactive, automatically increasing a total amount of received audio data concurrently stored in the buffer contained within the electronic device.

2. The method of claim 1 , further comprising, also in response to both a) detecting that at least one packet of audio data transmitted to the electronic device from the remote device has been lost prior to being delivered to the electronic device, and b) detecting that the current interactivity mode is non-interactive, indicating to the remote device that the remote device is to begin transmitting redundant data to the electronic device in order to enable error correction by the electronic device.

3. The method of claim 1 , further comprising:

detecting that the current interactivity mode has changed from non-interactive to interactive; and

in response to detecting that the current interactivity mode has changed from non-interactive to interactive, automatically reducing the total amount of received audio data concurrently stored in the buffer contained within the audio device.

4. The method of claim 3 , wherein automatically reducing the total amount of received audio data concurrently stored in the buffer contained within the audio device includes a) identifying audio data representing silence stored in the buffer, and b) removing the audio data representing silence from the buffer.

5. The method of claim 3 , wherein automatically reducing the total amount of received audio data concurrently stored in the buffer contained in the audio device includes a) identifying audio data representing noise stored in the buffer, and b) removing the audio data representing noise from the buffer.

6. The method of claim 3 , wherein the electronic device comprises a participant device; and

wherein detecting that the current interactivity mode has changed from non-interactive to interactive includes detecting that an audio input of the electronic device has transitioned from muted to un-muted.

7. The method of claim 3 , wherein the electronic device comprises a participant device, and

wherein detecting that the current interactivity mode has changed from non-interactive to interactive includes detecting speech received by an audio input device of the electronic device.

8. The method of claim 3 , wherein the electronic device comprises a participant device, and

wherein detecting that the current interactivity mode has changed from non-interactive to interactive includes detecting an audio signal output from an audio input device of the electronic device.

9. A method of handling packet loss during a communication session, the method comprising:

storing audio data received by an electronic device from a remote device during the communication session into a buffer contained within the electronic device, before the received audio data is provided from the buffer, on a first-in first out basis, to output logic of the electronic device;

during the communication session, detecting a current interactivity mode, wherein the current interactivity mode is selected from the group consisting of a) non-interactive, indicating a low level of interactivity between participants in the communication session, and b) interactive, indicating a high level of interactivity between participants in the communication session;

storing an indication of the detected current interactivity mode;

detecting, during the communication session, that at least one packet of audio data transmitted to the electronic device from a remote device has been lost prior to being delivered to the electronic device;

in response to detecting that at least one packet of audio data transmitted to the electronic device from the remote device has been lost prior to being delivered to the electronic device, a) checking the stored indication of the current interactivity mode, and b) in response to the stored indication of the current interactivity mode indicating that the current interactivity mode is non-interactive, automatically increasing a total amount of received audio data concurrently stored in the buffer contained within the electronic device;

determining a round trip time between the electronic device and the remote device; and

wherein automatically increasing the total amount of received audio data concurrently stored in the buffer within the electronic device comprises increasing the total amount of received audio data concurrently stored in the buffer within the electronic device to equal an amount of audio data spanning a time interval having a duration that is greater than the round trip time between the electronic device and the remote device.

10. The method of claim 9 , wherein automatically increasing the total amount of received audio data concurrently stored in the buffer within the electronic device comprises time stretching audio data stored in the buffer by decreasing a play-out speed at which audio data is output from the buffer to the output logic to a rate that is slower than an input rate at which audio data is received and added to the buffer, until the total amount of received audio data concurrently stored in the buffer within the electronic device spans the time interval having a duration that is greater than the round trip time between the electronic device and the remote device.

11. The method of claim 10 , wherein the electronic device comprises a participant device; and

wherein detecting the current interactivity mode includes a) determining whether an audio input device of the electronic device is currently muted, and b) in response to determining that the audio input device of the electronic device is currently muted, detecting that the current interactivity mode is non-interactive.

12. The method of claim 10 , wherein the electronic device comprises a participant device; and

wherein detecting the current interactivity mode includes a) monitoring, during the communication session, an audio input device of the electronic device for the presence of speech received by the audio input device, and b) in response to detecting that no speech has been received by the audio input device of the electronic device during the communication session, detecting that the current interactivity mode is non-interactive.

13. The method of claim 10 , wherein the electronic device comprises a participant device; and

wherein detecting the current interactivity mode includes a) monitoring, during the communication session, an audio input device of the electronic device for the presence of an audio signal output from the audio input device, and b) in response to detecting that no audio signal has been output from the audio input device of the electronic device during the communication session, detecting that the current interactivity mode is non-interactive.

14. The method of claim 10 , wherein the electronic device comprises a participant device, and wherein the buffer stores audio data received by the electronic device during the communication session prior to providing the received audio data from the buffer on a first-in first-out basis to the output logic in the electronic device for playback of the received audio data by the output logic through at least one speaker of the participant device.

15. The method of claim 10 , wherein the electronic device comprises a bridge device, and wherein the buffer stores audio data received by the electronic device during the communication session prior to providing the received audio data from the buffer on a first-in first-out basis to the output logic in the electronic device for transmission of the audio data output from the buffer in packets from the bridge device to at least one participant device.

16. An electronic device, comprising:

processing circuitry; and

memory communicably coupled to the processing circuitry, the memory storing program code for handling packet loss during a communication session, wherein the program code, when executed by the processing circuitry, causes the processing circuitry to:

store audio data received by an electronic device from a remote device during the communication session into a buffer contained within the electronic device, before the received audio data is provided from the buffer, on a first-in first out basis, to output logic of the electronic device;

during the communication session, detect a current interactivity mode, wherein the current interactivity mode is selected from the group consisting of a) non-interactive, indicating a low level of interactivity between participants in the communication session, and b) interactive, indicating a high level of interactivity between participants in the communication session;

store an indication of the detected current interactivity mode;

detect, during the communication session, that at least one packet of audio data transmitted to the electronic device from a remote device has been lost prior to being delivered to the electronic device; and

in response to detecting that at least one packet of audio data transmitted to the electronic device from the remote device has been lost prior to being delivered to the electronic device, a) check the stored indication of the current interactivity mode, and b) in response to the stored indication of the current interactivity mode indicating that the current interactivity mode is non-interactive, automatically increase a total amount of received audio data concurrently stored in the buffer contained within the electronic device.

17. The electronic device of claim 16 , wherein the program code, when executed by the processing circuitry, further causes the processing circuitry to automatically increase the total amount of received audio data concurrently stored in the buffer within the electronic device by time stretching audio data stored in the buffer by decreasing a play-out speed at which audio data is output from the buffer to the output logic to a rate that is slower than an input rate at which audio data is received and added to the buffer, until the total amount of received audio data concurrently stored in the buffer within the electronic device spans the time interval having a duration that is greater than the round trip time between the electronic device and the remote device.

18. The electronic device of claim 16 , wherein the program code, when executed by the processing circuitry, further causes the processing circuitry to, also in response to both a) detecting that at least one packet of audio data transmitted to the electronic device from the remote device has been lost prior to being delivered to the electronic device, and b) detecting that the current interactivity mode is non-interactive, indicate to the remote device that the remote device is to begin transmitting redundant data to the electronic device in order to enable error correction by the electronic device.

19. The electronic device of claim 16 , wherein the program code, when executed by the processing circuitry, further causes the processing circuitry to:

detect that the current interactivity mode has changed from non-interactive to interactive; and

in response to detecting that the current interactivity mode has changed from non-interactive to interactive, automatically reduce the total amount of received audio data concurrently stored in the buffer contained within the audio device.

20. An electronic device, comprising:

processing circuitry; and

memory communicably coupled to the processing circuitry, the memory storing program code for handling packet loss during a communication session, wherein the program code, when executed by the processing circuitry, causes the processing circuitry to:

store audio data received by an electronic device from a remote device during the communication session into a buffer contained within the electronic device, before the received audio data is provided from the buffer, on a first-in first out basis, to output logic of the electronic device;

during the communication session, detect a current interactivity mode, wherein the current interactivity mode is selected from the group consisting of a) non-interactive, indicating a low level of interactivity between participants in the communication session, and b) interactive, indicating a high level of interactivity between participants in the communication session;

store an indication of the detected current interactivity mode;

detect, during the communication session, that at least one packet of audio data transmitted to the electronic device from a remote device has been lost prior to being delivered to the electronic device;

determine a round trip time between the electronic device and the remote device; and

in response to detecting that at least one packet of audio data transmitted to the electronic device from the remote device has been lost prior to being delivered to the electronic device, a) check the stored indication of the current interactivity mode, and b) in response to the stored indication of the current interactivity mode indicating that the current interactivity mode is non-interactive, automatically increase the total amount of received audio data concurrently stored in the buffer within the electronic device by increasing the total amount of received audio data concurrently stored in the buffer within the electronic device to equal an amount of audio data spanning a time interval having a duration that is greater than the round trip time between the electronic device and the remote device.

21. A computer program product including a non-transitory computer readable medium which stores program code that, when executed by processing circuitry, is operable to handle packet loss during a communication session, the program code comprising:

program code for storing audio data received by an electronic device from a remote device during the communication session into a buffer contained within the electronic device, before the received audio data is provided from the buffer, on a first-in first out basis, to output logic of the electronic device;

program code for, during the communication session, detecting a current interactivity mode, wherein the current interactivity mode is selected from the group consisting of a) non-interactive, indicating a low level of interactivity between participants in the communication session, and b) interactive, indicating a high level of interactivity between participants in the communication session;

program code for storing an indication of the detected current interactivity mode;

program code for detecting, during the communication session, that at least one packet of audio data transmitted to the electronic device from a remote device has been lost prior to being delivered to the electronic device; and

program code for, in response to detecting that at least one packet of audio data transmitted to the electronic device from the remote device has been lost prior to being delivered to the electronic device, a) checking the stored indication of the current interactivity mode, and b) in response to the stored indication of the current interactivity mode indicating that the current interactivity mode is non-interactive, automatically increasing a total amount of received audio data concurrently stored in the buffer contained within the electronic device.

Assignments (14)

Continuity (1)