IP Library Granted Patent US 10,225,395
Granted Patent B2
US 10,225,395 · App. 14/963,374 · Granted Mar 5, 2019

Techniques to dynamically engage echo cancellation

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,225,395
App. No.
14/963,374
Granted
Mar 5, 2019
Kind
B2
Abstract

Techniques to dynamically engage echo cancellation are described. In one embodiment, an apparatus may comprise a streaming component operative to establish a audio connection between the first client device and a second client device via the network interface controller; and receive a far-end audio stream at the first client device from the second client device via the audio connection; an audio capture component operative to capture a near-end audio stream at the first client device; and an echo processing component operative to compare the near-end audio stream and the far-end audio stream to determine whether a far-end echo is present in the near-end audio stream; and use an echo-cancellation module at the first client device where the far-end echo is present in the near-end audio stream. Other embodiments are described and claimed.

Claims (70)

1. A computer-implemented method, comprising:

establishing an audio connection between a first client device and a second client device;

capturing a near-end audio stream at the first client device;

generating a data packet comprising a header and an audio frame of the near-end audio stream, the header comprising an indication that the audio frame exceeds a high-intensity energy threshold, and transmitting the data packet to the second client device;

receiving a corresponding data packet for a far-end audio stream at the first client device from the second client device via the audio connection, the corresponding data packet comprising a corresponding header;

comparing the near-end audio stream and the far-end audio stream to determine whether a far-end echo is present in the near-end audio stream, wherein the determining comprises:

comparing an audio energy level for the audio frame in the near-end audio stream to a high-intensity energy threshold,

determining that a corresponding audio frame in the far-end audio stream exceeds the high-intensity energy threshold by identifying an indication in the corresponding header that the far-end audio stream exceeds the high-intensity energy threshold, and

determining that the far-end echo is present in the near-end audio stream when the frame of the near-end audio stream and the corresponding frame of the far-end audio stream both exceed the high-intensity energy threshold; and

using an echo-cancellation module at the first client device where the far-end echo is present in the near-end audio stream.

2. The method of claim 1 , wherein comparing the near-end audio stream and the far-end audio stream to determine whether the far-end echo is present in the near-end audio stream comprises:

performing a high-intensity overlap presence comparison of the near-end audio stream and the far-end audio stream to determine a high-intensity overlap presence measure; and

comparing the high-intensity overlap presence measure to the high-intensity energy threshold.

3. The method of claim 2 , wherein performing the high-intensity overlap presence comparison of the near-end audio stream and the far-end audio stream to determine the high-intensity overlap presence measure comprises:

determining a near-end high-intensity presence indicator based on a comparison of the near-end audio stream and the high-intensity energy threshold;

determining a far-end high-intensity presence indicator based on a comparison of the far-end audio stream and the high-intensity energy threshold; and

comparing the near-end high-intensity presence indicator and the far-end high-intensity presence indicator.

4. The method of claim 2 , the high-intensity overlap presence measure comprising a weighted moving average over a span of the audio connection.

5. The method of claim 1 , wherein comparing the near-end audio stream and the far-end audio stream to determine whether the far-end echo is present in the near-end audio stream comprises:

performing a voice overlap presence comparison of the near-end audio stream and the far-end audio stream to determine a voice overlap presence measure; and

comparing the voice overlap presence measure to a predefined voice overlap presence threshold.

6. The method of claim 5 , wherein performing the voice overlap presence comparison of the near-end audio stream and the far-end audio stream to determine the voice overlap presence measure comprises:

determining a near-end voice presence indicator based on the near-end audio stream;

determining a far-end voice presence indicator based on the far-end audio stream; and

comparing the near-end voice presence indicator and the far-end voice presence indicator.

7. The method of claim 5 , wherein performing the voice overlap presence comparison of the near-end audio stream and the far-end audio stream to determine the voice overlap presence measure comprises:

determining a near-end voice presence indicator based on the near-end audio stream;

receiving a far-end voice presence indicator from the second client device via the audio connection; and

comparing the near-end voice presence indicator and the far-end voice presence indicator.

8. The method of claim 5 , the voice overlap presence measure comprising a weighted moving average over a span of the audio connection.

9. The method of claim 1 , wherein far-end echo is determined to be present in the near-end audio stream at a first time, and further comprising:

activating the echo-cancellation module at the first client device at the first time while the far-end echo is present in the near-end audio stream;

determining that far-end echo is not present in the near-end audio stream at a second time after the first time; and

deactivating the echo-cancellation module at the first client device at the second time while the far-end echo is not present in the near-end audio stream.

10. An apparatus, comprising:

a processor circuit on a first client device;

a network interface controller on the device;

a streaming component operative on the processor circuit to establish an audio connection between the first client device and a second client device via the network interface controller; and receive a far-end audio stream at the first client device from the second client device via the audio connection;

an audio capture component operative on the processor circuit to capture a near-end audio stream at the first client device, wherein the network interface controller is configured to generate a data packet comprising a header and an audio frame of the near-end audio stream, the header comprising an indication that the audio frame exceeds a high-intensity energy threshold, and transmitting the data packet to the second client device, and the far-end audio stream comprises a corresponding data packet comprising a corresponding header; and

an echo processing component operative on the processor circuit to compare the near-end audio stream and the far-end audio stream to determine whether a far-end echo is present in the near-end audio stream, wherein the determining comprises:

comparing an audio energy level for an audio frame in the near-end audio stream to a high-intensity energy threshold,

comparing an audio energy level for a corresponding audio frame in the far-end audio stream to the high-intensity energy threshold by identifying an indication in the corresponding header that the far-end audio stream exceeds the high-intensity energy threshold, and

determining that the far-end echo is present in the near-end audio stream when the frame of the near-end audio stream and the corresponding frame of the far-end audio stream both exceed the high-intensity energy threshold; and

use an echo-cancellation module at the first client device where the far-end echo is present in the near-end audio stream.

11. The apparatus of claim 10 , wherein the echo processing component is further operative to perform a high-intensity overlap presence comparison of the near-end audio stream and the far-end audio stream to determine a high-intensity overlap presence measure; and compare the high-intensity overlap presence measure to the high-intensity energy threshold.

12. The apparatus of claim 11 , the high-intensity overlap presence measure comprising a weighted moving average over a span of the audio connection.

13. The apparatus of claim 10 , wherein comparing the near-end audio stream and the far-end audio stream to determine whether the far-end echo is present in the near-end audio stream comprises:

the echo processing component operative to perform a voice overlap presence comparison of the near-end audio stream and the far-end audio stream to determine a voice overlap presence measure; and compare the voice overlap presence measure to a predefined voice overlap presence threshold.

14. The apparatus of claim 13 , the voice overlap presence measure comprising a weighted moving average over a span of the audio connection.

15. The apparatus of claim 10 , wherein:

far-end echo is determined to be present in the near-end audio stream at a first time, the echo processing component operative to activate the echo-cancellation module at the first client device at the first time while the far-end echo is present in the near-end audio stream; and

far-end echo is determined not to be present in the near end audio stream at a second time after the first time, and the echo processing component is operative to deactivate the echo-cancellation module at the first client device at the second time while the far-end echo is not present in the near-end audio stream.

16. At least one non-transitory computer-readable storage medium comprising instructions that, when executed, cause a system to:

establish an audio connection between a first client device and a second client device;

capture a near-end audio stream at the first client device;

generate a data packet comprising a header and an audio frame of the near-end audio stream, the header comprising an indication that the audio frame exceeds a high-intensity energy threshold, and transmitting the data packet to the second client device;

receive a corresponding data packet for a far-end audio stream at the first client device from the second client device via the audio connection, the corresponding data packet comprising a corresponding header;

compare the near-end audio stream and the far-end audio stream to determine whether a far-end echo is present in the near-end audio stream, wherein the determining comprises:

comparing an audio energy level for the audio frame in the near-end audio stream to a high-intensity energy threshold,

determining that a corresponding audio frame in the far-end audio stream exceeds the high-intensity energy threshold by identifying an indication in the corresponding header that the far-end audio stream exceeds the high-intensity energy threshold, and

determining that the far-end echo is present in the near-end audio stream when the frame of the near-end audio stream and the corresponding frame of the far-end audio stream both exceed the high-intensity energy threshold; and

use an echo-cancellation module at the first client device where the far-end echo is present in the near-end audio stream.

17. The computer-readable storage medium of claim 16 , wherein comparing the near-end audio stream and the far-end audio stream to determine whether the far-end echo is present in the near-end audio stream comprises further instructions that, when executed, cause a system to:

perform a high-intensity overlap presence comparison of the near-end audio stream and the far-end audio stream to determine a high-intensity overlap presence measure; and

compare the high-intensity overlap presence measure to the high-intensity energy threshold.

18. The computer-readable storage medium of claim 17 , the high-intensity overlap presence measure comprising a weighted moving average over a span of the audio connection.

19. The computer-readable storage medium of claim 16 , wherein comparing the near-end audio stream and the far-end audio stream to determine whether the far-end echo is present in the near-end audio stream comprises further instructions that, when executed, cause a system to:

perform a voice overlap presence comparison of the near-end audio stream and the far-end audio stream to determine a voice overlap presence measure; and

compare the voice overlap presence measure to a predefined voice overlap presence threshold.

20. The computer-readable storage medium of claim 19 , the voice overlap presence measure comprising a weighted moving average over a span of the audio connection.

Assignments (2)
CHANGE OF NAME Recorded Jun 22, 2021
From: WHATSAPP INC.
To: WHATSAPP LLC
Reel/Frame 056646/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 23, 2017
From: SINGH, MANPREET; WANG, YUANYUAN
To: WHATSAPP INC.
Reel/Frame 041695/0388 →