IP Library Granted Patent US 11,095,769
Granted Patent B1
US 11,095,769 · App. 16/255,133 · Granted Aug 17, 2021

Techniques to dynamically engage echo cancellation

Inventors: Manpreet Singh (Los Altos Hills, CA); YuanYuan Wang (Sunnyvale, CA)
Assignee: WHATSAPP LLC.
H04M3/002G10L25/84H04L69/16H04M9/082G10L2021/02082H04L2012/6491
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,095,769
App. No.
16/255,133
Granted
Aug 17, 2021
Kind
B1
Abstract

Techniques to dynamically engage echo cancellation are described. In one embodiment, an apparatus may comprise a streaming component operative to establish a audio connection between the first client device and a second client device via the network interface controller; and receive a far-end audio stream at the first client device from the second client device via the audio connection; an audio capture component operative to capture a near-end audio stream at the first client device; and an echo processing component operative to compare the near-end audio stream and the far-end audio stream to determine whether a far-end echo is present in the near-end audio stream; and use an echo-cancellation module at the first client device where the far-end echo is present in the near-end audio stream. Other embodiments are described and claimed.

Claims (66)

1. A computer-implemented method, comprising:

receiving a near-end audio stream comprising a plurality of data packets containing near-end audio frames;

receiving a far-end audio stream comprising a plurality of data packets containing far-end audio frames;

determining if a far-end echo is present in the near-end audio stream based on whether a high-intensity signal that exceeds a pre-defined energy-level threshold exists in a first one of the near-end audio frames and exists in a corresponding one of the far-end audio frames;

when determining that the far-end echo is present in the near-end audio stream and further determining that an echo cancellation function is currently not engaged, engaging the echo cancellation function; and

when determining that the far-end echo is not present in the near-end audio stream and further determining that an echo cancellation function is currently engaged, disengaging the echo cancellation function.

2. The method of claim 1 , wherein determining if the far-end echo is present in the near-end audio stream further comprises:

determining, for each corresponding audio frame in the near-end audio stream and the far-end audio stream, that the high-intensity signal exceeding the pre-defined energy-level threshold exists in both the near-end audio frame and the far-end audio frame.

3. The method of claim 2 , further comprising:

performing a high-intensity overlap presence comparison of the high-intensity signal of the first one of the near-end audio frames and the high-intensity signal of the corresponding one of the far-end audio frames to determine a high-intensity overlap presence measure; and

determining that the high-intensity overlap presence measure exceeds a predefined high-intensity overlap presence threshold.

4. The method of claim 3 , wherein performing the high-intensity overlap presence comparison comprises:

determining a near-end high-intensity presence indicator based on a comparison of the high-intensity signal of the first one of the near-end audio frames and a high-intensity presence threshold;

determining a far-end high-intensity presence indicator based on a comparison of the high-intensity signal of the corresponding one of the far-end audio frames and the high-intensity presence threshold; and

comparing the near-end high-intensity presence indicator and the far-end high-intensity presence indicator.

5. The method of claim 2 , further comprising:

determining a weighted moving average of a count of corresponding audio frames in the near-end audio stream and the far-end audio stream that exceed the energy-level threshold; and

engaging the echo cancellation function when the weighted moving average exceeds a pre-defined moving average threshold.

6. The method of claim 5 , wherein the echo cancellation function is engaged after a delay comprising a pre-defined number of audio frames.

7. The method of claim 1 , wherein determining if the far-end echo is present in the near-end audio stream further comprises:

determining that voice content is present in corresponding near-end audio frames and far-end audio frames.

8. The method of claim 7 , further comprising:

determining a weighted moving average of a count of corresponding audio frames in the near-end audio stream and the far-end audio stream having voice content present; and

engaging the echo cancellation function when the weighted moving average exceeds a pre-defined voice presence threshold.

9. The method of claim 8 , wherein the echo cancellation function is engaged after a delay comprising a pre-defined number of audio frames.

10. An apparatus, comprising:

a processor;

memory, operatively coupled to the processor, the memory to store software for execution by the processor, the software comprising an echo detection routine operative to:

receive a near-end audio stream comprising a plurality of data packets containing near-end audio frames;

receive a far-end audio stream comprising a plurality of data packets containing far-end audio frames;

determine if a far-end echo is present in the near-end audio stream based on whether a high-intensity signal that exceeds a pre-defined energy-level threshold exists in a first one of the near-end audio frames and exists in a corresponding one of the far-end audio frames;

when determining that the far-end echo is present in the near-end audio stream and further determining that an echo cancellation module is currently not activated, activating an echo cancellation module; and

when determining that the far-end echo is not present in the near-end audio stream and further determining that the echo cancellation module is currently activated, deactivating the echo cancellation module.

11. The apparatus of claim 10 wherein the echo detection routine is further operative to:

determine if the far-end echo is present in the near-end audio stream by further determining, for each corresponding audio frame in the near-end audio stream and the far-end audio stream, that the high-intensity signal exceeding the pre-defined energy-level threshold exists in both the near-end audio frame and the far-end audio frame.

12. The apparatus of claim 11 , wherein the echo detection routine is further operative to:

determine a weighted moving average of a count of corresponding audio frames in the near-end audio stream and the far-end audio stream that exceed the energy-level threshold; and

activate the echo cancellation module when the weighted moving average exceeds a pre-defined moving average threshold;

wherein the echo cancellation module is activated after a pre-defined delay comprising a defined number of audio frames.

13. The apparatus of claim 10 , wherein the echo detection routine is further operative to:

determine if the far-end echo is present in the near-end audio stream by further determining that voice content is present in corresponding near-end audio frames and far-end audio frames.

14. The apparatus of claim 10 , wherein the echo detection routine is further operative to:

determine a weighted moving average of a count of corresponding audio frames in the near-end audio stream and the far-end audio stream having voice content present; and

activate the echo cancellation module when the weighted moving average exceeds a pre-defined voice presence threshold;

wherein the echo cancellation module is activated after predefined delay comprising a defined number of audio frames.

15. A non-transitory computer-readable storage medium comprising instructions that, when executed, cause a system to:

receive a near-end audio stream comprising a plurality of data packets containing near-end audio frames;

receive a far-end audio stream comprising a plurality of data packets containing far-end audio frames;

determine if a far-end echo is present in the near-end audio stream based on whether a high-intensity signal that exceeds a pre-defined energy-level threshold exists in a first one of the near-end audio frames and exists in a corresponding one of the far-end audio frames;

when determining that the far-end echo is present in the near-end audio stream and further determining that an echo cancellation function is currently not engaged, engaging the echo cancellation function; and

when determining that the far-end echo is not present in the near-end audio stream and further determining that an echo cancellation function is currently engaged, disengaging the echo cancellation function.

16. The computer-readable storage medium of claim 15 , comprising further instructions that cause the system to:

determine that the far-end echo is present in the near-end audio stream by further determining, for each corresponding audio frame in the near-end audio stream and the far-end audio stream, that the high-intensity signal exceeding the pre-defined energy-level threshold exists in both the near-end audio frame and the far-end audio frame.

17. The computer-readable storage medium of claim 16 , comprising further instructions that cause the system to:

perform a high-intensity overlap presence comparison of the high-intensity signal of the first one of the near-end audio frames and the high-intensity signal of the corresponding one of the far-end audio frames to determine a high-intensity overlap presence measure; and

determine that the high-intensity overlap presence measure exceeds a predefined high-intensity overlap presence threshold.

18. The computer-readable storage medium of claim 16 , comprising further instructions that cause the system to:

determine a weighted moving average of a count of corresponding audio frames in the near-end audio stream and the far-end audio stream that exceed the energy-level threshold; and

engage the echo cancellation function when the weighted moving average exceeds a pre-defined moving average threshold;

wherein the echo cancellation function is engaged after a delay comprising a pre-defined number of audio frames.

19. The computer-readable storage medium of claim 15 , comprising further instructions that cause the system to:

determining if the far-end echo is present in the near-end audio stream by further determining that voice content is present in corresponding near-end audio frames and far-end audio frames.

20. The computer-readable storage medium of claim 19 , comprising further instructions that cause the system to:

determine a weighted moving average of a count of corresponding audio frames in the near-end audio stream in the far-end audio stream having voice content present; and

engage the echo cancellation function when the weighted moving average exceeds a pre-defined voice presence threshold;

wherein the echo cancellation function is engaged after delay comprising a pre-defined number of audio frames.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 16, 2022
From: SINGH, MANPREET; WANG, YUANYUAN
To: WHATSAPP INC.
Reel/Frame 061801/0332 →
CHANGE OF NAME Recorded Jun 22, 2021
From: WHATSAPP INC.
To: WHATSAPP LLC
Reel/Frame 056646/0001 →
Continuity (1)
Continuation 14963374 · Dec 9, 2015