IP Library Granted Patent US 11,620,976
Granted Patent B2
US 11,620,976 · App. 16/897,039 · Granted Apr 4, 2023

Systems, devices, and methods of acoustic echo cancellation based on display orientation

Inventors: Tyler Back (San Francisco, CA); Joshua Warren Bingham (Palo Alto, CA); Yuhuan Zeng (San Carlos, CA); Plamen Ivanov (Mountain View, CA); Christopher Evans (Menlo Park, CA); Jens Nilsson (Foster City, CA); Michael Asfaw (Foster City, CA)
Assignee: META PLATFORMS TECHNOLOGIES, LLC
G10K11/17854G10L21/0208H04M3/568H04R1/403H04R1/406
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,620,976
App. No.
16/897,039
Granted
Apr 4, 2023
Kind
B2
Abstract

An electronic device includes a microphone array to capture audio input data, a speaker array to render audio output data for playback; one or more sensors to detect an orientation of the microphone array, acoustic echo cancellation logic, and an interface. The acoustic echo cancellation logic applies acoustic echo cancellation to the audio input data to form echo-cancelled audio input data based on the orientation of the microphone array. The interface transmits the echo-cancelled audio input data over a communications channel as part of an audiovisual communication system.

Claims (39)

1. An electronic device comprising:

a display device rotatable with respect to a base;

a microphone array configured to capture audio input data, wherein the microphone array if fixedly attached to the display device;

a speaker array configured to render audio output data for playback, wherein the speaker array is housed in the base;

one or more sensors configured to detect data indicating an orientation of the display device with respect to the base;

acoustic echo cancellation logic configured to apply acoustic echo cancellation to the audio input data to form echo-cancelled audio input data based on the data indicating the orientation of the display device with respect to the base; and

an interface configured to transmit the echo-cancelled audio input data over a communications channel as part of an audiovisual communication system.

2. The electronic device of claim 1 , wherein the acoustic echo cancellation logic is further configured to determine a set of echo cancellation parameters based on the orientation of the microphone array, and wherein to apply the acoustic echo cancellation to the audio input data, the acoustic echo cancellation logic is configured to apply the acoustic echo cancellation based on the determined set of echo cancellation parameters.

3. The electronic device of claim 2 , wherein to apply the acoustic echo cancellation to the audio data, the acoustic echo cancellation logic is configured to:

configure an adaptive filter based on the determined set of echo cancellation parameters; and

apply, to the audio data, the adaptive filter configured based on the determined set of echo cancellation parameters.

4. The electronic device of claim 3 , wherein to configure the adaptive filter, the acoustic echo cancellation logic is configured to control an adaptation or a training of the adaptive filter based on the determined set of echo cancellation parameters.

5. The electronic device of claim 3 , wherein to determine the set of echo cancellation parameters, the acoustic echo cancellation logic is configured to select the set of echo cancellation parameters from a plurality of sets of echo cancellation parameters using a lookup table.

6. The electronic device of claim 3 , wherein to determine the set of echo cancellation parameters, the acoustic echo cancellation logic is configured to apply a finite element solution that determines the set of echo cancellation parameters.

7. The electronic device of claim 3 , wherein to determine the set of echo cancellation parameters, the acoustic echo cancellation logic is configured to apply an artificial intelligence model or a machine learning model trained with a mapping of rotation angles of the microphone array to respective sets of echo cancellation parameters to predict the set of echo cancellation parameters.

8. The electronic device of claim 3 , wherein to apply the adaptive filter to the audio data, the acoustic echo cancellation logic is configured to determine signal coherence among signals of the audio data using the adaptive filter.

9. The electronic device of claim 1 , wherein the acoustic echo cancellation logic is further configured to determine a relative positioning between the speaker array and the microphone array based on the data indicating the orientation of the display device detected by the one or more sensors, and wherein to apply the acoustic echo cancellation to the audio data, the acoustic echo cancellation logic is configured to: determine a set of echo cancellation parameters based on the relative positioning between the speaker array and the microphone array; and apply the acoustic echo cancellation based on the determined set of echo cancellation parameters.

10. The electronic device of claim 1 , wherein the one or more sensors includes one or more of an accelerometer, a position encoder, a gyroscope, a motion sensor, camera hardware, an imaging system, a radio frequency probe, an antenna, or a Hall probe.

11. The electronic device of claim 1 , wherein the one or more sensors is further configured to detect configured to detect an orientation of a physical component of the electronic device relative the microphone array, and wherein to apply the acoustic echo cancellation to the audio input data to form the echo-cancelled audio input data based on the orientation of the microphone array, the acoustic echo cancellation logic is configured to process the audio data to form the echo-cancelled audio input data based on the orientation of the physical component relative to the microphone array.

12. A method comprising:

capturing, by a microphone array fixedly attached to a display of an electronic device, audio input data;

rendering, by a speaker array housed in a base of the electronic device, audio output data for playback, wherein the display is rotatable with respect to the base;

detecting, by one or more sensors of the electronic device, data indicating an orientation of the display with respect to the base;

applying, by acoustic echo cancellation logic of the electronic device, based on the data indicating the orientation of the display with respect to the base, acoustic echo cancellation to the audio input data to form echo-cancelled audio input data; and

transmitting, via an interface of the electronic device, the echo-cancelled audio input data over a communications channel.

13. The method of claim 12 , further comprising determining, by the acoustic echo cancellation logic, a set of echo cancellation parameters based on the orientation of the microphone array, wherein applying the acoustic echo cancellation to the audio input data comprises applying, by the acoustic echo cancellation logic the acoustic echo cancellation based on the determined set of echo cancellation parameters.

14. The method of claim 13 , wherein applying the acoustic echo cancellation to the audio data comprises:

configuring, by the acoustic echo cancellation logic, an adaptive filter based on the determined set of echo cancellation parameters; and

applying, by the acoustic echo cancellation logic, to the audio data, the adaptive filter configured based on the determined set of echo cancellation parameters.

15. The method of claim 14 , wherein determining the set of echo cancellation parameters comprises selecting, by the acoustic echo cancellation logic, the set of echo cancellation parameters from a plurality of sets of echo cancellation parameters using a lookup table.

16. The method of claim 14 , wherein determining the set of echo cancellation parameters comprises applying, by the acoustic echo cancellation logic, a finite element solution that determines the set of echo cancellation parameters.

17. The method of claim 14 , wherein determining the set of echo cancellation parameters comprises applying, by the acoustic echo cancellation logic, artificial intelligence model or a machine learning model trained with a mapping of rotation angles of the microphone array to respective sets of echo cancellation parameters to predict the set of echo cancellation parameters.

18. The method of claim 14 , wherein applying the adaptive filter to the audio data comprises determining, by the acoustic echo cancellation logic, signal coherence among signals of the audio data using the adaptive filter.

19. A non-transitory computer-readable storage medium storing one or more programs configured for execution by one or more processors of an electronic device having a display, an interface, sensors, and a speaker array, the one or more programs including instructions, which when executed by the one or more processors, cause the electronic device to:

capture, using a microphone array fixed attached to the display of the electronic device, audio input data;

render, using the speaker array housed in a base of the electronic device, audio output data for playback, wherein the display is rotatable with respect to the base;

detect, using sensors of the electronic device, data indicating an orientation of the display with respect to the base

apply, using acoustic echo cancellation logic of the electronic device, based on the data indicating the orientation of the display with respect to the base, acoustic echo cancellation to the audio input data to form echo-cancelled audio input data; and

transmit, via an interface of the electronic device, the echo-cancelled audio input data over a communications channel as part of a conferencing session.

Assignments (3)
CHANGE OF NAME Recorded Jul 21, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060802/0799 →
CORRECTIVE ASSIGNMENT TO CORRECT THE UPDATE 3RD CONVEYOR'S NAME DUE TO TYPOGRAPHICAL ERROR IS YUHUAN ZENG PREVIOUSLY RECORDED AT REEL: 053242 FRAME: 0873. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Apr 18, 2022
From: BACK, TYLER; BINGHAM, JOSHUA WARREN; ZENG, YUHUAN; IVANOV, PLAMEN; EVANS, CHRISTOPHER; NILSSON, JENS; ASFAW, MICHAEL
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 059725/0433 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 17, 2020
From: BACK, TYLER; BINGHAM, JOSHUA WARREN; TSANG, EVA; IVANOV, PLAMEN; EVANS, CHRISTOPHER; NILSSON, JENS; ASFAW, MICHAEL
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 053242/0873 →
Continuity (1)
Related Publication 20210383786A1 · Dec 9, 2021