IP Library Granted Patent US 11,340,861
Granted Patent B2
US 11,340,861 · App. 16/897,018 · Granted May 24, 2022

Systems, devices, and methods of manipulating audio data based on microphone orientation

Inventors: Tyler Back (San Francisco, CA); Joshua Warren Bingham (Palo Alto, CA); Yuhuan Zeng (San Carlos, CA); Plamen Ivanov (Mountain View, CA); Christopher Evans (Menlo Park, CA); Jens Nilsson (Foster City, CA); Michael Asfaw (Foster City, CA)
Assignee: FACEBOOK TECHNOLOGIES, LLC
G06F3/162H04R1/406H04N7/142H04N7/147H04N7/15
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,340,861
App. No.
16/897,018
Granted
May 24, 2022
Kind
B2
Abstract

An electronic device includes a microphone array configured to capture audio data, one or more sensors configured to detect an orientation of the microphone array, and digital signal processing (DSP) logic, and an interface. The DSP logic processes, based on the orientation of the microphone array detected by the one or more sensors, the audio data captured by the microphone array to form audio input data. The interface configured to transmit the audio input data over a communications channel to be output by another electronic device.

Claims (43)

1. An electronic device comprising:

a display device repositionable relative to a base;

a microphone array configured to capture audio data, the microphone array fixedly attached to the display device;

one or more audio output devices housed in the base;

one or more sensors configured to detect an orientation of the display device relative to the base;

digital signal processing (DSP) logic configured to process, based on the orientation of the display device relative to the base detected by the one or more sensors, the audio data captured by the microphone array to form audio input data;

an interface configured to transmit the audio input data over a communications channel to the one or more audio output devices; and

driver logic that drives the one or more audio output devices based on the orientation of the display device relative to the base.

2. The electronic device of claim 1 , wherein to process the audio data to form the audio input data, the DSP logic is configured to drop signals received from one or more individual microphones of the microphone array based on the orientation of the microphone array.

3. The electronic device of claim 1 , wherein to process the audio data to form the audio input data, the DSP logic is configured to constrain a search space to signals received from one or more individual microphones of the microphone array based on the orientation of the microphone array.

4. The electronic device of claim 1 , wherein to process the audio data to form the audio input data, the DSP logic is configured to process the audio data using a set of audio capture parameters corresponding to the orientation of the microphone array.

5. The electronic device of claim 4 , wherein to determine the set of audio capture parameters corresponding to the orientation of the microphone array, the DSP logic is configured to select the set of audio capture parameters corresponding to the orientation of the microphone array from a plurality of sets of audio capture parameters using a lookup table.

6. The electronic device of claim 4 , wherein to determine the set of audio capture parameters corresponding to the orientation of the microphone array, the DSP logic is configured to apply a finite element solution that determines the set of audio capture parameters corresponding to the orientation of the microphone array.

7. The electronic device of claim 4 , wherein to determine the set of audio capture parameters corresponding to the orientation of the microphone array, the DSP logic is configured to apply an artificial intelligence model or a machine learning model trained with a mapping of rotation angles of the microphone array to respective sets of audio capture parameter sets to predict the set of audio capture parameters corresponding to the orientation of the microphone array.

8. The electronic device of claim 1 , wherein:

the DSP logic is configured to determine horizontal coordinate angles of the microphone array based on the orientation of the display device, and

to process the audio data to form the audio input data, the DSP logic is configured to process the audio data using the horizontal coordinate angles of the microphone array.

9. The electronic device of claim 1 , wherein to process the audio data to form the audio input data, the DSP logic is configured to:

generate a virtual directional microphone associated with one or more individual microphones of the microphone array based on the orientation of the display device array; and

form the audio input data using signals received from the one or more individual microphones that correspond to the virtual directional microphone.

10. The electronic device of claim 1 , wherein the one or more sensors include one or more of an accelerometer, a position encoder, a gyroscope, or a motion sensor.

11. A method comprising:

capturing, by a microphone array, audio data;

detecting, by one or more sensors, an orientation of a display device relative to a base, wherein the microphone array is fixedly attached to the display device, and the display device is repositionable relative to the base;

processing, by digital signal processing (DSP) logic, based on the orientation of the display device relative to the base detected by the one or more sensors, the audio data captured by the microphone array to form audio input data; and

transmitting, by an input/output interface, the audio input data over a communications channel to a speaker array housed in the base; and

driving the speaker array based on the orientation of the display device relative to the base.

12. The method of claim 11 , wherein processing the audio data to form the audio input data comprises dropping, by the DSP logic, signals received from one or more individual microphones of the microphone array based on the orientation of the microphone array.

13. The method of claim 11 , wherein processing the audio data to form the audio input data comprises constraining, by the DSP logic, a search space to signals received from one or more individual microphones of the microphone array based on the orientation of the microphone array.

14. The method of claim 11 , wherein processing the audio data to form the audio input data comprises processing, by the DSP logic, the audio data using a set of audio capture parameters corresponding to the orientation of the display device.

15. The method of claim 14 , wherein determining the set of audio capture parameters corresponding to the orientation of the microphone array comprises applying, by the driver logic, an artificial intelligence model or a machine learning model trained with a mapping of rotation angles of the microphone array to respective sets of audio capture parameter sets to predict the set of audio capture parameters corresponding to the orientation of the device.

16. The method of claim 14 , wherein determining the set of audio capture parameters corresponding to the orientation of the microphone array comprises selecting, by the driver logic, the set of audio capture parameters corresponding to the orientation of the display device from a plurality of sets of audio capture parameters using a lookup table.

17. The method of claim 14 , wherein determining the set of audio capture parameters corresponding to the orientation of the display device comprises applying, by the driver logic, a finite element solution that determines the set of audio capture parameters corresponding to the orientation of the display device.

18. The method of claim 11 , further comprising:

determining, by the DSP logic, horizontal coordinate angles of the microphone array based on the orientation of the display device,

wherein processing the audio data to form the audio input data comprises processing, by the DSP logic, the audio data using the horizontal coordinate angles of the microphone array.

19. A non-transitory computer-readable storage medium storing one or more programs configured for execution by one or more processors of an electronic device having a microphone array, an interface, one or more sensors, and a speaker array, the one or more programs including instructions, which when executed by the one or more processors, cause the electronic device to:

capture, using the microphone array, audio data;

detect, using one or more sensors, an orientation of a display device relative to a base, wherein the microphone array is fixedly attached to the display device, and the display device is repositionable relative to the base;

process, based on the orientation of the display device relative to the base detected by the one or more sensors, the audio data captured by the microphone array to form audio input data; and

transmit, via the interface, the audio input data over a communications channel to a speaker array housed in the base; and

drive the speaker array based on the orientation of the display device relative to the base.

20. The electronic device of claim 1 , wherein the display device is repositionable by a user or a telephonic system.

Assignments (3)
CHANGE OF NAME Recorded Jul 21, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060802/0799 →
CORRECTIVE ASSIGNMENT TO CORRECT THE THIRD INVENTOR'S NAME AND EXECUTION PREVIOUSLY RECORDED AT REEL: 053233 FRAME: 0236. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT . Recorded Apr 15, 2022
From: BACK, TYLER; BINGHAM, JOSHUA WARREN; ZENG, YUHUAN; IVANOV, PLAMEN; EVANS, CHRISTOPHER; NILSSON, JENS; ASFAW, MICHAEL
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 059721/0605 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 16, 2020
From: BACK, TYLER; BINGHAM, JOSHUA WARREN; TSANG, EVA; IVANOV, PLAMEN; EVANS, CHRISTOPHER; NILSSON, JENS; ASFAW, MICHAEL
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 053233/0236 →
Continuity (1)
Related Publication 20210382678A1 · Dec 9, 2021