IP Library › Granted Patent US 10,986,444
Granted Patent B2
US 10,986,444 · App. 16/798,706 · Granted Apr 20, 2021

Modeling room acoustics using acoustic waves

Inventors: Mohamed Mansour (Cupertino, CA); Guangdong Pan (Quincy, MA)
Assignee: Amazon Technologies, Inc.
H04R3/005H04R1/406
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,986,444
App. No.
16/798,706
Granted
Apr 20, 2021
Kind
B2
Abstract

Techniques for simulating a microphone array and generating synthetic audio data to analyze the microphone array geometry. This reduces the development cost of new microphone arrays by enabling an evaluation of performance metrics (False Rejection Rate (FRR), Word Error Rate (WER), etc.) without building device hardware or collecting data. To generate the synthetic audio data, the system performs acoustic modeling to determine a room impulse response associated with a prototype device (e.g., potential microphone array) in a room. The acoustic modeling is based on two parameters—a device response (information about acoustics and geometry of the prototype device) and a room response (information about acoustics and geometry of the room). The device response can be simulated based on the microphone array geometry, and the room response can be determined using a specialized microphone and a plane wave decomposition algorithm.

Claims (99)

1. A computer-implemented method comprising:

receiving first audio data including a first representation of speech;

determining first estimated impulse response data corresponding to an estimate of a first microphone array positioned at a first location;

generating, using the first audio data and the first estimated impulse response data, a first portion of first output audio data, the first output audio data including a second representation of the speech as though captured by the first microphone array positioned at the first location;

receiving second audio data representing acoustic noise;

generating, using the second audio data and the first estimated impulse response data, a second portion of the first output audio data; and

generating the first output audio data by combining the first portion of the first output audio data and the second portion of the first output audio data.

2. The computer-implemented method of claim 1 , further comprising:

sending third audio data to a loudspeaker that is at a second location in a room;

generating fourth audio data using a second microphone array at the first location in the room;

determining first acoustic characteristics data corresponding to the first location, wherein the determining is based on the fourth audio data and second acoustic characteristics data representing a first frequency response associated with the second microphone array; and

receiving third acoustic characteristics data representing a second frequency response associated with the first microphone array, the first microphone array not present in the room,

wherein determining the first estimated impulse response data further comprises:

generating the first estimated impulse response data based on the third audio data, the first acoustic characteristics data, and the third acoustic characteristics data.

3. The computer-implemented method of claim 2 , wherein determining the first acoustic characteristics data further comprises:

receiving the second acoustic characteristics data corresponding to the second microphone array; and

determining the first acoustic characteristics data by performing plane wave decomposition on the fourth audio data using the second acoustic characteristics data.

4. The computer-implemented method of claim 1 , further comprising:

receiving third audio data corresponding to audio output by a loudspeaker;

receiving first acoustic characteristics data corresponding to the first location in a room;

receiving second acoustic characteristics data representing a frequency response associated with the first microphone array, the first microphone array not present in the room;

generating, using the first acoustic characteristics data and the second acoustic characteristics data, fourth audio data corresponding to a simulation of the audio output being captured by the first microphone array at the first location; and

determining cross-spectrum analysis data corresponding to a cross-spectrum analysis between the third audio data and the fourth audio data,

wherein determining the first estimated impulse response data further comprises:

determining, using the cross-spectrum analysis data, the first estimated impulse response data.

5. The computer-implemented method of claim 1 , further comprising:

determining second estimated impulse response data corresponding to an estimate of the first microphone array positioned at a second location;

generating, using the first audio data and the second estimated impulse response data, a first portion of second output audio data, the second output audio data including a third representation of the speech as though captured by the first microphone array positioned at the second location;

generating, using the second audio data and the second estimated impulse response data, a second portion of the second output audio data; and

generating the second output audio data by combining the first portion of the second output audio data and the second portion of the second output audio data.

6. The computer-implemented method of claim 1 , further comprising:

determining second estimated impulse response data corresponding to an estimate of a second microphone array positioned at the first location;

generating, using the first audio data and the second estimated impulse response data, a first portion of second output audio data, the second output audio data including a third representation of the speech as though captured by the second microphone array positioned at the first location;

generating, using the second audio data and the second estimated impulse response data, a second portion of the second output audio data; and

generating the second output audio data by combining the first portion of the second output audio data and the second portion of the second output audio data.

7. The computer-implemented method of claim 1 , further comprising:

receiving first text data representing text corresponding to the first representation of the speech;

performing speech processing on the first output audio data to determine second text data; and

determining, using the first text data and the second text data, a performance parameter associated with the first microphone array.

8. The computer-implemented method of claim 1 , further comprising:

receiving first text data representing text corresponding to the first representation of the speech;

processing the first output audio data using configuration data to generate second output audio data;

performing speech processing on the second output audio data to determine second text data; and

determining, using the first text data and the second text data, a performance parameter associated with the configuration data.

9. The computer-implemented method of claim 1 , further comprising:

generating a digital model for a device that includes the first microphone array; and

performing acoustic modeling to determine first acoustic characteristics data associated with the first microphone array, the first acoustic characteristics data representing a plurality of vectors, a first vector of the plurality of vectors corresponding to a first acoustic wave of a plurality of acoustic waves,

wherein determining the first estimated impulse response data further comprises:

determining the first estimated impulse response data using the first acoustic characteristics data.

10. The computer-implemented method of claim 9 , wherein the first acoustic characteristics data represents at least a first anechoic response of the first microphone array to an acoustic plane wave and a second anechoic response of the first microphone array to a spherical acoustic wave.

11. A system comprising:

at least one processor; and

memory including instructions operable to be executed by the at least one processor to cause the system to:

receive first audio data including a first representation of speech;

determine first estimated impulse response data corresponding to an estimate of a first microphone array positioned at a first location;

generate, using the first audio data and the first estimated impulse response data, a first portion of first output audio data, the first output audio data including a second representation of the speech as though captured by the first microphone array positioned at the first location;

receive second audio data representing acoustic noise;

generate, using the second audio data and the first estimated impulse response data, a second portion of the first output audio data; and

generate the first output audio data by combining the first portion of the first output audio data and the second portion of the first output audio data.

12. The system of claim 11 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

send third audio data to a loudspeaker that is at a second location in a room;

generate fourth audio data using a second microphone array at the first location in the room;

determine first acoustic characteristics data corresponding to the first location, wherein the determining is based on the fourth audio data and second acoustic characteristics data representing a first frequency response associated with the second microphone array;

receive third acoustic characteristics data representing a second frequency response associated with the first microphone array, the first microphone array not present in the room; and

generate the first estimated impulse response data based on the third audio data, the first acoustic characteristics data, and the third acoustic characteristics data.

13. The system of claim 12 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

receive the second acoustic characteristics data corresponding to the second microphone array; and

determine the first acoustic characteristics data by performing plane wave decomposition on the fourth audio data using the second acoustic characteristics data.

14. The system of claim 11 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

receive third audio data corresponding to audio output by a loudspeaker;

receive first acoustic characteristics data corresponding to the first location in a room;

receive second acoustic characteristics data representing a frequency response associated with the first microphone array, the first microphone array not present in the room;

generate, using the first acoustic characteristics data and the second acoustic characteristics data, fourth audio data corresponding to a simulation of the audio output being captured by the first microphone array at the first location;

determine cross-spectrum analysis data corresponding to a cross-spectrum analysis between the third audio data and the fourth audio data; and

determine, using the cross-spectrum analysis data, the first estimated impulse response data.

15. The system of claim 11 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

determine second estimated impulse response data corresponding to an estimate of the first microphone array positioned at a second location;

generate, using the first audio data and the second estimated impulse response data, a first portion of second output audio data, the second output audio data including a third representation of the speech as though captured by the first microphone array positioned at the second location;

generate, using the second audio data and the second estimated impulse response data, a second portion of the second output audio data; and

generate the second output audio data by combining the first portion of the second output audio data and the second portion of the second output audio data.

16. The system of claim 11 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

determine second estimated impulse response data corresponding to an estimate of a second microphone array positioned at the first location;

generate, using the first audio data and the second estimated impulse response data, a first portion of second output audio data, the second output audio data including a third representation of the speech as though captured by the second microphone array positioned at the first location;

generate, using the second audio data and the second estimated impulse response data, a second portion of the second output audio data; and

generate the second output audio data by combining the first portion of the second output audio data and the second portion of the second output audio data.

17. The system of claim 11 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

receive first text data representing text corresponding to the first representation of the speech;

perform speech processing on the first output audio data to determine second text data; and

determine, using the first text data and the second text data, a performance parameter associated with the first microphone array.

18. The system of claim 11 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

receive first text data representing text corresponding to the first representation of the speech;

process the first output audio data using configuration data to generate second output audio data;

perform speech processing on the second output audio data to determine second text data; and

determine, using the first text data and the second text data, a performance parameter associated with the configuration data.

19. The system of claim 11 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

generate a digital model for a device that includes the first microphone array;

perform acoustic modeling to determine first acoustic characteristics data associated with the first microphone array, the first acoustic characteristics data representing a plurality of vectors, a first vector of the plurality of vectors corresponding to a first acoustic wave of a plurality of acoustic waves; and

determine the first estimated impulse response data using the first acoustic characteristics data.

20. The system of claim 19 , wherein the first acoustic characteristics data represents at least a first anechoic response of the first microphone array to an acoustic plane wave and a second anechoic response of the first microphone array to a spherical acoustic wave.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 24, 2020
From: MANSOUR, MOHAMED; PAN, GUANGDONG
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 051905/0852 →
Continuity (2)
Continuation 16216599 · Dec 11, 2018
Related Publication 20200196053A1 · Jun 18, 2020
Cited By (2)
US 12,198,715 US 12,242,972