Method and apparatus for acoustic area monitoring by exploiting ultra large scale arrays of microphones
Systems and methods are provided to create an acoustic map of a space containing multiple acoustic sources. Source localization and separation takes place by sampling an ultra large microphone array containing over 1020 microphones. The space is divided into a plurality of masks, wherein each masks represents a pass region and a complementary rejection region. Each mask is associated with a subset of microphones and beamforming filters that maximize a gain for signals coming from the pass region of the mask and minimizes the gain for signals from the complementary region according to an optimization criterion. The optimization criterion may be a minimization of a performance function for the beamforming filters. The performance function is preferably a convex function. A processor provides a scan applying the plurality of masks to locate a target source. Processor based systems to perform the optimization are also provided.
1. A method for creating an acoustic map of an environment having at least one acoustic source, comprising:
surrounding the environment with an array of microphones that contains 1000 or more microphones in the array;
a processor determining a plurality of disjoint spatial masks associated with the array of microphones covering the environment, each mask defining a different pass region for a signal and a plurality of complementary rejection regions, wherein the environment is monitored by the array of microphones;
the processor determining for each mask in the plurality of disjoint spatial masks a defined subset of microphones in the array of microphones and a beamforming filter for each of the microphones in the defined subset of microphones that maximizes a gain for the pass region and minimizes gain for the complementary rejection regions associated with each mask according to an optimization criterion that does not depend on the at least one acoustic source in the environment; and
the processor applying the plurality of disjoint spatial masks in a scanning action across the environment on signals generated by microphones in the array of microphones to detect the acoustic source and its location in the environment, wherein for each applied spatial mask only samples generated by the corresponding defined subset of microphones are processed by the processor.
2. The method of claim 1 , further comprising:
the processor characterizing one or more acoustic sources detected as a result of the scanning action into targets or interferences, based on their spectral and spatial characteristics, or prior knowledge or information.
3. The method of claim 2 , further comprising:
changing a first subset of microphones and beamforming filters for the first subset of microphones based on the one or more detected acoustic sources.
4. The method of claim 1 , wherein the optimization criterion includes minimizing an effect of an interfering source based on a performance of a filter related to the defined subset of microphones.
5. The method of claim 4 , wherein the performance of the filter is expressed as:
J
(
(
K
n
r
(
ω
)
)
n
∈
Ω
)
=
(
∑
n
∈
Ω
K
n
r
(
ω
)
2
)
(
∑
n
∈
Ω
H
n
,
r
(
ω
)
2
)
-
∑
n
∈
Ω
K
n
r
(
ω
)
H
n
,
r
(
ω
)
2
;
wherein J is an objective function that is minimized;
K n r (ω) defines a beamforming filter for a source r to a microphone n in the subset of microphones Ω in a frequency domain;
H n,r is a transfer function from a source r to microphone n in the frequency domain; and
ω defines a frequency.
6. The method of claim 1 , comprising repeating the steps of claim 1 to track an acoustical source.
7. The method of claim 6 , wherein a performance of the filter is expressed as an optimized convex function as follows:
D
(
Z
)
=
Z
T
RZ
+
μ
log
(
∑
l
=
0
,
l
≠
r
L
ⅇ
Z
T
Q
l
Z
)
+
λ
Z
1
,
wherein
Z is a vector in a frequency domain containing a real part of coefficients and an imaginary part of coefficients defining the filter;
Q l is a matrix defined by a real part and an imaginary part of a transfer function from a source l to a microphone in the frequency domain
R is a matrix defined by a real part and an imaginary part of a transfer function from a source r to a microphone in the frequency domain;
r indicates a target source;
T indicates a transposition;
e indicates the base of the natural logarithm;
μ and λ are cost factors; and
∥Z∥ 1 is an l 1 -norm of Z.
8. The method of claim 6 , wherein the convex function is expressed as:
F ( Z ) =τ+λ∥Z∥ 1 , wherein:
Z is a vector in a frequency domain containing a real part of coefficients and an imaginary part of coefficients defining the filter;
F(Z) is the convex function;
τ is a maximum processing gain from an interference source;
λ is a cost factor; and
∥Z∥ 1 is an l 1 -norm of Z.
9. The method of claim 6 , wherein the convex function is expressed as:
F
(
Z
1
,
Z
2
,
…
,
Z
P
)
=
∑
p
=
1
P
τ
p
+
λ
∑
k
=
1
N
max
1
≤
p
≤
P
Z
k
p
,
wherein Z p is a vector in a frequency domain containing a real part of coefficients and an imaginary part of coefficients defining the filter in the frequency domain for a frequency p;
F(Z 1 , Z 2 , . . . , Z F ) represents the convex function;
τ p is a maximum processing gain from interference sources at frequency p;
λ is a cost factor; and
Z k p represents a real and imaginary part of a coefficient for microphone k defining the filter for frequency p.
10. The method of claim 1 , wherein the plurality of disjoint spatial masks covering the environment includes at least one mask of a single pass region that is completely surrounded by rejection regions.
11. A system to create an acoustic map of an environment having at least one acoustic source, comprising:
an array of microphones containing a plurality of 1000 or more microphones that surround the environment;
a memory enabled to store data;
a processor enabled to execute instructions to perform the steps:
determining a plurality of disjoint spatial masks associated with the array of microphones covering the environment, each mask defining a different pass region for a signal and a plurality of complementary rejection regions, wherein the environment is monitored by the array of microphones;
determining for each mask in the plurality of disjoint spatial masks a defined subset of microphones in the plurality of microphones and a beamforming filter for each of the microphones in the subset of microphones that maximizes a gain for the pass region and minimizes gain for the complementary rejection regions associated with each mask according to an optimization criterion that does not depend on the at least one acoustic source in the environment; and
applying the plurality of disjoint spatial masks in a scanning action across the environment on signals generated by microphones in the array of microphones to detect the acoustic source and its location in the environment, wherein for each applied spatial mask only samples generated by the corresponding defined subset of microphones are processed by the processor.
12. The system of claim 11 , further comprising:
characterizing one or more acoustic sources detected as a result of the scanning action into a target or an interference, based on spectral and spatial characteristics.
13. The system of claim 12 , further comprising:
changing a first subset of microphones and beamforming filters for the first subset of microphones based on the one or more detected acoustic sources.
14. The system of claim 11 , wherein the optimization criterion includes minimizing an effect of an interfering source on a performance of a filter related to the defined subset of microphones.
15. The system of claim 14 , wherein the filter is a matched filter and the performance of the matched filter is expressed as:
J
(
(
K
n
r
(
ω
)
)
n
∈
Ω
)
=
(
∑
n
∈
Q
K
n
r
(
ω
)
2
)
(
∑
n
∈
Q
H
n
,
r
(
ω
)
2
)
-
∑
n
∈
Ω
K
n
r
(
ω
)
H
n
,
r
(
ω
)
2
;
wherein J is an objective function that is minimized;
K n r (ω) defines a beamforming filter for a source r to a microphone n in the subset of microphones Ω in a frequency domain;
H n,r (ω) is a transfer function from a source r to microphone n in the frequency domain; and
ω defines a frequency.
16. The system of claim 14 , wherein the wherein the performance of the filter is expressed as a convex function that is optimized.
17. The system of claim 16 , wherein the convex function is expressed as:
D
(
Z
)
=
Z
T
RZ
+
μ
log
(
∑
l
=
0
,
l
≠
r
L
ⅇ
Z
T
Q
l
Z
)
+
λ
Z
1
,
Z is a vector in a frequency domain containing a real part of coefficients and an imaginary part of coefficients defining the filter;
Q l is a matrix defined by a real part and an imaginary part of a transfer function from a source l to a microphone in the frequency domain;
R is a matrix defined by a real part and an imaginary part of a transfer function from a source r to a microphone in the frequency domain;
r indicates a target source;
T indicates a transposition;
e indicates the base of the natural logarithm;
μ and λ are cost factors; and
∥Z∥ 1 is an l 1 -norm of Z.
18. The system of claim 16 , wherein the convex function is expressed as:
F ( Z )=τ+λ∥ Z∥ 1 , wherein:
Z is a vector in a frequency domain containing a real part of coefficients and an imaginary part of coefficients defining the filter;
F(Z) is the convex function;
σ is a maximum processing gain from an interference source;
λ is a cost factor; and
∥Z∥ 1 is an l 1 -norm of Z.
19. The system of claim 16 , wherein the convex function is expressed as:
F
(
Z
1
,
Z
2
,
…
,
Z
P
)
=
∑
p
=
1
P
τ
p
+
λ
∑
k
=
1
N
max
1
≤
p
≤
P
Z
k
p
,
wherein
Z p is a vector in a frequency domain containing a real part of coefficients and an imaginary part of coefficients defining the filter in the frequency domain for a frequency p;
F(Z 1 , Z 2 , . . . , Z F ) represents the convex function;
τ p is a maximum processing gain from interference sources at frequency p;
λ is a cost factor; and
Z k p represents a real and imaginary part of a coefficient for microphone k defining the filter for frequency p.
20. The system of claim 11 , wherein the plurality of disjoint spatial masks covering the environment includes at least one mask of a single pass region that is completely surrounded by rejection regions.