UTILIZING LEVEL DIFFERENCES FOR SPEECH ENHANCEMENT
Systems and methods for utilizing level differences to attenuate noise and enhance speech are provided. In exemplary embodiments, energy estimates of acoustic signals, representing captured sound, are determined in order to determine a level difference. This level difference in combination with a noise estimate, based only on a primary acoustic signal, allow a filter estimate to be derived. In some embodiments, the derived filter estimate may be smoothed. The filter estimate is then applied to the acoustic signal from the primary acoustic signal to generate a speech estimate.
1 . A method for enhancing speech, comprising:
determining a filter estimate, using at least one hardware processor, during a frame, the filter estimate based on a noise estimate of a primary acoustic signal, an energy estimate of the primary acoustic signal, and a level difference based on the primary acoustic signal and a secondary acoustic signal, the primary acoustic signal and the secondary acoustic signal each representing at least one captured sound; and
applying the filter estimate to the primary acoustic signal to produce a speech estimate.
2 . The method of claim 1 , further comprising determining an energy estimate for each of the acoustic signals during the frame.
3 . The method of claim 2 , wherein the energy estimate of the primary acoustic signal is approximated as E 1 (t,ω)=λ E |X 1 (t,ω)| 2 +(1−λ E )E 1 (t−1,ω).
4 . The method of claim 2 , wherein the energy estimate of the secondary acoustic signal is approximated as E 2 (t,ω)=λ E |X 2 (t,ω)| 2 +(1−λ E )E 2 (t−1,ω).
5 . The method of claim 2 , further comprising using the energy estimates to determine the level difference for the frame.
6 . The method of claim 5 , wherein the level difference is approximated by
ILD
(
t
,
ω
)
=
[
1
-
2
E
1
(
t
,
ω
)
E
2
(
t
,
ω
)
E
1
2
(
t
,
ω
)
+
E
2
2
(
t
,
ω
)
]
*
sign
(
E
1
(
t
,
ω
)
-
E
2
(
t
,
ω
)
)
.
7 . The method of claim 5 , wherein the level difference is approximated by
ILD
(
t
,
ω
)
=
E
1
(
t
,
ω
)
-
E
2
(
t
,
ω
)
E
1
(
t
,
ω
)
+
E
2
(
t
,
ω
)
.
8 . The method of claim 1 , wherein the noise estimate is based on the energy estimate of the primary acoustic signal and the level difference.
9 . The method of claim 8 , wherein the noise estimate is approximated as N(t,ω)=λ I (t,ω)E 1 (t,ω)+(1−λ I (t,ω))min[N(t−1,ω)E 1 (t,ω)].
10 . The method of claim 1 , further comprising smoothing the filter estimate prior to applying the filter estimate to the primary acoustic signal.
11 . The method of claim 10 , wherein the smoothing is approximated as M(t,ω)=λ s (t,ω)W(t,ω)+(1−λ s (t,ω))M(t−1,ω).
12 . The method of claim 1 , further comprising converting the speech estimate to a time domain.
13 . The method of claim 1 , further comprising outputting the speech estimate to a user.
14 . The method of claim 1 , wherein the at least one hardware processor comprises an audio processing engine.
15 . A system for enhancing speech on a device, comprising:
a primary microphone configured to receive a primary acoustic signal;
a secondary microphone located a distance away from the primary microphone and configured to receive a secondary acoustic signal; and
an audio processing engine configured to enhance speech in the primary acoustic signal, the audio processing engine comprising:
a noise estimate module configured to determine a noise estimate for the primary acoustic signal based on an energy estimate of the primary acoustic signal and a level difference, the level difference being based on the primary acoustic signal and a secondary acoustic signal; and
a filter module configured to determine a filter estimate to be applied to the primary acoustic signal to generate a filtered acoustic signal, the filter estimate based on (i) the noise estimate of the primary acoustic signal, (ii) the energy estimate of the primary acoustic signal, and (iii) the level difference.
16 . The system of claim 15 , wherein the audio processing engine further comprises an energy module configured to determine an energy estimate for each of the acoustic signals during a frame.
17 . The system of claim 15 , wherein the audio processing engine further comprises a filter smoothing module configured to smooth the filter estimate prior to applying the filter estimate to the primary acoustic signal.
18 . The system of claim 15 , wherein the audio processing engine further comprises a masking module configured to determine the speech estimate.
19 . A non-transitory computer readable medium having embodied thereon a program, the program being executable by a machine to perform a method for enhancing speech, the method comprising:
determining a filter estimate, using at least one hardware processor, during a frame, the filter estimate based on:
(i) a noise estimate of a primary acoustic signal,
(ii) an energy estimate of the primary acoustic signal, and
(iii) a level difference based on the primary acoustic signal and a secondary acoustic signal, the primary acoustic signal representing at least one captured sound and the secondary acoustic signal representing at least one other captured sound; and
applying the filter estimate to the primary acoustic signal to produce a speech estimate.
20 . The non-transitory computer readable medium of claim 19 , wherein the at least one hardware processor comprises an audio processor.