Method for eliminating an unwanted signal from a mixture via time-frequency masking
View Patent ↗A method is presented for eliminating an unwanted signal (e.g., background music, interference, etc.) from a mixture of a desired signal and the unwanted signal via time-frequency masking. Given a mixture of the desired signal and the unwanted signal, the goal of the present invention is to eliminate or at least reduce the effects of the unwanted signal to obtain an estimate of the desired signal.
1. A method for eliminating or reducing an unwanted signal from a recorded mixture of a desired signal and an unwanted signal given a recording of the unwanted signal without the desired signal, comprising:
aligning the recorded mixture and the recording of the unwanted signal without the desired signal;
computing a time-frequency representation of the recorded mixture to create a time-frequency recorded mixture;
computing a time-frequency representation of the redefined recording of the unwanted signal to create a time-frequency redefined recording of the unwanted signal;
determining a segment of time when only the redefined recording of the unwanted signal is present in the recorded mixture;
computing a value α(ω), wherein α(ω) is a modulus of a Widrow-Hoff estimate;
generating a time-frequency mask using the value α(ω), the time-frequency recorded mixture and the time-frequency redefined recording of the unwanted signal;
applying the time-frequency mask on the recorded mixture to compute a time-frequency desired signal; and
inverting the time-frequency desired signal to create a desired signal.
2. The method of claim 1 , wherein aligning the recorded mixture and the recording of the unwanted signal comprises:
estimating a delay between the recorded mixture and the recording of the unwanted signal; and
redefining the recording of the unwanted signal with respect to a delay between the recorded mixture and the recording of the unwanted signal to create a redefined recording of the unwanted signal.
3. The method of claim 2 , wherein estimating a delay between the recorded mixture and the recording of the unwanted signal comprises manually estimating the delay through optical inspection.
4. The method of claim 2 , wherein estimating a delay between the recorded mixture and the recording of the unwanted signal comprises performing cross-correlation alignment.
5. The method of claim 1 , wherein computing a time-frequency representation of the recorded mixture to create a time-frequency recorded mixture comprises computing
F
W
(
x
(
·
)
)
(
t
,
ω
)
=
1
2
π
∫
-
∞
∞
W
(
τ
-
t
)
x
(
τ
)
ⅇ
-
ⅈ
ωτ
ⅆ
τ
.
6. The method of claim 1 , wherein computing a time-frequency representation of the redefined recording of the unwanted signal to create a time-frequency redefined recording of the unwanted signal comprises computing
F
W
(
x
(
·
)
)
(
t
,
ω
)
=
1
2
π
∫
-
∞
∞
W
(
τ
-
t
)
x
(
τ
)
ⅇ
-
ⅈ
ωτ
ⅆ
τ
.
7. The method of claim 1 , wherein determining a segment of time when only the redefined recording of the unwanted signal is present in the recorded mixture comprises determining a segment of time when the desired signal is not of a sufficient auditory level to be heard by a human.
8. The method of claim 1 , wherein determining a segment of time when only the redefined recording of the unwanted signal is present in the recorded mixture comprises determining a segment of time when the desired signal is not present in the mixture.
9. The method of claim 1 , wherein computing a value α(ω) comprises computing
a
(
ω
)
=
∫
∈
(
t
0
,
t
1
)
x
^
(
t
,
ω
)
r
^
0
(
t
,
ω
)
_
ⅆ
t
∫
∈
(
t
0
,
t
1
)
r
^
(
t
,
ω
)
2
ⅆ
t
.
wherein {circumflex over (x)}(t,ω) is a windowed Fourier transform, and
{circumflex over (r)}(t,ω) is a filter process.
10. The method of claim 1 , wherein computing a value α(ω) comprises setting the value α(ω) to 1.
11. The method of claim 1 wherein computing a value α(ω) comprises computing adaptive updates to the value α(ω).
12. The method of claim 1 , wherein generating a time-frequency mask using the time-frequency recorded mixture and the time-frequency redefined original recording comprises computing
m
(
t
,
ω
)
=
{
1
if
x
^
(
t
,
ω
)
2
a
2
(
ω
)
r
^
(
t
,
ω
)
2
>
α
0
if
otherwise
.
13. The method of claim 1 , wherein generating a time-frequency mask using the time-frequency recorded mixture and the time-frequency redefined recording of the unwanted signal comprises computing
m
(
t
,
ω
)
=
1
{
x
^
(
t
,
ω
)
r
^
2
(
t
,
ω
)
>
α
}
,
wherein |{circumflex over (r)} 2 (t,ω)| is estimated from r 2 (t) and wherein r 2 (t) is a rerecording of the original recording in a similar environment and setup as the recorded mixture.
14. The method of claim 1 , wherein generating a time-frequency mask using the time-frequency recorded mixture and the time-frequency redefined original recording comprises computing m(t,ω)=1 {α(ω)|{circumflex over (r)} 0 (t,ω)|>β} .
15. The method of claim 1 , wherein inverting the time-frequency desired signal to create a desired signal comprises computing an inverted
F
W
(
x
(
·
)
)
(
t
,
ω
)
=
1
2
π
∫
-
∞
°
W
(
τ
-
t
)
x
(
τ
)
ⅇ
-
ⅈ
ω
τ
ⅆ
τ
.
16. A computer-readable medium having instructions stored thereon for execution by a processor to perform a method for eliminating or reducing an unwanted signal from a recorded mixture of a desired signal and an unwanted signal given a recording of the unwanted signal without the desired signal, comprising:
aligning the recorded mixture and the recording of the unwanted signal without the desired signal;
computing a time-frequency representation of the recorded mixture to create a time-frequency recorded mixture;
computing a time-frequency representation of the redefined original recording to create a time-frequency redefined original recording;
determining a segment of time when only the redefined original recording is present in the recorded mixture;
computing a value α(ω), wherein α(ω) is a modulus of a Widrow-Hoff estimate;
generating a time-frequency mask using the time-frequency recorded mixture and the time-frequency redefined original recording;
applying the time-frequency mask on the recorded mixture to compute a time-frequency desired signal; and
inverting the time-frequency desired signal to create a desired signal.
17. A method for eliminating or reducing an unwanted signal from a recorded mixture of a desired signal and an unwanted signal given a recording of the unwanted signal without the desired signal, comprising:
aligning the recorded mixture and the recording of the unwanted signal without the desired signal;
computing a time-scale representation of the recorded mixture to create a time-scale recorded mixture;
computing a time-scale representation of the redefined original recording to create a time-scale redefined original recording;
determining a segment of time when only the redefined original recording is present in the recorded mixture; computing a value α(ω), wherein α(ω) is a modulus of a Widrow-Hoff estimate;
generating a time-scale mask using the value α(ω), the time-scale recorded mixture and the time-scale redefined original recording;
applying the time-scale mask on the recorded mixture to compute a time-scale desired signal; and
inverting the time-scale desired signal to create a desired signal.