Time-warping of decoded audio signal after packet loss
A technique is described for use in a decoder configured to decode a series of frames representing an encoded audio signal. The technique is for transitioning between a lost frame and one or more received frames following the lost frame in the series of frames. In accordance with the technique, an output audio signal associated with the lost frame is synthesized. An extrapolated signal is generated based on the synthesized output audio signal. A time lag is calculated between the extrapolated signal and a decoded audio signal associated with the received frame(s), wherein the time lag represents a phase difference between the extrapolated signal and the decoded audio signal. The decoded audio signal is time-warped based on the time lag, wherein time-warping the decoded audio signal comprises stretching or shrinking the decoded audio signal in the time domain.
1. A method for generating an output audio signal corresponding to an encoded audio signal represented by a series of frames, comprising:
obtaining a first decoded audio signal corresponding to one or more received frames that precede one or more lost frames in the series of frames;
generating a concealment signal based on the first decoded audio signal;
obtaining a second decoded audio signal corresponding to one or more received frames that follow the lost frame(s) in the series of frames;
calculating a time lag between the concealment signal and the second decoded audio signal, the time lag representing a phase difference between the concealment signal and the second decoded audio signal; and
time-warping the second decoded audio signal based on the time lag, wherein time-warping the second decoded audio signal comprises stretching or shrinking the second decoded audio signal in the time domain.
2. The method of claim 1 , wherein calculating the time lag between the concealment signal and the second decoded audio signal comprises maximizing a correlation between the concealment signal and the second decoded audio signal.
3. The method of claim 2 , wherein maximizing the correlation between the concealment signal and the second decoded audio signal comprises searching for a peak of a normalized cross-correlation function R(k) between the concealment signal and the second decoded audio signal for a time lag range of ±MAXOS around zero:
R
(
k
)
=
∑
i
-
0
LSW
-
1
es
(
i
-
k
)
·
x
(
i
)
∑
i
=
0
LSW
-
1
es
2
(
i
-
k
)
∑
i
=
0
LSW
-
1
x
2
(
i
)
,
k
=
-
MAXOS
,
…
,
MAXOS
where es is the concealment signal, x is the second decoded audio signal, MAXOS is a maximum allowed offset, LSW is a length of a lag search window, and i=0 represents a first sample in the lag search window.
4. The method of claim 1 , wherein calculating the time lag between the concealment signal and the second decoded audio signal comprises:
searching for a first peak of a normalized cross-correlation function between the concealment signal and the second decoded audio signal using a first lag search range and a first lag search window to identify a coarse time lag, wherein the first lag search range specifies a range over which a starting point of the concealment signal is shifted during the search and the first lag search window specifies a number of samples over which the normalized cross-correlation function is computed; and
searching for a second peak of a normalized cross-correlation function between the concealment signal and the second decoded audio signal using a second lag search range and a second lag search window to identify a refined time lag, wherein the second lag search range is smaller than the first lag search range.
5. The method of claim 4 , wherein searching for the first peak of the normalized cross-correlation function between the concealment signal and the second decoded audio signal comprises searching for a peak of a normalized cross-correlation function between down-sampled representations of the concealment signal and the second decoded audio signal.
6. The method of claim 4 , wherein the second lag search window is smaller than the first lag search window.
7. The method of claim 4 , wherein searching for the second peak of the normalized cross-correlation function between the concealment signal and the second decoded audio signal using the second lag search range and the second lag search window comprises aligning the second lag search window with a center of an overlap add region of the second decoded audio signal.
8. The method of claim 1 , wherein calculating the time lag between the concealment signal and the second decoded audio signal comprises:
partially decoding the received frame(s) that follow the lost frame(s) in the series of frames to generate an approximation of the second decoded audio signal; and
calculating a time lag between the concealment signal and the approximation of the second decoded audio signal.
9. The method of claim 8 , wherein partially decoding the received frame(s) that follow the lost frame(s) in the series of frames comprises:
decoding a low-band bit stream associated with the received frame(s) that follow the lost frame(s) in the series of frames in a low-band adaptive differential pulse code modulation (ADPCM) decoder to generate a low-band reconstructed signal; and
using the low-band reconstructed signal as the approximation of the second decoded audio signal.
10. The method of claim 9 , wherein decoding the low-band bit stream associated with the received frame(s) that follow the lost frame(s) in the series of frames in the low-band ADPCM decoder comprises fixing coefficients of a two-pole, six-zero adaptive filter during the decoding of the low-band bit stream.
11. The method of claim 1 , further comprising:
overlap-adding the time-warped second decoded audio signal and a portion of the concealment signal.
12. The method of claim 1 , wherein overlap-adding the time-warped second decoded audio signal and the portion of the concealment signal comprises:
moving an overlap-add region associated with the time-warped second decoded audio signal forward in time to account for a period of decoder instability.
13. The method of claim 1 , wherein stretching the second decoded audio signal in the time domain comprises periodically performing the following steps:
repeating a sample of the second decoded audio signal; and
overlap-adding a portion of the second decoded audio signal up to and including the repeated sample and a portion of the second decoded audio signal following the repeated sample.
14. The method of claim 1 , wherein shrinking the second decoded audio signal in the time domain comprises periodically performing the following steps:
dropping a sample from the second decoded audio signal; and
overlap-adding a portion of the second decoded audio signal prior to the dropped sample and a portion of the second decoded audio signal following the dropped sample.
15. The method of claim 1 , further comprising:
time-warping a portion of the concealment signal based on the time lag, wherein time-warping the portion of the concealment signal comprises stretching or shrinking the portion of the concealment signal in the time domain.
16. A system, comprising:
a decoder configured to obtain a first decoded audio signal corresponding to one or more received frames that precede one or more lost frames in a series of frames of an encoded audio signal and a second decoded audio signal corresponding to one or more received frames that follow the lost frame(s) in the series of frames;
an audio signal synthesizer configured to generate a concealment signal based on the first decoded audio signal; and
time-warping logic configured to calculate a time lag between the concealment signal and the second decoded audio signal and to time-warp the second decoded audio signal based on the time lag;
wherein the time lag represents a phase difference between the concealment signal and the second decoded audio signal and wherein time-warping the second decoded audio signal comprises stretching or shrinking the second decoded audio signal in the time domain.
17. The system of claim 16 , wherein the time-warping logic is configured to calculate the time lag between the concealment signal and the second decoded audio signal by maximizing a correlation between the concealment signal and the second decoded audio signal.
18. The system of claim 17 , wherein the time-warping logic is configured to maximize the correlation between the concealment signal and the second decoded audio signal by searching for a peak of a normalized cross-correlation function R(k) between the concealment signal and the second decoded audio signal for a time lag range of ±MAXOS around zero:
R
(
k
)
=
∑
i
=
0
LSW
-
1
es
(
i
-
k
)
·
x
(
i
)
∑
i
=
0
LSW
-
1
es
2
(
i
-
k
)
∑
i
=
0
LSW
-
1
x
2
(
i
)
,
k
=
-
MAXOS
,
…
,
MAXOS
where es is the concealment signal, x is the second decoded audio signal, MAXOS is a maximum allowed offset, LSW is a length of a lag search window, and i=0 represents a first sample in the lag search window.
19. The system of claim 16 , wherein the time-warping logic is configured to search for a first peak of a normalized cross-correlation function between the concealment signal and the second decoded audio signal using a first lag search range and a first lag search window to identify a coarse time lag, wherein the first lag search range specifies a range over which a starting point of the concealment signal is shifted during the search and the first lag search window specifies a number of samples over which the normalized cross-correlation function is computed, and to search for a second peak of a normalized cross-correlation function between the concealment signal and the second decoded audio signal using a second lag search range and a second lag search window to identify a refined time lag, wherein the second lag search range is smaller than the first lag search range.
20. The system of claim 19 , wherein the time-warping logic is configured to search for the first peak of the normalized cross-correlation function between the concealment signal and the second decoded audio signal by searching for a peak of a normalized cross-correlation function between down-sampled representations of the concealment signal and the second decoded audio signal.
21. The system of claim 19 , wherein the second lag search window is smaller than the first lag search window.
22. The system of claim 19 , wherein the time-warping logic is configured to align the second lag search window with a center of an overlap add region of the second decoded audio signal.
23. The system of claim 16 , wherein the time-warping logic is configured to partially decode the received frame(s) that follow the lost frame(s) in the series of frames to generate an approximation of the second decoded audio signal and to calculate a time lag between the concealment signal and the approximation of the second decoded audio signal.
24. The system of claim 23 , wherein the time-warping logic is configured to partially decode the received frame(s) that follow the lost frame(s) in the series of frames by decoding a low-band bit stream associated with the received frame(s) that follow the lost frame(s) in the series of frames in a low-band adaptive differential pulse code modulation (ADPCM) decoder to generate a low-band reconstructed signal and by using the low-band reconstructed signal as the approximation of the second decoded audio signal.
25. The system of claim 24 , wherein the time-warping logic is configured to fix coefficients of a two-pole, six-zero adaptive filter during the decoding of the low-band bit stream.
26. The system of claim 16 , wherein the time-warping logic is further configured to overlap-add the time-warped second decoded audio signal and a portion of the concealment signal.
27. The system of claim 16 , wherein the time-warping logic is further configured to move an overlap-add region associated with the time-warped second decoded audio signal forward in time to account for a period of decoder instability.
28. The system of claim 16 , wherein the time-warping logic is configured to stretch the second decoded audio signal in the time domain by periodically performing the following steps: repeating a sample of the second decoded audio signal and overlap-adding a portion of the second decoded audio signal up to and including the repeated sample and a portion of the second decoded audio signal following the repeated sample.
29. The system of claim 16 , wherein the time-warping logic is configured to shrink the second decoded audio signal in the time domain by periodically performing the following steps: dropping a sample from the second decoded audio signal and overlap-adding a portion of the second decoded audio signal prior to the dropped sample and a portion of the second decoded audio signal following the dropped sample.
30. The system of claim 16 , wherein the time-warping logic is further configured to time-warp a portion of the concealment signal based on the time lag, wherein time-warping the portion of the concealment signal comprises stretching or shrinking the waveform segment in the time domain.
31. A computer program product comprising a computer-readable medium having computer program logic recorded thereon for enabling a processor to generate an output audio signal corresponding to an encoded audio signal represented by a series of frames, the computer program logic comprising:
first means for enabling the processor to obtain a first decoded audio signal corresponding to one or more received frames that precede one or more lost frames in the series of frame;
second means for enabling the processor to generate a concealment signal based on the first decoded audio signal;
third means for enabling the processor to obtain a second decoded audio signal corresponding to one or more received frames that following the lost frame(s) in the series of frames;
fourth means for enabling the processor to calculate a time lag between the concealment signal and the second decoded audio signal, the time lag representing a phase difference between the concealment signal and the second decoded audio signal; and
time-warping the second decoded audio signal based on the time lag, wherein time-warping the second decoded audio signal comprises stretching or shrinking the second decoded audio signal in the time domain.