Method and apparatus for using incremental search sequence in audio error concealment
A method, a computer readable medium, and an apparatus for audio error concealment are provided. The apparatus may receive a plurality of audio frames. The apparatus may receive a first audio frame after the receiving of the plurality of audio frames. The apparatus may detect a second audio frame being lost in transmission. The second audio frame is transmitted after the plurality of audio frames and before the first audio frame. The apparatus may identify an audio segment within the plurality of audio frames that best matches a reference audio pattern of the plurality of audio frames. The identified audio segment may be received before a last audio frame of the plurality of audio frames. The apparatus may reconstruct the second audio frame based on audio data received subsequent to the identified audio segment.
1. A method of audio error concealment, the method comprising:
receiving a plurality of audio frames;
receiving a first audio frame after the receiving of the plurality of audio frames;
detecting a second audio frame being lost in transmission, the second audio frame being transmitted after the plurality of audio frames and before the first audio frame;
performing a first stage search within the plurality of audio frames using an incremental search sequence, wherein the incremental search starts from an oldest audio frame and gradually proceeds to newer audio frames in the plurality of audio frames, and a size of search indexes of the incremental search is incremented based on an incremental search sequence;
finding a first highest similarity value among respective first similarity values of first audio segments in the plurality of audio frames searched in the first stage search, a similarity value being a measure of similarity between a reference audio pattern and audio segments within the plurality of audio frames;
identifying a first stage best-match audio segment that best matches the reference audio pattern of the plurality of audio frames, wherein the first stage best-match audio segment has the first highest similarity value;
identifying a search window within the plurality of audio frames based on the first stage best-match audio segment;
performing a second stage search within the search window using a linear search sequence;
finding a second highest similarity value among respective second similarity values of second audio segments in the search window searched in the second stage search;
determining the second highest similarity value is higher than a threshold subsequent to finding the second highest similarity value;
identifying a second stage best-match audio segment in response to determining the second highest similarity value is higher than the threshold, the second stage best-match audio segment having the second highest similarity value;
repeating the first stage search using a next incremental search sequence in response to determining the second highest similarity value is less than or equal to the threshold, the next incremental search sequence being a different incremental search sequence; and
reconstructing the second audio frame based on audio data received subsequent to the identified second stage best-match audio segment.
2. The method of claim 1 , wherein the incremental search sequence is one of a linear incremental sequence, a linear incremental repeat sequence, or a Fibonacci sequence.
3. The method of claim 1 , wherein each audio frame comprises amplitude information of an audio signal over a predetermined period of time.
4. The method of claim 1 , wherein each audio frame is assigned a frame sequence number to indicate an order of transmission.
5. The method of claim 1 , wherein finding the first highest similarity value for identifying the first stage best-match audio segment and/or finding the second highest similarity value for identifying the second stage best-match audio segment comprises,
for each candidate audio segment, performing at least one of:
comparing audio waveform of the candidate audio segment and audio waveform of the reference audio pattern; or determining correlation between the candidate audio segment and the reference audio pattern.
6. The method of claim 5 , wherein the identified first stage best-match audio segment and/or the identified second stage best-match audio segment has at least one of:
a smallest audio waveform difference with the reference audio pattern; or a largest correlation with the reference audio pattern.
7. The method of claim 1 , further comprising playing back the reconstructed second audio frame after the plurality of audio frames and before the first audio frame.
8. The method of claim 1 , wherein the reference audio pattern is a last audio segment of a last audio frame of the plurality of audio frames.
9. The method of claim 1 , wherein the search window is centered at the first stage best-match audio segment identified at the first stage.
10. The method of claim 1 , wherein a current search index for the first stage best-match audio segment identified at the first stage corresponds to a current number in the incremental search sequence, and the search window starts from a previous search index corresponding to a previous number in the incremental search sequence to a next search index corresponding to a next number in the incremental search sequence, the current search index being the previous search index incremented by the current number and the next search index being the current search index incremented by the next number.
11. An apparatus for audio error concealment, the apparatus comprising:
a memory; and
at least one processor coupled to the memory and configured to:
receive a plurality of audio frames;
receive a first audio frame after the receiving of the plurality of audio frames;
detect a second audio frame being lost in transmission, the second audio frame being transmitted after the plurality of audio frames and before the first audio frame;
perform a first stage search within the plurality of audio frames using an incremental search sequence, wherein the incremental search starts from an oldest audio frame and gradually proceeds to newer audio frames in the plurality of audio frames, and
a size of search indexes of the incremental search is incremented based on an incremental search sequence;
find a first highest similarity value among respective first similarity values of first candidate audio segments in the plurality of audio frames searched in the first stage search, a similarity value being a measure of similarity between a reference audio pattern and audio segments within the plurality of audio frames;
identify a first stage best-match audio segment that best matches the reference audio pattern of the plurality of audio frames, wherein the first stage best-match audio segment has a first highest similarity value;
identify a search window within the plurality of audio frames based on the first stage best-match audio segment;
perform a second stage search within the search window using a linear search sequence;
find a second highest similarity value among respective second similarity values of second candidate audio segments in the search window searched in the second stage search;
determine the second highest similarity value is higher than a threshold subsequent to finding the second highest similarity value;
identify a second stage best-match audio segment in response to the determination of the second highest similarity value being higher than the threshold, the second stage best-match audio segment having the second highest similarity value;
repeat the first stage search using a next incremental search sequence in response to determining the second highest similarity value is less than or equal to the threshold, the next incremental search sequence being a different incremental search sequence; and
reconstruct the second audio frame based on audio data received subsequent to the identified second stage best-match audio segment.
12. The apparatus of claim 11 , wherein the incremental search sequence is one of a linear incremental sequence, a linear incremental repeat sequence, or a Fibonacci sequence.
13. The apparatus of claim 11 , wherein each audio frame comprises amplitude information of an audio signal over a predetermined period of time.
14. The apparatus of claim 11 , wherein each audio frame is assigned a frame sequence number to indicate an order of transmission.
15. The apparatus of claim 11 , wherein, to find the first highest similarity value for identifying the first stage best-match audio segment and/or find the second highest similarity value for identifying the second stage best-match audio segment, the at least one processor is configured to, for each candidate audio segment, perform at least one of:
comparing audio waveform of the candidate audio segment and audio waveform of the reference audio pattern; or determining correlation between the candidate audio segment and the reference audio pattern.
16. The apparatus of claim 15 , wherein the identified first stage best-match audio segment and/or the identified second stage best-match audio segment has at least one of:
a smallest audio waveform difference with the reference audio pattern; or a largest correlation with the reference audio pattern.
17. The apparatus of claim 11 , wherein the at least one processor is further configured to play back the reconstructed second audio frame after the plurality of audio frames and before the first audio frame.
18. The apparatus of claim 11 , wherein the reference audio pattern is a last audio segment of a last audio frame of the plurality of audio frames.
19. The apparatus of claim 11 , wherein a current search index for the first stage best-match audio segment identified at the first stage corresponds to a current number in the incremental search sequence, and the search window starts from a previous search index corresponding to a previous number in the incremental search sequence to a next search index corresponding to a next number in the incremental search sequence, the current search index being the previous search index incremented by the current number and the next search index being the current search index incremented by the next number.