IP Library Granted Patent US 7,711,563
Granted Patent B2
US 7,711,563 · App. 10/183,608 · Granted May 4, 2010

Method and system for frame erasure concealment for predictive speech coding based on extrapolation of speech waveform

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,711,563
App. No.
10/183,608
Granted
May 4, 2010
Kind
B2
Abstract

A method and system are provided for synthesizing a corrupted frame output from a decoder including one or more predictive filters. The corrupted frame is representative of one segment of a decoded signal output from the decoder. The method comprises extrapolating a replacement frame based upon another segment of the decoded signal and substituting the replacement frame for the corrupted frame. Finally, the internal states of the filters are updated based upon the substituting.

Claims (759)

1. A method of synthesizing a corrupted frame output from a decoder including one or more predictive filters, the corrupted frame being representative of one segment of a decoded signal output from the decoder, the method comprising:

extrapolating a waveform in the speech domain based upon another segment of the decoded signal output from the decoder;

merging via a processor by overlap-adding the extrapolated waveform with a ringing signal to determine a replacement waveform;

substituting a replacement frame for the corrupted frame output from the decoder, the substituting being in speech domain, and wherein the replacement frame includes the replacement waveform; and

updating internal states of the filters based upon the substituting.

2. The method of claim 1 , wherein each frame includes a predetermined number of samples.

3. The method of claim 2 , further comprising storing samples of the other segment of the decoded signal in a memory.

4. The method of claim 3 , further comprising determining a preliminary time lag (ppfep) based upon examining (K) number of stored samples in accordance with an analysis window.

5. The method of claim 4 , further comprising determining a final time lag (ppfe) and a scaling factor (ptfe).

6. The method of claim 5 , wherein the final time lag (ppfe) is determined based upon an examination of a number of stored samples; and

wherein the replacement frame is based upon the determined final time lag.

7. The method of claim 6 , wherein the preliminary time lag (ppfep) is chosen from candidate preliminary time lags (j); and

wherein correlation values (c(j)) respectively associated with each of the candidate preliminary time lags (j) are determined in accordance with the expression:

c

(

j

)

=

n

=

N

-

K

+

1

N

sq

(

n

)

sq

(

n

-

j

)

wherein,

(K) represents the size of the analysis window; and

sq(n) represents the decoded signal.

8. The method of claim 7 , wherein the chosen preliminary time lag (ppfep) maximizes a pitch prediction gain within the analysis window.

9. The method of claim 7 , wherein the chosen preliminary time lag (ppfep) maximizes the expression:

n

c

(

j

)

=

(

n

=

N

-

K

+

1

N

sq

(

n

)

sq

(

n

-

j

)

)

2

n

=

N

-

k

+

1

N

sq

2

(

n

-

j

)

.

10. The method of claim 9 , wherein a periodic repetition flag is set based upon independently applying the analysis window to the (K) number of stored samples.

11. The method of claim 10 , wherein the decoded signal is a speech signal and the independently applying determines an amount of speech energy (E) within the analysis window.

12. The method of claim 10 , wherein the independently applying determines an amount of energy (E) within the analysis window.

13. The method of claim 12 , wherein the amount of energy (E) is determined in accordance with the expression:

E

=

n

=

N

-

K

+

1

N

sq

2

(

n

)

.

14. The method of claim 13 , wherein the periodic repetition flag is set based upon a comparison between the determined amount of energy (E) and a predetermined threshold.

15. The method of claim 14 , wherein when the amount of energy (E) meets the predetermined threshold requirement; and

wherein a first normalized autocorrelation coefficient (ρ 1 ) is determined in accordance with the expression:

ρ

1

=

n

=

N

-

K

+

2

N

sq

(

n

)

sq

(

n

-

1

)

E

.

16. The method of claim 15 , wherein the periodic repetition flag is set if the first normalized autocorrelation coefficient (ρ 1 ) meets a predetermined threshold requirement.

17. The method of claim 16 , wherein if the periodic repetition flag is not set, the final time lag (ppfe) is determined based upon time lag search of the stored samples of the decoded signal.

18. The method of claim 17 , wherein the final time lag (ppfe) search of the stored samples is conducted in accordance with the expression:

m ×ppfep≧ T 0

wherein,

(m) represents an integer value; and

(T 0 ) a minimum interval threshold.

19. The method of claim 18 , wherein time lags (j) are determined within a range of values; and

wherein the final time lag (ppfe) minimizes the expression:

D

(

j

)

=

n

=

1

d

(

sq

(

N

+

n

-

j

)

-

r

(

n

)

)

2

.

20. The method of claim 19 , wherein the scaling factor (ptfe) is determined in accordance with the expression:

ptfe

=

n

=

1

d

r

(

n

)

n

=

1

d

sq

(

N

+

n

-

ppfe

)

wherein,

(d) represents a first number of samples of ringing of one of the filters; and

(r(n)) represents a ringing signal output from the decoder.

21. The method of claim 1 , wherein the decoded signal is a speech signal.

22. The method of claim 1 , wherein the ringing signal is derived by calculating a predetermined number of samples of a zero-input response of the filters.

23. The method of claim 22 , wherein the samples are collected from the beginning of the corrupted frame.

24. A method of changing memory content in a decoder including one or more predictive filters, the decoder being configured to synthesize a corrupted frame representative of one segment of a decoded signal output from the decoder, the method comprising:

extrapolating a waveform in the speech domain based upon another segment of the decoded signal output from the decoder;

merging via a processor by overlap-adding the extrapolated waveform with a ringing signal to determine a replacement waveform;

substituting a replacement frame for the corrupted frame output from the decoder, the substituting being in speech domain, and wherein the replacement frame includes the replacement waveform; and

updating internal states of the filters based upon the substituting;

wherein the updating includes updating a first memory of a short-term variety of one of the predictive filters to match a last (M) number of samples of the replacement waveform signal in reversed order when the short-term predictive filter is of an order (M).

25. The method of claim 24 , wherein the updating of the first memory of the short-term predictive filter occurs in accordance with the expression:

stsm ( k )= sq ( N+N f +1− k ), k=1, 2, . . . , M

wherein,

stsm(k) represents the k-th memory value of a short-term synthesis filter associated with the short-term predictive filters;

(M) represents the last M samples of the one segment of the decoded signal;

N represents the number of decoder output speech samples in previous frames that are stored; and

N f represent the number of samples in a frame.

26. The method of claim 25 , wherein a long-term variety of one of the predictive filters is updated by performing short-term prediction error filtering of the extrapolated signal;

wherein a second memory of the short-term predictive filter is updated to a last (M) number of the (N) number of stored samples; and

wherein the storing is performed in a reversed order.

27. The method of claim 26 , wherein the updating of the second memory of the short-term predictive filter occurs in accordance with the expression:

stpm ( k )= sq ( N+ 1− k ), k=1, 2, . . . , M

wherein,

stpm(k) represents the k-th memory value for a short-term prediction error filter associated with a second of the predictive filters.

28. An apparatus for synthesizing a corrupted frame output from a decoder including one or more predictive filters, the corrupted frame being representative of one segment of a decoded signal output from the decoder, the apparatus comprising:

means for extrapolating a waveform in the speech domain based upon another segment of the decoded signal;

means for merging by overlap-adding the extrapolated waveform with a ringing signal to determine a replacement waveform;

means for substituting a replacement frame for the corrupted frame, the substituting being in speech domain, and wherein the replacement frame includes the replacement waveform; and

means for updating internal states of the filters based upon the substituting.

29. The apparatus of claim 28 , wherein each frame includes a predetermined number of samples.

30. The apparatus of claim 29 , further comprising means for storing samples of the other segment of the decoded signal in a memory.

31. The apparatus of claim 30 , further comprising means for determining a preliminary time lag (ppfep) based on examining (K) number of stored samples in accordance with an analysis window.

32. The apparatus of claim 31 , further comprising means for determining a final time lag (ppfe) and a scaling factor (ptfe).

33. The apparatus of claim 32 , wherein the final time lag (ppfe) is determined based upon an examination of a number of stored samples; and

wherein the replacement frame is based upon the determined final time lag.

34. The apparatus of claim 33 , wherein the preliminary time lag (ppfep) is chosen from candidate preliminary time lags (j); and

wherein correlation values (c(j)) respectively associated with each of the candidate preliminary time lags (j) are determined in accordance with the expression:

c

(

j

)

=

n

=

N

-

K

+

1

N

sq

(

n

)

sq

(

n

-

j

)

wherein,

(K) represents the size of the analysis window; and

sq(n) represents the decoded signal.

35. The apparatus of claim 34 , wherein the chosen preliminary time lag (ppfep) maximizes a pitch prediction gain within the analysis window.

36. The apparatus of claim 34 , wherein the chosen preliminary time lag (ppfep) maximizes the expression:

n

c

(

j

)

=

(

n

=

N

-

K

+

1

N

sq

(

n

)

sq

(

n

-

j

)

)

2

n

=

N

-

K

+

1

N

sq

2

(

n

-

j

)

.

37. The apparatus of claim 36 , further comprising means for applying the analysis window;

wherein a periodic repetition flag is set based upon independently applying the analysis window to the (K) number of stored samples.

38. The apparatus of claim 37 , wherein the decoded signal is a speech signal and the means for applying determines an amount of speech energy (E) within the analysis window.

39. The apparatus of claim 37 , wherein the means for applying determines an amount of energy (E) within the analysis window.

40. The apparatus of claim 39 , wherein the amount of energy (E) is determined in accordance with the expression:

E

=

n

=

N

-

K

+

1

N

sq

2

(

n

)

.

41. The apparatus of claim 40 , wherein the periodic repetition flag is set based upon a comparison between the determined amount of energy (E) and a predetermined threshold.

42. The apparatus of claim 41 , wherein when the amount of energy (E) meets the predetermined threshold requirement; and

wherein a first normalized autocorrelation coefficient (ρ 1 ) is determined in accordance with the expression:

ρ

1

=

n

=

N

-

K

+

2

N

sq

(

n

)

sq

(

n

-

1

)

E

.

43. The apparatus of claim 42 , wherein the periodic repetition flag is set if the first normalized autocorrelation coefficient (ρ 1 ) meets a predetermined threshold requirement.

44. The apparatus of claim 43 , wherein if the periodic repetition flag is not set, the final time lag (ppfe) is determined based upon time lag search of the stored samples of the decoded signal.

45. The apparatus of claim 44 , wherein the final time lag (ppfe) search of the stored samples is conducted in accordance with the expression:

m ×ppfep≧ T 0

wherein,

(in) represents an integer value; and

(T 0 ) a minimum interval threshold.

46. The apparatus of claim 45 , wherein time lags (j) are determined within a range of values; and

wherein the final time lag (ppfe) minimizes the expression:

D

(

j

)

=

n

=

1

d

(

sq

(

N

+

n

-

j

)

-

r

(

n

)

)

2

.

47. The apparatus of claim 46 , wherein the scaling factor (ptfe) is determined in accordance with the expression:

ptfe

=

n

=

1

d

r

(

n

)

n

=

1

d

sq

(

N

+

n

-

ppfe

)

wherein,

(d) represents a first number of samples of ringing of one of the filters; and

(r(n)) represents a ringing signal output from the decoder.

48. The apparatus of claim 28 , wherein the decoded signal is a speech signal.

49. An apparatus for changing memory content in a decoder including one or more predictive filters, the decoder being configured to synthesize a corrupted frame representative of one segment of a decoded signal output from the decoder, the apparatus comprising:

means for extrapolating a waveform in the speech domain based upon another segment of the decoded signal;

means for merging by overlap-adding the extrapolated waveform with a ringing signal to determine a replacement waveform;

means for substituting a replacement frame for the corrupted frame, the substituting being in speech domain, and wherein the replacement frame includes the replacement waveform; and

means for updating internal states of the filters based upon the substituting;

wherein the means for updating update a first memory of a short-term variety of one of the predictive filters to match a last (M) number of samples of the replacement waveform signal in reversed order when the short-term predictive filter is of an order (M).

50. The apparatus of claim 49 , wherein the updating of the first memory of the short-term predictive filter occurs in accordance with the expression:

stsm ( k )= sq ( N+N f +1− k ), k=1, 2, . . . , M

wherein,

stsm(k) represents the k-th memory value of a short-term synthesis filter associated with the short-term predictive filters;

(M) represents the last M samples of the one segment of the decoded signal;

N represents the number of decoder output speech samples in previous frames that are stored; and

N f represent the number of samples in a frame.

51. The apparatus of claim 50 , wherein a long-term variety of one of the predictive filters is updated by performing short-term prediction error filtering of the extrapolated signal;

wherein a second memory of the short-term predictive filter is updated to a last (M) number of the (N) number of stored samples; and

wherein the storing is performed in a reversed order.

52. The apparatus of claim 51 , wherein the updating of the second memory of the short-term predictive filter occurs in accordance with the expression:

stpm ( k )= sq ( N+ 1− k ), k=1, 2, . . . , M

wherein,

stpm(k) represents the k-th memory value for a short-term prediction error filter associated with a second of the predictive filters.

53. A computer readable tangible medium carrying one or more sequences of one or more instructions for execution by one or more processors to perform a method of synthesizing a corrupted frame output from a decoder including one or more predictive filters, the corrupted frame being representative of one segment of a decoded signal output from the decoder, the instructions when executed by the one or more processors, cause the one or more processors to perform the steps of:

extrapolating a waveform in the speech domain based upon another segment of the decoded signal output from the decoder;

merging by overlap-adding the extrapolated waveform with a ringing signal to determine a replacement waveform;

substituting a replacement frame for the corrupted frame output from the decoder, the substituting being in speech domain, and wherein the replacement frame includes the replacement waveform; and

updating internal states of the filters based upon the substituting.

54. The computer readable tangible medium of claim 53 , wherein each frame includes a predetermined number of samples.

55. The computer readable tangible medium of claim 54 , carrying the one or more instructions, further causing the one or more processors to store samples of the other segment of the decoded signal in a memory.

56. The computer readable tangible medium of claim 55 , further causing the one or more processors to determine a preliminary time lag (ppfep) based upon examining (K) number of stored samples in accordance with an analysis window.

57. The computer readable tangible medium of claim 56 , carrying the one or more instructions, further causing the one or more processors to determine a final time lag (ppfe) and a scaling factor (ptfe).

58. The computer readable tangible medium of claim 57 , wherein the final time lag (ppfe) is determined based upon an examination of a number of stored samples; and

wherein the replacement frame is based upon the determined final time lag.

59. The computer readable tangible medium of claim 58 , wherein the preliminary time lag (ppfep) is chosen from candidate preliminary time lags (j); and

wherein correlation values (c(j)) respectively associated with each of the candidate preliminary time lags (j) are determined in accordance with the expression:

c

(

j

)

=

n

=

N

-

K

+

1

N

sq

(

n

)

sq

(

n

-

j

)

wherein,

(K) represents the size of the analysis window; and

sq(n) represents the decoded signal.

60. The computer readable tangible medium of claim 59 , wherein the chosen preliminary time lag (ppfep) maximizes a pitch prediction gain within the analysis window.

61. The computer readable tangible medium of claim 59 , wherein the chosen preliminary time lag (ppfep) maximizes the expression:

n

c

(

j

)

=

(

n

=

N

-

K

+

1

N

sq

(

n

)

sq

(

n

-

j

)

)

2

n

=

N

-

K

+

1

N

sq

2

(

n

-

j

)

.

62. The computer readable tangible medium of claim 61 , wherein a periodic repetition flag is set based upon independently applying the analysis window to the (K) number of stored samples.

63. The computer readable tangible medium of claim 62 , wherein the decoded signal is a speech signal and the independently applying determines an amount of speech energy (E) within the analysis window.

64. The computer readable tangible medium of claim 62 , wherein the independently applying determines an amount of energy (E) within the analysis window.

65. The computer readable tangible medium of claim 64 , wherein the amount of energy (E) is determined in accordance with the expression:

E

=

n

=

N

-

K

+

1

N

sq

2

(

n

)

.

66. The computer readable tangible medium of claim 65 , wherein the periodic repetition flag is set based upon a comparison between the determined amount of energy (E) and a predetermined threshold.

67. The computer readable tangible medium of claim 66 , wherein when the amount of energy (E) meets the predetermined threshold requirement; and

wherein a first normalized autocorrelation coefficient (ρ 1 ) is determined in accordance with the expression:

ρ

1

=

n

=

N

-

K

+

2

N

sq

(

n

)

sq

(

n

-

1

)

E

.

68. The computer readable tangible medium of claim 67 , wherein the periodic repetition flag is set if the first normalized autocorrelation coefficient (ρ 1 ) meets a predetermined threshold requirement.

69. The computer readable tangible medium of claim 68 , wherein if the periodic repetition flag is not set, the final time lag (ppfe) is determined based upon time lag search of the stored samples of the decoded signal.

70. The computer readable tangible medium of claim 69 , wherein the final time lag (ppfe) search of the stored samples is conducted in accordance with the expression:

m ×ppfep≧ T 0

wherein,

(m) represents an integer value; and

(T 0 ) a minimum interval threshold.

71. The computer readable tangible medium of claim 70 , wherein time lags (j) are determined within a range of values; and

wherein the final time lag (ppfe) minimizes the expression:

D

(

j

)

=

n

=

1

d

(

sq

(

N

+

n

-

j

)

-

r

(

n

)

)

2

.

72. The computer readable tangible medium of claim 71 , wherein the scaling factor (ptfe) is determined in accordance with the expression:

ptfe

=

n

=

1

d

|

r

(

n

)

|

n

=

1

d

|

sq

(

N

+

n

-

ppfe

)

|

wherein,

(d) represents a first number of samples of ringing of one of the filters; and

(r(n)) represents a ringing signal output from the decoder.

73. The computer readable tangible medium of claim 53 , wherein the decoded signal is a speech signal.

74. A computer readable tangible medium carrying one or more sequences of one or more instructions for execution by one or more processors, the instructions when executed by the one or more processors, cause the one or more processors to perform the steps of:

extrapolating a waveform in the speech domain based upon another segment of the decoded signal output from the decoder;

merging by overlap-adding the extrapolated waveform with a ringing signal to determine a replacement waveform;

substituting a replacement frame for the corrupted frame output from the decoder, the substituting being in speech domain, and wherein the replacement frame includes the replacement waveform; and

updating internal states of the filters based upon the substituting;

wherein the updating includes updating a first memory of a short-term variety of one of the predictive filters to match a last (M) number of samples of the replacement waveform signal in reversed order when the short-term predictive filter is of an order (M).

75. The computer readable tangible medium of claim 74 , wherein the updating of the first memory of the short-term predictive filter occurs in accordance with the expression:

stsm ( k )= sq ( N+N f +1− k ), k=1, 2, . . . , M

wherein,

stsm(k) represents the k-th memory value of a short-term synthesis filter associated with the short-term predictive filters;

(M) represents the last M samples of the one segment of the decoded signal;

N represents the number of decoder output speech samples in previous frames that are stored; and

N f represent the number of samples in a frame.

76. The computer readable tangible medium of claim 75 , wherein a long-term variety of one of the predictive filters is updated by performing short-term prediction error filtering of the extrapolated signal;

wherein a second memory of the short-term predictive filter is updated to a last (M) number of the (N) number of stored samples; and

wherein the storing is performed in a reversed order.

77. The computer readable tangible medium of claim 76 , wherein the updating of the second memory of the short-term predictive filter occurs in accordance with the expression:

stpm ( k )= sq ( N+ 1− k ), k=1, 2, . . . , M

wherein,

stpm(k) represents the k-th memory value for a short-term prediction error filter associated with a second of the predictive filters.

Assignments (7)
CORRECTIVE ASSIGNMENT TO CORRECT THE PROPERTY NUMBERS PREVIOUSLY RECORDED AT REEL: 47630 FRAME: 344. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Mar 21, 2019
From: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD.
To: AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE. LIMITED
Reel/Frame 048883/0267 →
CORRECTIVE ASSIGNMENT TO CORRECT THE EFFECTIVE DATE OF MERGER TO 9/5/2018 PREVIOUSLY RECORDED AT REEL: 047196 FRAME: 0687. ASSIGNOR(S) HEREBY CONFIRMS THE MERGER. Recorded Oct 29, 2018
From: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD.
To: AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE. LIMITED
Reel/Frame 047630/0344 →
MERGER Recorded Oct 4, 2018
From: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD.
To: AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE. LIMITED
Reel/Frame 047196/0687 →
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS Recorded Feb 3, 2017
From: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
To: BROADCOM CORPORATION
Reel/Frame 041712/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 1, 2017
From: BROADCOM CORPORATION
To: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD.
Reel/Frame 041706/0001 →
PATENT SECURITY AGREEMENT Recorded Feb 11, 2016
From: BROADCOM CORPORATION
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 037806/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 28, 2002
From: CHEN, JUIN-HWEY
To: BROADCOM CORPORATION
Reel/Frame 013063/0666 →