IP Library Granted Patent US 10,957,331
Granted Patent B2
US 10,957,331 · App. 16/222,833 · Granted Mar 23, 2021

Phase reconstruction in a speech decoder

Inventors: Soren Skak Jensen (Vancouver, CA); Sriram Srinivasan (Sammamish, WA); Koen Bernard Vos (Singapore, SG)
Assignee: Microsoft Technology Licensing, LLC
G10L19/0018G10L19/265G10L25/12G10L25/69G10L25/72
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,957,331
App. No.
16/222,833
Granted
Mar 23, 2021
Kind
B2
Abstract

Innovations in phase quantization during speech encoding and phase reconstruction during speech decoding are described. For example, to encode a set of phase values, a speech encoder omits higher-frequency phase values and/or represents at least some of the phase values as a weighted sum of basis functions. Or, as another example, to decode a set of phase values, a speech decoder reconstructs at least some of the phase values using a weighted sum of basis functions and/or reconstructs lower-frequency phase values then uses at least some of the lower-frequency phase values to synthesize higher-frequency phase values. In many cases, the innovations improve the performance of a speech codec in low bitrate scenarios, even when encoded data is delivered over a network that suffers from insufficient bandwidth or transmission quality problems.

Claims (69)

1. In a computer system that implements a speech decoder, a method comprising:

receiving encoded data as part of a bitstream;

decoding the encoded data to reconstruct speech, including:

decoding residual values, including:

decoding a set of phase values, including reconstructing at least some of the set of phase values using a linear component and a weighted sum of basis functions; and

reconstructing the residual values based at least in part on the set of phase values; and

filtering the residual values according to linear prediction coefficients; and

storing the reconstructed speech for output.

2. The method of claim 1 , wherein the reconstructing the residual values includes:

repeating the set of phase values for one or more subframes of a current frame;

based at least in part on the repeated sets of phase values for the respective subframes, reconstructing complex amplitude values for the respective subframes; and

applying an inverse frequency transform to the complex amplitude values for the respective subframes.

3. The method of claim 1 , wherein the reconstructed phase values are a first subset of the set of phase values, and wherein the decoding the set of phase values further includes using at least some of the first subset to synthesize a second subset of the set of phase values, each of the second subset having a frequency above a cutoff frequency.

4. The method of claim 3 , wherein the decoding the set of phase values further includes determining the cutoff frequency based at least in part on a target bitrate for the encoded data and/or pitch cycle information.

5. The method of claim 1 , wherein the basis functions are sine functions.

6. The method of claim 1 , wherein the decoding the set of phase values further includes:

decoding a set of coefficients that weight the basis functions;

decoding an offset value and a slope value that parameterize the linear component; and

using the set of coefficients, the offset value, and the slope value as part of the reconstructing the at least some of the set of phase values.

7. The method of claim 1 , wherein the decoding the set of phase values further includes, based at least in part on a target bitrate for the encoded data, determining a count of coefficients that weight the basis functions.

8. The method of claim 1 , wherein the reconstructing the residual values includes:

based at least in part on the set of phase values, reconstructing complex amplitude values for one or more subframes;

adaptively smoothing the complex amplitude values for the respective subframes based at least in part on one or more of pitch cycle information and differences in amplitude values across boundaries;

applying an inverse frequency transform to the smoothed complex amplitude values for the respective subframes; and

selectively adding noise to the residual values based at least in part on correlation values and a sparseness value.

9. One or more computer-readable memory or storage devices having stored thereon computer-executable instructions for causing one or more processors, when programmed thereby, to perform operations of a speech decoder, the operations comprising:

receiving encoded data as part of a bitstream;

decoding the encoded data to reconstruct speech, including:

decoding residual values, including:

decoding a set of phase values, including reconstructing a first subset of the set of phase values and using at least some of the first subset to synthesize a second subset of the set of phase values, each of the second subset having a frequency above a cutoff frequency; and

reconstructing the residual values based at least in part on the set of phase values; and

filtering the residual values according to linear prediction coefficients; and

storing the reconstructed speech for output.

10. The one or more computer-readable memory or storage devices of claim 9 , wherein the decoding the set of phase values further includes determining the cutoff frequency based at least in part on a target bitrate for the encoded data and/or pitch cycle information.

11. The one or more computer-readable memory or storage devices of claim 9 , wherein the using the at least some of the first subset to synthesize the second subset includes:

determining a pattern in a range of the first subset; and

repeating the pattern above the cutoff frequency.

12. The one or more computer-readable memory or storage devices of claim 11 , wherein the determining the pattern includes:

identifying the range of the first subset; and

determining, as the pattern, differences between adjacent phase values in the range of the first subset.

13. The one or more computer-readable memory or storage devices of claim 12 , wherein the using the at least some of the first subset to synthesize the second subset further includes:

after the repeating, integrating the differences between adjacent phase values to determine the second subset.

14. The one or more computer-readable memory or storage devices of claim 9 , wherein the reconstructing the first subset uses a linear component and a weighted sum of basis functions.

15. A computer system comprising:

an input buffer, implemented in memory of the computer system, configured to receive encoded data as part of a bitstream;

a speech decoder, implemented using one or more processors of the computer system, configured to decode the encoded data to reconstruct speech, the speech decoder including:

a residual decoder configured to decode residual values, wherein the residual decoder is configured to:

decode a set of phase values, including performing operations to reconstruct a first subset of the set of phase values using a linear component and a weighted sum of basis functions and/or use at least some of the first subset to synthesize a second subset of the set of phase values, each of the second subset having a frequency above a cutoff frequency; and

reconstruct the residual values based at least in part on the set of phase values; and

one or more synthesis filters configured to filter the residual values according to linear prediction coefficients; and

an output buffer configured to store the reconstructed speech for output.

16. The computer system of claim 15 , wherein, to decode the set of phase values, the residual decoder is further configured to determine the cutoff frequency based at least in part on a target bitrate for the encoded data and/or pitch cycle information.

17. The computer system of claim 15 , wherein, to decode the set of phase values, the residual decoder is further configured to perform operations to:

based at least in part on target bitrate for the encoded data, determine a count of coefficients that weight the basis functions;

decode a set of coefficients;

decode an offset value and a slope value that parameterize the linear component; and

use the set of coefficients, the offset value, and the slope value to reconstruct the first subset.

18. The computer system of claim 15 , wherein the speech decoder further includes:

a filter bank configured to combine multiple bands that result from filtering of the residual values in corresponding bands by synthesis filters, wherein the first subset is for a low band among the corresponding bands of the residual values, and wherein the second subset is for a high band among the corresponding bands of the residual values.

19. The computer system of claim 15 , wherein the speech decoder further includes one or more of:

(a) one or more LPC recovery modules configured to reconstruct the linear prediction coefficients; and

(b) a post-processing filter configured to selectively filter the reconstructed speech.

20. The computer system of claim 15 , wherein the residual decoder is further configured to:

reconstruct sets of magnitude values for one or more subframes;

reconstruct complex amplitude values for the respective subframes based at least in part on the sets of magnitude values for the respective subframes and the set of phase values;

adaptively smooth the complex amplitude values for the respective subframes based at least in part on one or more of pitch cycle information and differences in amplitude values across boundaries;

apply an inverse one-dimensional frequency transform to the smoothed complex amplitude values for the respective subframes;

decode a sparseness value and correlation values; and

selectively add noise to the residual values based at least in part on the correlation values and the sparseness value.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 18, 2018
From: JENSEN, SOREN SKAK; SRINIVASAN, SRIRAM; VOS, KOEN BERNARD
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 047805/0356 →
Continuity (1)
Related Publication 20200194017A1 · Jun 18, 2020
Cited By (1)
US 12,300,253