IP Library › Granted Patent US 11,410,668
Granted Patent B2
US 11,410,668 · App. 16/290,587 · Granted Aug 9, 2022

Audio encoder and decoder using a frequency domain processor, a time domain processor, and a cross processing for continuous initialization

Inventors: Sascha Disch (Fürth, DE); Martin Dietz (Nuremberg, DE); Markus Multrus (Nuremberg, DE); Guillaume Fuchs (Bubenreuth, DE); Emmanuel Ravelli (Erlangen, DE); Matthias Neusinger (Rohr, DE); Markus Schnell (Nuremberg, DE); Benjamin Schubert (Nuremberg, DE); Bernhard Grill (Rückersdorf, DE)
G10L19/0208G10L19/022G10L19/18G10L19/24G10L19/02G10L19/028G10L19/04G10L19/083G10L19/26G10L21/038G10L2019/0001
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,410,668
App. No.
16/290,587
Granted
Aug 9, 2022
Kind
B2
Abstract

An audio encoder for encoding an audio signal includes: a first encoding processor for encoding a first audio signal portion in a frequency domain, wherein the first encoding processor includes: a time frequency converter for converting the first audio signal portion into a frequency domain representation having spectral lines up to a maximum frequency of the first audio signal portion; a spectral encoder for encoding the frequency domain representation; a second encoding processor for encoding a second different audio signal portion in the time domain; a cross-processor for calculating, from the encoded spectral representation of the first audio signal portion, initialization data of the second encoding processor, so that the second encoding processing is initialized to encode the second audio signal portion immediately following the first audio signal portion in time in the audio signal; a controller configured for analyzing the audio signal and for determining, which portion of the audio signal is the first audio signal portion encoded in the frequency domain and which portion of the audio signal is the second audio signal portion encoded in the time domain; and an encoded signal former for forming an encoded audio signal including a first encoded signal portion for the first audio signal portion and a second encoded signal portion for the second audio signal portion.

Claims (129)

1. An audio encoder for encoding an audio signal, comprising:

a first encoding processor configured for encoding a first audio signal portion in a frequency domain, wherein the first encoding processor comprises:

a time-frequency converter configured for converting the first audio signal portion into a frequency domain representation comprising spectral lines up to a maximum frequency of the first audio signal portion; and

a spectral encoder configured for encoding the frequency domain representation;

a second encoding processor configured for encoding a second different audio signal portion in a time domain;

a cross-processor configured for calculating, from an encoded spectral representation of the first audio signal portion, initialization data of the second encoding processor, so that the second encoding processor is initialized to encode the second audio signal portion immediately following the first audio signal portion in time in the audio signal;

a controller configured for analyzing the audio signal and configured for determining, which portion of the audio signal is the first audio signal portion encoded in the frequency domain and which portion of the audio signal is the second audio signal portion encoded in the time domain; and

an encoded signal former configured for forming an encoded audio signal comprising a first encoded signal portion for the first audio signal portion and a second encoded signal portion for the second audio signal portion,

wherein the cross-processor comprises:

a noise shaper configured for shaping quantized spectral values of a frequency domain representation using LPC coefficients derived from the first audio signal portion;

a spectral decoder configured for decoding spectrally shaped spectral portions of the frequency domain representation with a high spectral resolution to acquire a decoded spectral representation; and

a frequency-time converter configured for converting the decoded spectral representation into the time domain to acquire a decoded first audio signal portion, wherein a sampling rate associated with the decoded first audio signal portion is different from a sampling rate of the audio signal, and a sampling rate associated with an output signal of the frequency-time converter is different from a sampling rate associated with the audio signal input into the time-frequency converter, or

wherein the second encoding processor comprises an associated second sampling rate, wherein the first encoding processor has associated therewith a first sampling rate being different from the second sampling rate, and wherein the cross-processor comprises a frequency-time converter configured for generating a time domain signal at the second sampling rate, wherein the frequency-time converter comprises:

a selector configured for selecting a portion of a spectrum input into the frequency-time converter in accordance with a ratio of the first sampling rate and the second sampling rate, a transform processor configured for using a transform length being different from a transform length used by the time-frequency converter; and

a synthesis windower configured for windowing using a window comprising a different number of window coefficients compared to a window used by the time-frequency converter.

2. The audio encoder of claim 1 , wherein the audio signal comprises a high band and a low band, and

wherein the second encoding processor comprises a sampling rate converter configured for converting the second audio signal portion to a lower sampling rate representation having a second sampling rate, the second sampling rate of the lower sampling rate representation being lower than a sampling rate of the audio signal, wherein the lower sampling rate representation does not comprise the high band of the audio signal; and

wherein the audio encoder further comprises:

a time domain low band encoder configured for time domain encoding the lower sampling rate representation; and

a time domain bandwidth extension encoder configured for parametrically encoding the high band.

3. The audio encoder of claim 1 , further comprising:

a preprocessor configured for preprocessing the first audio signal portion and the second audio signal portion, and

wherein the preprocessor comprises a prediction analyzer configured for determining prediction coefficients;

wherein the encoded signal former is configured for introducing an encoded version of the prediction coefficients into the encoded audio signal.

4. The audio encoder of claim 1 , comprising:

a preprocessor comprising a resampler configured for resampling the audio signal to a sampling rate of the second encoding processor; and

a prediction analyzer configured to determine the prediction coefficients using a resampled audio signal, or

wherein the preprocessor further comprises a long term prediction analysis stage configured for determining one or more long term prediction parameters for the first audio signal portion.

5. The audio encoder of claim 1 , wherein the cross-processor comprises:

a spectral decoder configured for calculating a decoded version of the first encoded signal portion;

a delay stage configured for feeding a delayed version of the decoded version into a de-emphasis stage of the second encoding processor for initialization;

a weighted prediction coefficient analysis filtering block configured for feeding a filter output into a codebook determiner of the second encoding processor for initialization;

an analysis filtering stage configured for filtering the decoded version or a pre-emphasized version and configured for feeding a filter residual into an adaptive codebook determiner of the second encoding processor for initialization; or

a pre-emphasis filter configured for filtering the decoded version and configured for feeding a delayed or pre-emphasized version to a synthesis filtering stage of the second encoding processor for initialization.

6. The audio encoder of claim 1 ,

wherein the second encoding processor comprises at least one element of the following group of elements:

a prediction analysis filter;

an adaptive codebook stage;

an innovative codebook stage;

an estimator configured for estimating an innovative codebook entry;

an ACELP/gain coding stage;

a prediction synthesis filtering stage;

a de-emphasis stage; and

a bass post-filter analysis stage.

7. An audio decoder for decoding an encoded audio signal, comprising:

a first decoding processor configured for decoding a first encoded audio signal portion in a frequency domain to obtain a decoded spectral representation, the first decoding processor comprising;

a frequency-time converter configured for converting the decoded spectral representation into a time domain to acquire a decoded first audio signal portion;

a second decoding processor configured for decoding a second encoded audio signal portion in the time domain to acquire a decoded second audio signal portion;

a cross-processor configured for calculating, from the decoded spectral representation of the first encoded audio signal portion, initialization data of the second decoding processor, so that the second decoding processor is initialized to decode the second encoded audio signal portion following in time the first encoded audio signal portion in the encoded audio signal; and

a combiner configured for combining the decoded first audio signal portion and the decoded second audio signal portion to acquire a decoded audio signal,

wherein the second decoding processor comprises:

a time domain low band decoder configured for decoding a low band time domain signal;

a resampler configured for resampling the low band time domain signal;

a time domain bandwidth extension decoder configured for synthesizing a high band of a time domain output signal; and

a mixer configured for mixing a synthesized high band of the time domain output signal and a resampled low band time domain signal, or

wherein the cross-processor comprises:

an additional frequency-time converter operating at a second sampling rate being different from a first sampling rate associated with the frequency-time converter of the first decoding processor to acquire a further decoded first signal portion in the time domain, wherein the further decoded first signal portion in the time domain comprises the second sampling rate being different from the first sampling rate associated with an output of the frequency-time converter of the second decoding processor, wherein the additional frequency-time converter comprises:

a selector configured for selecting a portion of a spectrum input into the additional frequency-time converter in accordance with a ratio of the first sampling rate and the second sampling rate;

a transform processor comprising a transform length being different from a transform length used by the frequency-time converter of the first decoding processor; and

a synthesis windower using a window comprising a different number of coefficients compared to a window used by the frequency-time converter of the first decoding processor.

8. The audio decoder of claim 7 ,

wherein the second decoding processor comprises at least one element of the group of elements comprising:

a stage configured for decoding ACELP gains and an innovative codebook;

an adaptive codebook synthesis stage;

an ACELP post-processor;

a prediction synthesis filter; and

a de-emphasis stage.

9. A method of encoding an audio signal, comprising:

encoding a first audio signal portion in a frequency domain, comprising:

converting the first audio signal portion into a frequency domain representation comprising spectral lines up to a maximum frequency of the first audio signal portion; and

encoding the frequency domain representation;

encoding a second different audio signal portion in a time domain;

calculating, from an encoded spectral representation of the first audio signal portion, initialization data for the step of encoding the second different audio signal portion, so that the step of encoding the second different audio signal portion is initialized to encode the second audio signal portion immediately following the first audio signal portion in time in the audio signal;

analyzing the audio signal and determining, which portion of the audio signal is the first audio signal portion encoded in the frequency domain and which portion of the audio signal is the second audio signal portion encoded in the time domain; and

forming an encoded audio signal comprising a first encoded signal portion for the first audio signal portion and a second encoded signal portion for the second audio signal portion,

wherein the calculating the initialization data comprises:

shaping quantized spectral values of the frequency domain representation using LPC coefficients derived from the first audio signal portion;

decoding spectrally shaped spectral portions of the frequency domain representation with a high spectral resolution to acquire a decoded spectral representation; and

converting the decoded spectral representation into the time domain to acquire a decoded first audio signal portion, wherein a sampling rate associated with the decoded first audio signal portion is different from a sampling rate of the audio signal, and a sampling rate associated with an output signal of the converting into the time domain is different from a sampling rate associated with the audio signal input into the converting into the frequency domain, or

wherein the encoding the second different audio signal portion in the time domain comprises an associated second sampling rate, wherein the encoding the first audio signal portion in the frequency domain has associated therewith a first sampling rate being different from the second sampling rate, and wherein the calculating the initialization data comprises generating a time domain signal at the second sampling rate, wherein the generating comprises:

selecting a portion of a spectrum input into the generating in accordance with a ratio of the first sampling rate and the second sampling rate, using a transform length being different from a transform length used by the converting; and

windowing using a window comprising a different number of window coefficients compared to a window used by the converting.

10. A method of decoding an encoded audio signal, comprising:

decoding a first encoded audio signal portion in a frequency domain to obtain a decoded spectral representation, the decoding the first encoded audio signal portion comprising:

converting a decoded spectral representation into a time domain to acquire a decoded first audio signal portion;

decoding a second encoded audio signal portion in the time domain to acquire a decoded second audio signal portion;

calculating, from the decoded spectral representation of the first encoded audio signal portion, initialization data of the step of decoding the second encoded audio signal portion, so that the step of decoding the second encoded audio signal portion is initialized to decode the second encoded audio signal portion following in time the first encoded audio signal portion in the encoded audio signal; and

combining the decoded first audio signal portion and the decoded second audio signal portion to acquire a decoded audio signal,

wherein the decoding the second encoded audio signal portion comprises:

decoding a low band time domain signal;

resampling the low band time domain signal;

synthesizing a high band of a time domain output signal; and

mixing a synthesized high band of the time domain output signal and a resampled low band time domain signal, or

wherein the calculating the initialization data comprises:

an additional frequency-time converting operating at a second sampling rate being different from a first sampling rate associated with the converting used in the decoding the first encoded audio signal portion to acquire a further decoded first signal portion in the time domain, wherein the further decoded first signal portion in the time domain comprises the second sampling rate being different from the first sampling rate associated with an output of the converting used in the decoding the second encoded audio signal portion, wherein the additional frequency-time converting comprises:

selecting a portion of a spectrum input into the additional frequency-time converting in accordance with a ratio of the first sampling rate and the second sampling rate;

transform processing comprising a transform length being different from a transform length used by the converting used in the decoding the first encoded audio signal portion; and

synthesis windowing using a window comprising a different number of coefficients compared to a window used by the converting used in the decoding the first encoded audio signal portion.

11. A non-transitory digital storage medium having a computer program stored thereon to perform, when said computer program is run by a computer, the method of encoding an audio signal, the method comprising:

encoding a first audio signal portion in a frequency domain, comprising:

converting the first audio signal portion into a frequency domain representation comprising spectral lines up to a maximum frequency of the first audio signal portion; and

encoding the frequency domain representation;

encoding a second different audio signal portion in the time domain;

calculating, from the encoded spectral representation of the first audio signal portion, initialization data for the step of encoding the second different audio signal portion, so that the step of encoding the second different audio signal portion is initialized to encode the second audio signal portion immediately following the first audio signal portion in time in the audio signal;

analyzing the audio signal and determining, which portion of the audio signal is the first audio signal portion encoded in the frequency domain and which portion of the audio signal is the second audio signal portion encoded in the time domain; and

forming an encoded audio signal comprising a first encoded signal portion for the first audio signal portion and a second encoded signal portion for the second audio signal portion,

wherein the calculating the initialization data comprises:

shaping quantized spectral values of the frequency domain representation using LPC coefficients derived from the first audio signal portion;

decoding spectrally shaped spectral portions of the frequency domain representation with a high spectral resolution to acquire a decoded spectral representation; and

converting the decoded spectral representation into the time domain to acquire a decoded first audio signal portion, wherein a sampling rate associated with the decoded first audio signal portion is different from a sampling rate of the audio signal, and a sampling rate associated with an output signal of the converting into the time domain is different from a sampling rate associated with the audio signal input into the converting into the frequency domain, or

wherein the encoding the second different audio signal portion in the time domain comprises an associated second sampling rate, wherein the encoding the first audio signal portion in the frequency domain has associated therewith a first sampling rate being different from the second sampling rate, and wherein the calculating the initialization data comprises generating a time domain signal at the second sampling rate, wherein the generating comprises:

selecting a portion of a spectrum input into the generating in accordance with a ratio of the first sampling rate and the second sampling rate, using a transform length being different from a transform length used by the converting; and

windowing using a window comprising a different number of window coefficients compared to a window used by the converting.

12. A non-transitory digital storage medium having a computer program stored thereon to perform, when said computer program is run by a computer, the method of decoding an encoded audio signal, the method comprising:

decoding a first encoded audio signal portion in a frequency domain to obtain a decoded spectral representation, the decoding comprising;

converting the decoded spectral representation into a time domain to acquire a decoded first audio signal portion;

decoding a second encoded audio signal portion in the time domain to acquire a decoded second audio signal portion;

calculating, from the decoded spectral representation of the first encoded audio signal portion, initialization data of the step of decoding the second encoded audio signal portion, so that the step of decoding the second encoded audio signal portion is initialized to decode the second encoded audio signal portion following in time the first encoded audio signal portion in the encoded audio signal; and

combining the decoded first audio signal portion and the decoded second audio signal portion to acquire a decoded audio signal,

wherein the decoding the second encoded audio signal portion comprises:

decoding a low band time domain signal;

resampling the low band time domain signal;

synthesizing a high band of a time domain output signal; and

mixing a synthesized high band of the time domain output signal and a resampled low band time domain signal, or

wherein the calculating the initialization data comprises:

an additional frequency-time converting operating at a second sampling rate being different from a first sampling rate associated with the converting used in the decoding the first encoded audio signal portion to acquire a further decoded first signal portion in the time domain, wherein the further decoded first signal portion in the time domain comprises the second sampling rate being different from the first sampling rate associated with an output of the converting used in the decoding the second encoded audio signal portion, wherein the additional frequency-time converting comprises:

selecting a portion of a spectrum input into the additional frequency-time converting in accordance with a ratio of the first sampling rate and the second sampling rate;

transform processing comprising a transform length being different from a transform length used by the converting used in the decoding the first encoded audio signal portion; and

synthesis windowing using a window comprising a different number of coefficients compared to a window used by the converting used in the decoding the first encoded audio signal portion.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 23, 2022
From: DISCH, SASCHA; DIETZ, MARTIN; MULTRUS, MARKUS; FUCHS, GUILLAUME; RAVELLI, EMMANUEL; NEUSINGER, MATTHIAS; SCHNELL, MARKUS; SCHUBERT, BENJAMIN; GRILL, BERNHARD
To: FRAUNHOFER-GESELLSCHAFT ZUR FÖRDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
Reel/Frame 061195/0410 →
Priority Claims (1)
EP 14178819 · Jul 28, 2014 · regional
Continuity (3)
Continuation 15414289 · Jan 24, 2017
Continuation PCTEP2015067005 · Jul 24, 2015
Related Publication 20190267016A1 · Aug 29, 2019
Cited By (1)
US 12,725,620