IP Library › Granted Patent US 12,738,286
Granted Patent B2
US 12,738,286 · App. 16/892,648 · Granted Sep 15, 2026

Post-processor, pre-processor, audio encoder, audio decoder and related methods for enhancing transient processing

Inventors: Florin Ghido (Nuremberg, DE); Sascha Disch (Fürth, DE); Jürgen Herre (Erlangen, DE); Alexander Adami (Gundelsheim, DE); Franz Reutelhuber (Erlangen, DE)
Assignee: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
G10L19/032G10L19/008G10L19/26H03G5/005H03G5/165
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,738,286
App. No.
16/892,648
Granted
Sep 15, 2026
Kind
B2
Abstract

An audio post-processor for post-processing an audio signal having a time-variable high frequency gain information as side information includes: a band extractor for extracting a high frequency band of the audio signal and a low frequency band of the audio signal; a high band processor for performing a time-variable modification of the high frequency band in accordance with the time-variable high frequency gain information to obtain a processed high frequency band; and a combiner for combining the processed high frequency band and the low frequency band. Furthermore, a pre-processor is illustrated.

Claims (42)

1 . An audio post-processor for post-processing an audio signal comprising a time-variable high frequency gain information representing a side information of the audio signal, comprising:

a band extractor configured for extracting a high frequency band of the audio signal to obtain an extracted high frequency band comprising a block of high pass time domain values and configured for extracting a low frequency band of the block of time domain values of the audio signal to obtain an extracted low frequency band comprising a block of low pass time domain values;

a high band processor configured for performing a time-variable amplification of only the extracted high frequency band in accordance with the time-variable high frequency gain information to acquire a processed high frequency band,

wherein the high band processor is configured to apply the time-variable amplification to each time domain value of the block of high pass time domain values using the high frequency gain information to obtain a block of modified high pass time domain values as the processed high frequency band; and

a combiner configured for performing a sample-wise addition of the block of modified high pass time domain values and the block of low pass time domain values unmodified by the time-variable high frequency gain information,

wherein the audio signal comprises an additional control parameter as a further side information, wherein the high band processor is configured to apply the time-variable amplification also under consideration of the additional control parameter, wherein a time resolution of the additional control parameter is lower than a time resolution of the time-varying frequency gain information.

2 . The audio post-processor of claim 1 , in which the band extractor is configured to extract the low frequency band using a low pass filter device and to extract the high frequency band by subtracting the extracted low frequency band from the block of time domain values of the audio signal.

3 . The audio post-processor of claim 1 , wherein the audio signal comprises a sequence of blocks of time domain values, the sequence of blocks comprising the block of time domain values and a further later block of time domain values,

wherein the time-variable high frequency gain information is provided for the sequence of blocks of time domain values of the audio signal,

wherein the block of time domain values has associated therewith a first gain information and a the further later block of time domain values of the audio signal has a different second gain information,

wherein the band extractor is configured to extract, from the further later block of time domain values, a further block of low pass time domain values and a further block of high pass time domain values, and

wherein the high band processor is configured to modify, in the applying the time-variable amplification, the block of high pass time domain values using the first gain information to acquire the processed block of high pass time domain values and to modify the further block of high pass time domain values using the second gain information to acquire a further processed block of high pass time domain values, and

wherein the combiner is configured to combine the further later block of time domain values and the further processed block of high pass time domain values.

4 . The audio post-processor of claim 1 ,

wherein the band extractor and the high band processor and the combiner are configured to operate in overlapping blocks, and

wherein the audio post-processor further comprises an overlap-adder configured for calculating time domain values of a post-processed portion by adding time domain values of a first combined block obtained by the sample-wise addition of the block of modified high pass time domain values and the block of low pass time domain values and time domain samples of a second combined block in a block overlap range.

5 . An audio decoding apparatus, comprising:

an input interface configured for receiving an encoded audio signal comprising a core encoded signal, core side information and a time-variable high frequency gain information representing an additional side information;

a core decoder configured for decoding the core encoded signal using the core side information to acquire a decoded core signal; and

a post-processor configured for post-processing the decoded core signal using the time-variable high frequency gain information, the post-processor comprising: a band extractor configured for extracting a high frequency band of a block of time domain values of the audio signal to obtain an extracted high frequency band comprising a block of high pass time domain values and configured for extracting a low frequency band of the block of time domain values of the audio signal to obtain an extracted low frequency band comprising a block of low pass time domain values;

a high band processor configured for performing a time-variable amplification of only the extracted high frequency band in accordance with the time-variable high frequency gain information to acquire a processed high frequency band, wherein the high band processor is configured to apply the time-variable amplification to each time domain value of the block of high pass time domain values using the high frequency gain information to obtain a block of modified high pass time domain values as the processed high frequency band; and a combiner configured for performing a sample-wise addition of the block of modified high pass time domain values and the block of low pass time domain values unmodified by the time-variable high frequency gain information,

wherein the encoded audio signal comprises an additional control parameter as a further side information, wherein the high band processor is configured to apply the time-variable amplification also under consideration of the additional control parameter, wherein a time resolution of the additional control parameter is lower than a time resolution of the time-varying high frequency gain information.

6 . A method of post-processing an audio signal comprising a time-variable high frequency gain information representing a side information of the audio signal, comprising:

extracting a high frequency band of a block of time domain values of the audio signal to obtain an extracted high frequency band comprising a block of high pass time domain values and extracting a low frequency band of the block of time domain values of the audio signal to obtain an extracted low frequency band comprising a block of low pass time domain values;

performing a time-variable amplification of only the extracted high frequency band in accordance with the time-variable high frequency gain information to acquire a processed high frequency band, wherein the performing the time-variable amplification comprises applying the time-variable amplification to each time domain value of the block of high pass time domain values using the high frequency gain information to obtain a block of modified high pass time domain values as the processed high frequency band; and

performing a sample-wise addition of the block of modified high pass time domain values and the block of low pass time domain values unmodified by the time-variable high frequency gain information,

wherein the audio signal comprises an additional control parameter as a further side information, wherein the performing a time-variable amplification comprises applying the time-variable amplification also under consideration of the additional control parameter, where in a time resolution of the additional control parameter is lower than a time resolution of the time-varying high frequency gain information.

7 . A method of audio decoding, comprising:

receiving an encoded audio signal comprising a core encoded signal, core side information and a time-variable high frequency gain information representing an additional side information of the core encoded signal;

decoding the core encoded signal using the core side information to acquire a decoded core signal; and

post-processing the decoded core signal using the time-variable high frequency gain information representing the additional side information with a method of post-processing the method of post-processing comprising: extracting a high frequency band of a block of time domain values of the decoded core signal to obtain an extracted high frequency band comprising a block of high pass time domain values and extracting a high frequency band of the block of time domain values of the audio signal to obtain an extracted low frequency band of the comprising a block of low pass time domain values; performing a time-variable amplification of only the extracted high frequency in accordance with the time-variable high frequency gain information to acquire a processed high frequency band, wherein the performing the time-variable amplification comprises applying the time-variable amplification to each time domain value of the block of high pass time domain values using the high frequency gain information to obtain a block of modified high pass time domain values as the processed high frequency band; and performing a sample-wise addition of the block of modified high pass time domain values and the block of low pass time domain values unmodified by the time-variable high frequency gain information,

wherein the encoded audio signal comprises an additional control parameter as a further side information, wherein the performing a time-variable amplification comprises applying the time-variable amplification also under consideration of the additional control parameter, wherein a time resolution of the additional control parameter is lower than a time resolution of the time-varying high frequency gain.

8 . A non-transitory digital storage medium having a computer program stored thereon to perform, when said computer program is run by a computer, a method of post-processing an audio signal comprising a time-variable high frequency gain information representing side information of the audio signal, comprising:

extracting a high frequency band of a block of time domain values of the audio signal to obtain an extracted high frequency band comprising a block of high pass time domain values and extracting a low frequency band of the block of time domain values of the audio signal to obtain an extracted low frequency band comprising a block of low pass time domain values;

performing a time-variable amplification of only the extracted high frequency band in accordance with the time-variable high frequency gain to acquire a processed high frequency band, wherein the performing the time-variable amplification comprises applying the time-variable amplification to each time domain value of the block of high pass time domain values using the high frequency gain information to obtain a block of modified high pass time domain values as the processed high frequency band; and

performing a sample-wise addition of the block of modified high pass time domain values and the block of low pass time domain values unmodified by the time-variable high frequency gain information,

wherein the audio signal comprises an additional control parameter as a further side information, wherein the performing a time-variable amplification comprises applying the time-variable amplification also under consideration of the additional control parameter, wherein a time resolution of the additional control parameter is lower than a time resolution of the time-varying high frequency gain information.

9 . A non-transitory digital storage medium having a computer program stored thereon to perform, when said computer program is run by a computer, a method of audio decoding, the method of audio decoding comprising:

receiving an encoded audio signal comprising a core encoded signal, core side information and a time-variable high frequency gain information representing additional side information of the core encoded signal;

decoding the core encoded signal using the core side information to acquire a decoded core signal; and

post-processing the decoded core signal using the time-variable high frequency gain information in accordance with a method of post-processing the method of post-processing comprising: extracting a high frequency band of a block of time domain values of the audio signal to obtain an extracted high frequency band comprising a block of high pass time domain values and extracting a low frequency band of the block of time domain values of the audio signal to obtain an extracted low frequency band comprising a block of low pass time domain values; performing a time-variable amplification of only the extracted high frequency band in accordance with the time-variable high frequency gain information to acquire a processed high frequency band, wherein the performing the time-variable amplification comprises applying the time-variable amplification to each time domain value of the block of high pass time domain values using the high frequency gain information to obtain a block of modified high pass time domain values as the processed high frequency band; and performing a sample-wise addition of the block of low pass time domain values and the block of low pass time domain values unmodified by the time variable high frequency gain information,

wherein the encoded audio signal comprises an additional control parameter as a further side information, wherein the performing a time-variable amplification comprises applying the time-variable amplification also under consideration of the additional control parameter, wherein a time resolution of the additional control parameter is lower than a time resolution of the time-varying high frequency gain information.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 27, 2020
From: GHIDO, FLORIN; DISCH, SASCHA; HERRE, JUERGEN; ADAMI, ALEXANDER; REUTELHUBER, FRANZ
To: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
Reel/Frame 053615/0960 →
Priority Claims (1)
EP 16156200 · Feb 17, 2016 · regional
Continuity (4)
Continuation 16688938 · Nov 19, 2019
Division 15884190 · Jan 30, 2018
Continuation PCTEP2017053068 · Feb 10, 2017
Related Publication 20200402520A1 · Dec 24, 2020
References Cited (98)
US 4292467A · Odlen et al. · 1981 [cited by applicant]
US 5222189A · Fielder · 1993 [cited by applicant]
US 5978762A · Smyth · 1999 [cited by examiner]
US 6014621A · Chen · 2000 [cited by applicant]
US 7072477B1 · Kincaid · 2006 [cited by examiner]
US 7272556B1 · Aguilar · 2007 [cited by examiner]
US 7707034B2 · Sun et al. · 2010 [cited by applicant]
US 7720230B2 · Allamanche et al. · 2010 [cited by applicant]
US 7974713B2 · Disch et al. · 2011 [cited by applicant]
US 7983424B2 · Kjorling et al. · 2011 [cited by applicant]
US 8116459B2 · Disch et al. · 2012 [cited by applicant]
US 8140331B2 · Lou · 2012 [cited by applicant]
US 8204261B2 · Allamanche et al. · 2012 [cited by applicant]
US 8532998B2 · Gao · 2013 [cited by examiner]
US 8942988B2 · Gao · 2015 [cited by examiner]
US 9026452B2 · Koishida · 2015 [cited by examiner]
US 9812136B2 · Kjoerling · 2017 [cited by examiner]
US 10720170B2 · Ghido · 2020 [cited by examiner]
US 20050254719A1 · Sullivan · 2005 [cited by applicant]
US 20070150267A1 · Honma et al. · 2007 [cited by applicant]
US 20070253563A1 · Oxford et al. · 2007 [cited by applicant]
US 20080147415A1 · Schnell et al. · 2008 [cited by applicant]
US 20080154615A1 · Oomen et al. · 2008 [cited by applicant]
US 20080300866A1 · Mukhtar et al. · 2008 [cited by applicant]
US 20090103752A1 · Chou · 2009 [cited by examiner]
US 20090313029A1 · Luo · 2009 [cited by examiner]
US 20100262427A1 · Chivukula et al. · 2010 [cited by applicant]
US 20110202353A1 · Neuendorf et al. · 2011 [cited by applicant]
US 20110257979A1 · Gao · 2011 [cited by applicant]
US 20110257980A1 · Gao · 2011 [cited by examiner]
US 20110257984A1 · Virette et al. · 2011 [cited by applicant]
US 20110295598A1 · Yang et al. · 2011 [cited by applicant]
US 20120016667A1 · Gao · 2012 [cited by applicant]
US 20130275142A1 · Hatanaka et al. · 2013 [cited by applicant]
US 20140229170A1 · Atti et al. · 2014 [cited by applicant]
US 20150088527A1 · Naslund et al. · 2015 [cited by applicant]
US 20150215700A1 · Sun et al. · 2015 [cited by applicant]
US 20150279379A1 · Truman et al. · 2015 [cited by applicant]
US 20160019898A1 · Schreiner et al. · 2016 [cited by applicant]
US 20160019908A1 · Hedelin · 2016 [cited by examiner]
CN 1275234A · 2000 [cited by applicant]
CN 1992533A · 2007 [cited by applicant]
CN 101485094A · 2009 [cited by applicant]
CN 102257728A · 2011 [cited by applicant]
CN 102985970A · 2013 [cited by applicant]
CN 103210443A · 2013 [cited by applicant]
CN 103366751A · 2013 [cited by applicant]
CN 104269173A · 2015 [cited by applicant]
CN 104517610A · 2015 [cited by applicant]
CN 105122358A · 2015 [cited by applicant]
EP 2116997A1 · 2009 [cited by applicant]
EP 2352225A1 · 2011 [cited by applicant]
GB 2293733A · 1996 [cited by applicant]
JP 2000132193A · 2000 [cited by applicant]
JP 2008536169A · 2008 [cited by applicant]
JP WO2008108082A1 · 2010 [cited by applicant]
JP 2013512468A · 2013 [cited by applicant]
JP 6603414B2 · 2019 [cited by applicant]
KR 20070068270A · 2007 [cited by applicant]
WO WO2009056027A1 · 2009 [cited by applicant]
WO WO2011134415A1 · 2011 [cited by applicant]
WO WO2012111767A1 · 2012 [cited by applicant]
WO WO2014111290A1 · 2014 [cited by applicant]
WO WO2014161996A2 · 2014 [cited by applicant]
WO WO2015077665A1 · 2015 [cited by applicant]
WO WO2015118260A1 · 2015 [cited by applicant]
Office Action dated Sep. 23, 2021 issued in the parallel Chinese patent application No. 201780002163.5 (7 pages). [cited by applicant]
Yong-tao Sha et al., High Frequency Reconstruction of Audio Signal Based on Chaotic Prediction Theory, Acoustics, Speech, and Signal Processing, 1988. ICASSP-88., 1988 International Conference on Apr. 2010. [cited by applicant]
Office Action dated Nov. 6, 2020 issued in related U.S. Appl. No. 16/688,938 (20 pages). [cited by applicant]
Office Action in the parallel Korean patent application No. 10-2017-7036732 dated Feb. 12, 2019 (18 pages). [cited by applicant]
Office Action dated Sep. 2, 2020 issued in the parallel Chinese patent application No. 201780002163.5. [cited by applicant]
Office Action dated May 29, 2020 in the parallel Indian patent application No. 201747038260 (5 pages). [cited by applicant]
Office Action dated Feb. 2, 2021 issued in the parallel Japanese patent application No. 2019-186928 (8 pages). [cited by applicant]
M. Bosi, K. Brandenburg, S. Quackenbush, L. Fielder, K. Akagiri, H. Fuchs, M. Dietz, J. Herre, G. Davidson, Oikawa: “MPEG-2 Advanced Audio Coding”, 101st AES Convention, Los Angeles 1996. [cited by applicant]
K. Brandenburg: “OCF—A New Coding Algorithm for High Quality Sound Signals”, Proc. IEEE ICASSP, 1987. [cited by applicant]
J. D. Johnston, K. Brandenburg: “Wideband Coding Perceptual Considerations for Speech and Music”, in S. Furui and M. M. Sondhi, editors: “Advances in Speech Signal Processing”, Marcel Dekker, New York, 1992. [cited by applicant]
B. Edler: “Codierung von Audiosignalen mit überlappender Transformation und adaptiven Fensterfunktionen”, Frequenz, vol. 43, pp. 252-256, 1989 (English abstract attached). [cited by applicant]
J. Herre, J. D. Johnston: “Enhancing the Performance of Perceptual Audio Coders by Using Temporal Noise Shaping (TNS)”, 101st AES Convention, Los Angeles 1996, Preprint 4384. [cited by applicant]
Gerard Hotho, Steven van de Par, and Jeroen Breebaart: “Multichannel coding of applause signals”, EURASIP Journal of Advances in Signal Processing, Hindawi, Jan. 2008, doi: 10.1155/2008/531693. [cited by applicant]
M. Link: “An Attack Processing of Audio Signals for Optimizing the Temporal Characteristics of a Low Bit-Rate Audio Coding System”, 95th AES convention, New York 1993, Preprint 3696. [cited by applicant]
B. C. J. Moore: “An Introduction to the Psychology of Hearing”, Academic Press, London, 1989. [cited by applicant]
ISO/IEC JTC1/SC29/WG11 MPEG, International Standard ISO 11172-3 “Coding of moving pictures and associated audio for digital storage media at up to about 1.5 Mbit/s”, Part 3: Audio, 1993. [cited by applicant]
T. Vaupel: “Ein Beitrag zur Transformationscodierung von Audiosignalen unter Verwendung der Methode der ‘Time Domain Aliasing Cancellation (TDAC)’ und einer Signalkompandierung im Zeitbereich”, PhD Thesis, Universität-G… [cited by applicant]
ISO/IEC DIS 23008-3, 3D Audio—Information technology—High efficiency coding and media delivery in heterogeneous environments—Part 3: 3D audio, 2015. [cited by applicant]
ISO/IEC DIS 23003-3, USAC—Information technology—MPEG audio technologies—Part 3: Unified speech and audio coding, 2011. [cited by applicant]
ISO/IEC DIS 14496-3, AAC—Information technology—Coding of audio-visual objects—Part 3: Audio, 2009. [cited by applicant]
ETRI Crosscheck Results for Fraunhofer IIS CE, ISO/IEC JTC1/SC29/WG11 MPEG2016/M37833, San Diego, USA, Feb. 2016 (Seungkwon Beack, Tae-jin Lee, ETRI crosscheck report for CE on High Resolution Envelope Processing), Feb.… [cited by applicant]
IDMT Crosscheck Results for Fraunhofer IIS CE, ISO/IEC JTC1/SC29/WG11 MPEG2016/M37715, San Diego, USA, Feb. 2016 (Judith Liebetrau, Thomas Sporer, Alexander Stojanow, Cross Check Report for CE on HREP (Test Site Fraunho… [cited by applicant]
Office Action dated May 14, 2019 issued in the parallel Japanese patent application No. 2018-527783 (9 pages with English translation). [cited by applicant]
Office Action dated Nov. 13, 2024 in related Brazilian patent application No. BR122024012459-9 (13 pages). [cited by applicant]
Office Action dated Nov. 13, 2024 in related Brazilian patent application No. BR122024012461-0 (10 pages). [cited by applicant]
Office Action dated Nov. 13, 2024 in related Brazilian patent application No. BR122024012453-0 (13 pages). [cited by applicant]
Office Action dated Nov. 12, 2024 in related Brazilian patent application No. BR122024012456-4 (13 pages). [cited by applicant]
Office action dated Mar. 13, 2026 issued in related India patent appl. No. 202448021315 (9 pages). [cited by applicant]
Office action dated Mar. 13, 2026 issued in related India patent appl. No. 202448021313 (12 pages). [cited by applicant]
Office action dated Mar. 13, 2026 issued in related India patent appl. No. 202448021317 (11 pages). [cited by applicant]
Office action dated Mar. 13, 2026 issued in related India patent appl. No. 202448021311 (8 pages). [cited by applicant]
Office action dated Mar. 13, 2026 issued in related India patent appl. No. 202448021314 (8 pages). [cited by applicant]