Granted Patent B2

US 12,475,907 · App. 18/014,953 · Granted Nov 18, 2025

Inventors: Niels Martinus Philippe Neumann (Tilburg, NL); John Gerard Beerends (Hengstdijk, NL)

Assignee: Nederlandse Organisatie voor toegepast-natuurwetenschappelijk onderzoek TNO

G10L21/0232G10L25/18G10L25/21G10L25/45G10L25/60G10L25/69G10L2021/02082

Abstract

Claims (63)

1 . A method of determining a perceptual impact of an amount of echo or reverberation in a degraded audio signal on a perceived quality thereof, wherein the degraded audio signal is received from an audio transmission system, wherein the degraded audio signal is obtained by conveying through the audio transmission system a reference audio signal so as to provide the degraded audio signal, the method comprising:

obtaining, by a controller, at least one degraded digital audio sample from the degraded audio signal and at least one reference digital audio sample from the reference audio signal;

determining, by the controller, based on the at least one degraded digital audio sample and the at least one reference digital audio sample, a local impulse response signal;

determining, by the controller, a local energy time curve based on the local impulse response signal, wherein the local energy time curve is proportional to a square root of an absolute value of the local impulse response signal; and

identifying one or more peaks in the local energy time curve, the one or more peaks in time occurring at a delay in the local energy time curve after an onset of the local energy time curve based on the local impulse response signal, and determining an estimate of the amount of echo or reverberation based on an amount of energy in the one or more peaks;

wherein the obtaining the at least one degraded digital audio sample comprises sampling the degraded audio signal in a time domain fraction, the sampling including performing a windowing operation on the degraded audio signal by multiplying the degraded audio signal with a window function so as to yield the at least one degraded digital audio sample;

wherein the obtaining the at least one reference digital audio sample comprises sampling the reference audio signal in the time domain fraction, the sampling including performing a windowing operation on the reference audio signal by multiplying the reference audio signal with the window function so as to yield the at least one reference digital audio sample; and

wherein the window function, used for obtaining the at least one reference digital audio sample and the at least one degraded digital audio sample, has a non-zero value in the time domain fraction to be sampled and a zero value outside the time domain fraction.

2 . The method according to claim 1 , wherein the obtaining the at least one degraded digital audio sample comprises obtaining a plurality of degraded digital audio samples from the degraded audio signal, wherein each sample of the plurality of degraded digital audio samples is obtained by performing the windowing operation, and wherein the time domain fractions of at least two sequential degraded digital audio samples of the plurality of degraded digital audio samples are overlapping.

3 . The method according to claim 2 , wherein an overlap between the at least two sequential degraded digital audio samples is within a range of 10% to 90% overlap between the time domain fractions.

4 . The method according to claim 1 , wherein the window function is a function taken from the group consisting of: a Hamming window, a Von Hann window, a Tukey window, a cosine window, a rectangular window, a B-spline window, a triangular window, a Bartlett window, a Parzen window, a Welch window, a n th power-of-cosine window wherein n>1, a Kaiser window, a Nuttall window, a Blackman window, a Blackman Harris window, a Blackman Nuttall window, and a Flattop window.

5 . The method according to claim 1 , wherein the determining the estimate of the amount of echo or reverberation includes weighing the amount of energy in each peak based on the magnitude of each peak or a delay position of each peak along the time-axis.

6 . The method according to claim 1 , wherein the method additionally comprises:

obtaining, by the controller, a degraded digital signal representing at least a part of the degraded audio signal and having a duration longer than the time domain fraction of the at least one degraded digital audio sample;

obtaining, by the controller, a reference digital signal representing at least a part of the reference audio signal and having a duration longer than the time domain fraction of the at least one reference digital audio sample;

determining, by the controller, based on the at least one degraded digital signal and the at least one reference digital signal, a global impulse response signal;

determining, by the controller, a global energy time curve based on the global impulse response signal, wherein the global energy time curve is proportional to a square root of an absolute value of the global impulse response signal; and

identifying one or more further peaks in the global energy time curve, the one or more further peaks in time occurring at a delay in the global energy time curve after an onset of the global energy time curve based on the global impulse response signal, and determining a further estimate of the amount of echo or reverberation based on an amount of energy in the one or more further peaks.

7 . The method according to claim 6 , wherein the determining the further estimate of the amount of echo or reverberation includes weighing the amount of energy in each peak based on the magnitude of each further peak or a delay position of each further peak along the time-axis.

8 . The method according to claim 1 , further comprising:

calculating, by the controller, a partial reverb indicator value based on the estimated amount of echo or reverberation obtained from the at least one degraded digital audio sample and the at least one reference digital audio sample.

9 . The method according to claim 6 , wherein the determining the local impulse response signal based on a plurality of degraded digital audio samples or the global impulse response signal based on the digital signals, comprises:

converting, by the controller, the plurality of degraded digital audio samples or the digital signals from a time domain into a frequency domain by applying a Fourier transform to the plurality of degraded digital audio samples or digital signals;

determining, by the controller, a transfer function from a power spectrum signal from the plurality of degraded digital audio samples or the digital signals in the frequency domain; and

converting, by the controller, the power spectrum signal from the frequency domain into the time domain so as to yield the local impulse response signal or the global impulse response signal.

10 . The method according to claim 6 , further comprising at least one operation taken from the group consisting of:

calculating, by the controller, a global reverb indicator value based on the further estimate of the amount of echo or reverberation; and

calculating, by the controller and as far as dependent on claim 6 , a final reverb indicator value based on the estimate of the amount of echo or reverberation and the further estimate of the amount of echo or reverberation.

11 . The method according to claim 1 , wherein the determining the local impulse response signal comprises using a weighting factor that gives a lower weight to degraded samples earlier in a window if a speech signal of the corresponding reference samples are below a threshold, indicating a perceptually silent interval.

12 . A method of evaluating quality or intelligibility of a degraded speech signal received from an audio transmission system, by conveying through the audio transmission system a reference speech signal so as to provide the degraded speech signal, wherein the method comprises:

sampling the reference speech signal into a plurality of reference signal frames, sampling the degraded speech signal into a plurality of degraded signal frames, and forming frame pairs by associating the reference signal frames and the degraded signal frames with each other;

providing for each frame pair a difference function representing a difference between the degraded signal frame and the associated reference signal frame;

compensating the difference function for one or more disturbance types so as to provide for each frame pair a disturbance density function which is adapted to a human auditory perception model;

deriving from the disturbance density functions of a plurality of frame pairs an overall quality parameter, the quality parameter being at least indicative of the quality or intelligibility of the degraded speech signal; and

determining an amount of reverberation in at least one of the degraded speech signal and the reference speech signal,

wherein the amount of reverberation is determined by applying a method of determining a perceptual impact of an amount of echo or reverberation in a degraded audio signal on a perceived quality thereof, wherein the degraded audio signal is received from the audio transmission system wherein the degraded audio signal is obtained by conveying through the audio transmission system a reference audio signal so as to provide the degraded audio signal, the method comprising:

obtaining, by a controller, at least one degraded digital audio sample from the degraded audio signal and at least one reference digital audio sample from the reference audio signal;

determining, by the controller, based on the at least one degraded digital audio sample and the at least one reference digital audio sample, a local impulse response signal;

determining, by the controller, a local energy time curve based on the local impulse response signal, wherein the local energy time curve is proportional to a square root of an absolute value of the local impulse response signal; and

identifying one or more peaks in the local energy time curve, the one or more peaks in time occurring at a delay in the local energy time curve after an onset of the local energy time curve based on the local impulse response signal, and determining an estimate of the amount of echo or reverberation based on an amount of energy in the one or more peaks;

wherein the obtaining the at least one degraded digital audio sample comprises sampling the degraded audio signal in a time domain fraction, the sampling including performing a windowing operation on the degraded audio signal by multiplying the degraded audio signal with a window function so as to yield the at least one degraded digital audio sample;

wherein the obtaining the at least one reference digital audio sample comprises sampling the reference audio signal in the time domain fraction, the sampling including performing a windowing operation on the reference audio signal by multiplying the reference audio signal with the window function so as to yield the at least one reference digital audio sample; and

wherein the window function, used for obtaining the at least one reference digital audio sample and the at least one degraded digital audio sample, has a non-zero value in the time domain fraction to be sampled and a zero value outside the time domain fraction.

13 . The method according to claim 12 , wherein the method is performed on the degraded speech signal within a predetermined frequency range below a threshold frequency or a frequency range corresponding with speech signals.

14 . The method of claim 13 , wherein the frequency range is below 5 kilohertz.

15 . The method of claim 13 , wherein the frequency range is between 2 kilohertz and 4 kilohertz.

16 . The method according to claim 12 , further comprising, prior to the determining the local impulse response signal, a noise suppression comprising:

performing a first scaling of at least one of the degraded speech signal or the reference speech signal so as to obtain a similar average volume;

processing the degraded speech signal for removing one or more of local signal peaks, clippings and signal losses therefrom; and

performing a second scaling of at least one of the degraded speech signal or the reference speech signal so as to obtain a similar average volume.

17 . The method according to claim 12 , wherein the obtaining, by the controller, the at least one degraded digital audio sample and the at least one reference digital audio sample is performed by forming the degraded and reference audio samples from a plurality of consecutive signal frames, the plurality of consecutive signal frames including one or more of the degraded signal frames and one or more of the reference signal frames.

18 . The method according to claim 17 , wherein the number of signal frames to be included in the plurality of consecutive signal frames is dependent on the duration of the time domain fraction of the at least one digital audio sample, wherein the duration is larger than 0.3 seconds.

19 . The method according to claim 17 , wherein for each frame pair, the compensating is performed by:

setting the determined amount of reverberation in the at least one of the degraded speech signal and the reference speech signal as one of the one or more disturbance types, and

compensating each frame pair for the determined amount of reverberation associated with the respective frame pair based on the forming of the digital audio sample.

20 . A non-transitory computer-readable medium comprising instructions that, when loaded into a memory of a computer system and processed by a controller of the computer system, cause the computer system to perform a method of determining a perceptual impact of an amount of echo or reverberation in a degraded audio signal on a perceived quality thereof, wherein the degraded audio signal is received from an audio transmission system, wherein the degraded audio signal is obtained by conveying through the audio transmission system a reference audio signal so as to provide the degraded audio signal, the method comprising:

obtaining, by a controller, at least one degraded digital audio sample from the degraded audio signal and at least one reference digital audio sample from the reference audio signal;

determining, by the controller, based on the at least one degraded digital audio sample and the at least one reference digital audio sample, a local impulse response signal;

determining, by the controller, a local energy time curve based on the local impulse response signal, wherein the local energy time curve is proportional to a square root of an absolute value of the local impulse response signal; and

identifying one or more peaks in the local energy time curve, the one or more peaks in time occurring at a delay in the local energy time curve after an onset of the local energy time curve based on the local impulse response signal, and determining an estimate of the amount of echo or reverberation based on an amount of energy in the one or more peaks;

wherein the obtaining the at least one degraded digital audio sample comprises sampling the degraded audio signal in a time domain fraction, the sampling including performing a windowing operation on the degraded audio signal by multiplying the degraded audio signal with a window function so as to yield the at least one degraded digital audio sample;

wherein the obtaining the at least one reference digital audio sample comprises sampling the reference audio signal in the time domain fraction, the sampling including performing a windowing operation on the reference audio signal by multiplying the reference audio signal with the window function so as to yield the at least one reference digital audio sample; and

wherein the window function, used for obtaining the at least one reference digital audio sample and the at least one degraded digital audio sample, has a non-zero value in the time domain fraction to be sampled and a zero value outside the time domain fraction.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (17)