IP Library › Granted Patent US 10,319,390
Granted Patent B2
US 10,319,390 · App. 15/703,721 · Granted Jun 11, 2019

Method and system for multi-talker babble noise reduction

Inventors: Roozbeh Soleymani (New York, NY); Ivan W. Selesnick (Maplewood, NJ); David M. Landsberger (Forest Hills, NY)
Assignee: New York University
G10L21/0205G10L19/0216G10L21/0208G10L2021/02087
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,319,390
App. No.
15/703,721
Granted
Jun 11, 2019
Kind
B2
Abstract

A system and method for improving intelligibility of speech is provided. The system and method may include obtaining an input audio signal frame, classifying the input audio signal frame into a first category or a second category, wherein the first category corresponds to the noise being stronger than the speech signal, and the second category corresponds to the speech signal being stronger than the noise, decomposing the input audio signal frame into a plurality of sub-band components; de-noising each sub-band component of the input audio signal frame in parallel by applying a first wavelet de-noising method including a first wavelet transform and a predetermined threshold for the sub-band component, and a second wavelet de-noising method including a second wavelet transform and the predetermined threshold for the sub-band component, wherein the predetermined threshold for each sub-band component is based on at least one previous noise-dominant signal frame received by the receiving arrangement.

Claims (41)

1. A method for reduction of noise, comprising:

receiving from a receiving arrangement an input audio signal frame comprising a speech signal and a noise;

classifying the input audio signal frame into a first category or a second category, wherein the first category corresponds to the noise being stronger than the speech signal, and the second category corresponds to the speech signal being stronger than the noise;

decomposing the input audio signal frame into a plurality of sub-band components; and

de-noising each sub-band component of the input audio signal frame in parallel by applying a first wavelet de-noising method including a first wavelet transform and a predetermined threshold for the sub-band component, and a second wavelet de-noising method including a second wavelet transform and the predetermined threshold for the sub-band component, wherein the predetermined threshold for each sub-band component is based on at least one previous noise-dominant signal frame received by the receiving arrangement,

wherein the first and second wavelet transforms are configured to more aggressively de-noise the input audio signal frame when the input audio signal frame is classified in the first category as compared to when the input audio signal frame is classified in the second category, and

wherein the classifying step comprises:

applying a principle component analysis using a plurality of features, wherein the plurality of features includes at least one of: (1) an envelope variance feature of the input audio signal frame; (2) an envelope mean crossing feature of the input audio signal frame; (3) a root mean square feature of the input audio signal frame as compared to a predetermined threshold value; and (4) an entropy feature of a histogram of the input audio signal frame; and

classifying the input audio signal frame into the first category when the applying step identifies predominantly noise from the input audio signal frame, and into the second category when the principle component analysis identifies predominantly speech signal from the input audio signal frame.

2. The method of claim 1 , wherein the noise comprises a multi-talker babble noise.

3. The method of claim 1 , wherein the input audio signal frame is less than 100 ms in duration.

4. The method of claim 1 , wherein each of the plurality of features is each weighted differently in the principle component analysis.

5. The method of claim 1 , wherein the predetermined threshold value for the root mean square feature is based on a previous audio signal frame received by the receiving arrangement.

6. The method of claim 5 , wherein the previous audio signal frame includes predominantly noise.

7. The method of claim 1 , further comprising:

adjusting the plurality of features based on the input audio signal by an iterative method using a Gaussian mixture model for a plurality of sub-categories, wherein the first and second categories are each further divided into the plurality of sub-categories.

8. The method of claim 1 , wherein the first and second wavelet transforms are Tunable Q-Factor Wavelet Transforms (TQWTs).

9. The method of claim 1 , wherein the first and second wavelet transforms are selected based whether the input audio signal is classified into the first category or the second category.

10. The method of claim 1 , wherein each sub-band component comprises a plurality of wavelet coefficients corresponding to an amplitude of the sub-band component of the input audio signal frame.

11. The method of claim 10 , wherein the predetermined threshold for each sub-band component is selected based on an amount of noise present in the sub-band component, and an energy level represented by the plurality of wavelet coefficients.

12. The method of claim 1 , further comprising:

enhancing an output signal by filtering each de-noised sub-band component with a low pass filter.

13. A non-transitory computer readable medium storing a computer program that is executable by at least one processing unit, the computer program comprising sets of instructions, when executed by the processor, causing the processor to perform steps of claim 1 .

14. A method for improving intelligibility of speech, comprising:

obtaining, from a receiving arrangement, an input audio signal frame comprising a speech signal and a noise;

classifying the input audio signal into a first category or a second category, wherein the first category corresponds to the noise being stronger than the speech signal, and the second category corresponds to the speech signal being stronger than the noise;

decomposing the input audio signal frame into a plurality of sub-band components; and

de-noising each sub-band component of the input audio signal frame in parallel by applying a plurality of wavelet de-noising methods, each wavelet de-noising method including a wavelet transform and a predetermined threshold for the sub-band component, wherein the predetermined threshold for each sub-band component is based on at least one previous noise-dominant signal frame received by the receiving arrangement,

wherein each of the wavelet transforms is different from other wavelet transforms, and is each configured to more aggressively de-noise the input audio signal frame when the input audio signal frame is classified in the first category as compared to when the input audio signal frame is classified in the second category,

wherein the classifying step comprises:

applying a principle component analysis using a plurality of features, wherein the plurality of features includes at least one of: (1) an envelope variance feature of the input audio signal frame; (2) an envelope mean crossing feature of the input audio signal frame; (3) a root mean square feature of the input audio signal frame as compared to a predetermined threshold value; and (4) an entropy feature of a histogram of the input audio signal frame; and

classifying the input audio signal frame into the first category when the applying step identifies predominantly noise from the input audio signal frame, and into the second category when the principle component analysis identifies predominantly speech signal from the input audio signal frame.

15. The method of claim 14 , wherein the noise comprises a multi-talker babble noise.

16. The method of claim 14 , wherein the input audio signal frame is less than 100 ms in duration.

17. The method of claim 14 , wherein the wavelet transforms are Tunable Q-Factor Wavelet Transforms (TQWTs).

18. A system for improving intelligibility for a user comprising:

a receiving arrangement configured to receive an input audio signal frame comprising a speech signal and a noise;

a processing arrangement configured to receive the input audio signal frame from the receiving arrangement, classify the input audio signal frame into a first category or a second category, wherein the first category corresponds to the noise being stronger than the speech signal, and the second category corresponds to the speech signal being stronger than the noise, decompose the input audio signal frame into a plurality of sub-band components, and de-noise each sub-band component of the input audio signal frame in parallel by applying a first wavelet de-noising method including a first wavelet transform and a predetermined threshold for the sub-band component, and a second wavelet de-noising method including a second wavelet transform and the predetermined threshold for the sub-band component, wherein the predetermined threshold for each sub-band component is based on at least one previous noise-dominant signal frame received by the receiving arrangement, wherein the first and second wavelet transforms are configured to more aggressively de-noise the input audio signal frame when the input audio signal frame is classified in the first category as compared to when the input audio signal frame is classified in the second category,

wherein the processing arrangement is further configured to classify the input audio signal frame by:

applying a principle component analysis using a plurality of features, wherein the plurality of features includes at least one of: (1) an envelope variance feature of the input audio signal frame; (2) an envelope mean crossing feature of the input audio signal frame; (3) a root mean square feature of the input audio signal frame as compared to a predetermined threshold value; and (4) an entropy feature of a histogram of the input audio signal frame; and

classifying the input audio signal frame into the first category when the applying step identifies predominantly noise from the input audio signal frame, and into the second category when the principle component analysis identifies predominantly speech signal from the input audio signal frame.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 10, 2018
From: SOLEYMANI, ROOZBEH; SELESNICK, IVAN W.; LANDSBERGER, DAVID M.
To: NEW YORK UNIVERSITY
Reel/Frame 047124/0977 →
Continuity (4)
Continuation In Part PCTUS2017018696 · Feb 21, 2017
Provisional Application 62394520 · Sep 14, 2016
Provisional Application 62297536 · Feb 19, 2016
Related Publication 20180012614A1 · Jan 11, 2018
Cited By (1)
US 12,597,432