IP Library Granted Patent US 10,796,713
Granted Patent B2
US 10,796,713 · App. 15/951,928 · Granted Oct 6, 2020

Identification of noise signal for voice denoising device

Inventor: Zhijun Du (Hangzhou, CN)
Assignee: Alibaba Group Holding Limited
G10L25/21G10L21/0232G10L21/0324G10L25/51G10L2021/02168G10L2025/783
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,796,713
App. No.
15/951,928
Granted
Oct 6, 2020
Kind
B2
Abstract

Methods, systems, and computer-readable storage media for voice denoising. Implementations include actions of performing a mathematical transform on each frame signal in an audio signal segment to generate multiple power spectra. Each power spectrum corresponds to a respective frame signal. Power value variances corresponding to frame signals at various frequencies are determined. A noise signal is identified in each frame signal based on the power value variance. The identified noise signal is removed from each frame signal of the plurality of frame signals.

Claims (56)

1. A computer-implemented method for voice denoising, the method being executed by one or more processors and comprising:

performing, by the one or more processors, a mathematical transform on each frame signal in an audio signal segment comprising a plurality of frame signals to generate a plurality of power spectra, each power spectrum of the plurality of power spectra corresponding to a respective frame signal;

determining, by the one or more processors, a plurality of power value variances, each power value variance of the plurality of power value variances corresponding to the respective frame signal by classifying power values of each frame signal at various frequencies into a first power value variance corresponding to a first frequency interval and a second power value variance corresponding to a second frequency interval;

generating, by the one or more processors, a ranking of the plurality of frame signals in the audio signal segment according to magnitudes of the plurality of power value variances by determining for each frame signal of the plurality of frame signals:

whether a first condition is satisfied, the first condition comprising the first power value variance being greater than a first threshold,

whether a second condition is satisfied, the second condition comprising the second power value variance being greater than a second threshold,

whether a third condition is satisfied, the third condition comprising a difference between the second power value variance at the respective frame signal and the second power value variance at a subsequent frame signal being greater than a third threshold, and

whether a fourth condition is satisfied, the fourth condition comprising a difference between the second power value variance and the first power value variance is greater than a fourth threshold;

in response to determining that at least one of the first condition, the second condition, the third condition and the fourth condition fails to be satisfied, identifying, by the one or more processors, a noise signal in the respective frame signal of the plurality of frame signals based on the ranking of the plurality of frame signals in the audio signal segment; and

removing, by the one or more processors, the noise signal from the respective frame signal of the plurality of frame signals from the audio signal segment.

2. The computer-implemented method of claim 1 , further comprising determining the audio signal segment based on comparing an amplitude variation to a threshold.

3. The computer-implemented method of claim 1 , wherein identifying the noise signal comprises comparing the each power value variance corresponding to the respective frame signal in the audio signal segment to a noise threshold.

4. The computer-implemented method of claim 1 , wherein determining the plurality of power value variances comprises:

at least classifying power values of the frame signal at various frequencies into a first power value set corresponding to a first frequency interval according to frequency intervals corresponding to the plurality of power spectra; and

determining a first variance of power values comprised in the first power value set.

5. The computer-implemented method of claim 1 , wherein the first frequency interval is lower than the second frequency interval.

6. The computer-implemented method of claim 1 , wherein the ranking of the plurality of frame signals in the audio signal segment comprises a low ranking frame signal comprising a small variance that is smaller than an average variance of the plurality of power value variances and a high ranking frame signal comprising a high variance that is greater than the average variance.

7. The computer-implemented method of claim 1 , further comprising: in response to ranking the frame signals, determining whether each frame signal in the audio signal segment is a noise signal based on the each power value variance of each ranked frame signal at various frequencies.

8. A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations for performing voice denoising, the operations comprising:

performing a mathematical transform on each frame signal in an audio signal segment comprising a plurality of frame signals to generate a plurality of power spectra,

each power spectrum of the plurality of power spectra corresponding to a respective frame signal;

determining a plurality of power value variances, each power value variance of the plurality of power value variances corresponding to the respective frame signal by classifying power values of each frame signal at various frequencies into a first power value variance corresponding to a first frequency interval and a second power value variance corresponding to a second frequency interval;

generating a ranking of the plurality of frame signals in the audio signal segment according to magnitudes of the plurality of power value variances by determining for each frame signal of the plurality of frame signals:

whether a first condition is satisfied, the first condition comprising the first power value variance being greater than a first threshold,

whether a second condition is satisfied, the second condition comprising the second power value variance being greater than a second threshold,

whether a third condition is satisfied, the third condition comprising a difference between the second power value variance at the respective frame signal and the second power value variance at a subsequent frame signal being greater than a third threshold, and

whether a fourth condition is satisfied, the fourth condition comprising a difference between the second power value variance and the first power value variance is greater than a fourth threshold;

in response to determining that at least one of the first condition, the second condition, the third condition and the fourth condition fails to be satisfied, identifying a noise signal in the respective frame signal of the plurality of frame signals based on the ranking of the plurality of frame signals in the audio signal segment; and

removing the noise signal from the respective frame signal of the plurality of frame signals from the audio signal segment.

9. The non-transitory, computer-readable medium of claim 8 , the operations further comprising determining the audio signal segment based on comparing an amplitude variation to a threshold.

10. The non-transitory, computer-readable medium of claim 8 , wherein identifying the noise signal comprises comparing the each power value variance corresponding to the respective frame signal in the audio signal segment to a noise threshold.

11. The non-transitory, computer-readable medium of claim 9 , wherein determining the plurality of power value variances comprises:

at least classifying power values of the frame signal at various frequencies into a first power value set corresponding to a first frequency interval according to frequency intervals corresponding to the plurality of power spectra; and

determining a first variance of power values comprised in the first power value set.

12. The non-transitory, computer-readable medium of claim 8 , wherein the first frequency interval is lower than the second frequency interval.

13. The non-transitory, computer-readable medium of claim 8 , wherein the ranking of the plurality of frame signals in the audio signal segment comprises a low ranking frame signal comprising a small variance that is smaller than an average variance of the plurality of power value variances and a high ranking frame signal comprising a high variance that is greater than the average variance.

14. The non-transitory, computer-readable medium of claim 8 , the operations further comprising in response to ranking the frame signals, determining whether each frame signal in the audio signal segment is a noise signal based on the each power value variance of each ranked frame signal at various frequencies.

15. A computer-implemented system for voice denoising, comprising:

one or more computers; and

one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing instructions that, if executed by the one or more computers, perform operations comprising:

performing a mathematical transform on each frame signal in an audio signal segment comprising a plurality of frame signals to generate a plurality of power spectra, each power spectrum of the plurality of power spectra corresponding to a respective frame signal;

determining a plurality of power value variances, each power value variance of the plurality of power value variances corresponding to the respective frame signal by classifying power values of each frame signal at various frequencies into a first power value variance corresponding to a first frequency interval and a second power value variance corresponding to a second frequency interval;

generating a ranking of the plurality of frame signals in the audio signal segment according to magnitudes of the plurality of power value variances by determining for each frame signal of the plurality of frame signals:

whether a first condition is satisfied, the first condition comprising the first power value variance being greater than a first threshold,

whether a second condition is satisfied, the second condition comprising the second power value variance being greater than a second threshold,

whether a third condition is satisfied, the third condition comprising a difference between the second power value variance at the respective frame signal and the second power value variance at a subsequent frame signal being greater than a third threshold, and

whether a fourth condition is satisfied, the fourth condition comprising a difference between the second power value variance and the first power value variance is greater than a fourth threshold;

in response to determining that at least one of the first condition, the second condition, the third condition and the fourth condition fails to be satisfied, identifying a noise signal in the respective frame signal of the plurality of frame signals based on the ranking of the plurality of frame signals in the audio signal segment; and

removing the noise signal from the respective frame signal of the plurality of frame.

16. The computer-implemented system of claim 15 , the operations further comprising determining the audio signal segment based on comparing an amplitude variation to a threshold.

17. The computer-implemented system of claim 15 , wherein identifying the noise signal comprises comparing the each power value variance corresponding to the respective frame signal in the audio signal segment to a noise threshold.

18. The computer-implemented system of claim 15 , wherein determining the plurality of power value variances comprises:

at least classifying power values of the frame signal at various frequencies into a first power value set corresponding to a first frequency interval according to frequency intervals corresponding to the plurality of power spectra; and

determining a first variance of power values comprised in the first power value set.

19. The computer-implemented system of claim 15 , wherein the first frequency interval is lower than the second frequency interval.

20. The computer-implemented system of claim 15 , wherein the ranking of the plurality of frame signals in the audio signal segment comprises a low ranking frame signal comprising a small variance that is smaller than an average variance of the plurality of power value variances and a high ranking frame signal comprising a high variance that is greater than the average variance.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 10, 2020
From: ADVANTAGEOUS NEW TECHNOLOGIES CO., LTD.
To: ADVANCED NEW TECHNOLOGIES CO., LTD.
Reel/Frame 053754/0625 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 31, 2020
From: ALIBABA GROUP HOLDING LIMITED
To: ADVANTAGEOUS NEW TECHNOLOGIES CO., LTD.
Reel/Frame 053743/0464 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 8, 2018
From: DU, ZHIJUN
To: ALIBABA GROUP HOLDING LIMITED
Reel/Frame 046584/0868 →
Priority Claims (1)
CN 2015 1 0670697 · Oct 13, 2015 · national
Continuity (2)
Continuation PCTCN2016101444 · Oct 8, 2016
Related Publication 20180293997A1 · Oct 11, 2018