IP Library › Granted Patent US 9,640,186
Granted Patent B2
US 9,640,186 · App. 14/268,459 · Granted May 2, 2017

Deep scattering spectrum in acoustic modeling for speech recognition

Inventors: Petr Fousek (Litomerice, CZ); Vaibhava Goel (Chappaqua, NY); Brian E. D. Kingsbury (Cortlandt Manor, NY); Etienne Marcheret (White Plains, NY); Shay Maymon (Raanana, IL); David Nahamoo (Great Neck, NY); Vijayaditya Peddinti (Baltimore, MD); Bhuvana Ramabhadran (Mt. Kisco, NY); Tara N. Sainath (Jersey City, NJ)
Assignee: International Business Machines Corporation
G10L19/02G10L19/26
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,640,186
App. No.
14/268,459
Filed
May 2, 2014
Granted
May 2, 2017
Kind
B2
Art Unit
2657
USPC
704/500
Abstract

Deep scattering spectral features are extracted from an acoustic input signal to generate a deep scattering spectral feature representation of the acoustic input signal. The deep scattering spectral feature representation is input to a speech recognition engine. The acoustic input signal is decoded based on at least a portion of the deep scattering spectral feature representation input to a speech recognition engine.

Claims (31)

1. A method, comprising: extracting deep scattering spectral features from an acoustic input signal to generate a deep scattering spectral feature representation of the acoustic input signal;

inputting the deep scattering spectral feature representation to a speech recognition engine;

decoding the acoustic input signal based on at least a portion of the deep scattering spectral feature representation input to a speech recognition engine; and

outputting the decoded acoustic input signal;

wherein the speech recognition engine utilizes a hybrid architecture comprising a combination of a deep neural network and a convolutional neural network to decode the acoustic input signal, and further wherein features in the deep scattering spectral feature representation that have a local correlation in frequency are fed into the convolutional neural network part of the hybrid architecture and features that remove the local correlation are fed into the deep neural network part of the hybrid architecture, and a set of probabilities output by the hybrid architecture are used to evaluate fit between a set of acoustic models and the acoustic input signal; and wherein a result of the evaluation of fit between the set of acoustic models and the acoustic input signal is output;

wherein the extracting, inputting, decoding, and outputting steps are executed via a computer system comprising an acoustic signal processing unit and a memory.

2. The method of claim 1 , wherein the extracting step further comprises extracting first-order scatter features from the acoustic input signal.

3. The method of claim 2 , wherein extracting first-order scatter features from the acoustic input signal further comprises passing the acoustic input signal through a first wavelet transform, a modulus operator, and a low pass filter.

4. The method of claim 3 , wherein the extracting step further comprises extracting second-order scatter features from the acoustic input signal.

5. The method of claim 4 , wherein extracting the second-order scatter features from the acoustic input signal further comprises passing at least a portion of the acoustic input signal through a second wavelet transform.

6. The method of claim 5 , wherein the extracting step further comprises normalizing the second-order scatter features based on the first-order scatter features.

7. The method of claim 5 , wherein the extracting step is performed in the time domain.

8. The method of claim 5 , wherein the extracting step is performed in the frequency domain.

9. The method of claim 5 , wherein the extracting step is performed in a multi-resolution manner based on wavelets having different quality factors.

10. The method of claim 1 , wherein the extracting step further comprises reducing a dimensionality of the deep scattering spectral features.

11. The method of claim 10 , wherein the dimensionality reducing step further comprises performing a principal component analysis.

12. The method of claim 10 , wherein the dimensionality reducing step further comprises performing a linear discriminant analysis.

13. The method of claim 10 , wherein the dimensionality reducing step further comprises applying a linear bottleneck layer.

14. The method of claim 1 , wherein the extracting step further comprises applying at least one of a speaker adaptation and a sequence training.

15. An article of manufacture comprising a non-transitory computer readable storage medium having tangibly embodied thereon computer readable program code which, when executed, causes an acoustic signal processing unit to:

extract deep scattering spectral features from an acoustic input signal to generate a deep scattering spectral feature representation of the acoustic input signal;

input the deep scattering spectral feature representation to a speech recognition engine;

decode the acoustic input signal based on at least a portion of the deep scattering spectral feature representation via the speech recognition engine; and

output the decoded acoustic input signal;

wherein the speech recognition engine utilizes a hybrid architecture comprising a combination of a deep neural network and a convolutional neural network to decode the acoustic input signal, and further wherein features in the deep scattering spectral feature representation that have a local correlation in frequency are fed into the convolutional neural network part of the hybrid architecture and features that remove the local correlation are fed into the deep neural network part of the hybrid architecture, and a set of probabilities output by the hybrid architecture are used to evaluate fit between a set of acoustic models and the acoustic input signal; and wherein a result of the evaluation of fit between the set of acoustic models and the acoustic input signal is output.

16. An apparatus, comprising:

a memory; and an acoustic signal processing unit operatively coupled to the memory and configured to:

extract deep scattering spectral features from an acoustic input signal to generate a deep scattering spectral feature representation of the acoustic input signal;

decode the acoustic input signal based on at least a portion of the deep scattering spectral feature representation via a speech recognition engine; and

output the decoded acoustic input signal;

wherein the speech recognition engine utilizes a hybrid architecture comprising a combination of a deep neural network and a convolutional neural network to decode the acoustic input signal, and further wherein features in the deep scattering spectral feature representation that have a local correlation in frequency are fed into the convolutional neural network part of the hybrid architecture and features that remove the local correlation are fed into the deep neural network part of the hybrid architecture, and a set of probabilities output by the hybrid architecture are used to evaluate fit between a set of acoustic models and the acoustic input signal; and wherein a result of the evaluation of fit between the set of acoustic models and the acoustic input signal is output.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 3, 2015
From: FOUSEK, PETR; GOEL, VAIBHAVA; KINGSBURY, BRIAN E.D.; MARCHERET, ETIENNE; MAYMON, SHAY; NAHAMOO, DAVID; PEDDINTI, VIJAYADITYA; RAMABHADRAN, BHUVANA; SAINATH, TARA N.
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 037200/0951 →
Continuity (1)
Related Publication 20150317990A1 · Nov 5, 2015