System and method for spectral pooling in streaming speech processing

1 . A computer-implemented method, implemented by a computing system that includes one or more processors and one or more computer-readable hardware storage media having stored thereon computer-executable instructions, the computer-executable instructions being executable by the one or more processors to configure the computer system to implement the method, the method comprising:

inserting a first spectral pooling layer into a neural network of a speech processing system directly between a first hidden layer of the neural network and a second hidden layer of the neural network;

filtering an output of the first hidden layer of the neural network using the first spectral pooling layer with a first non-integer stride to generate a first filtered output;

providing the first filtered output to the second hidden layer of the neural network;

inserting a second spectral pooling layer into the neural network directly between the second hidden layer of the neural network and a third hidden layer of the neural network;

filtering an output of the second hidden layer of the neural network using the second spectral pooling layer with a second non-integer stride to generate a second filtered output; and

providing the second filtered output to the third hidden layer of the neural network.

2 . The computer-implemented method of claim 1 , wherein filtering the output of the first hidden layer of the neural network includes converting the output of the first hidden layer of the neural network from a time-domain signal to a frequency-domain signal.

3 . The computer-implemented method of claim 2 , wherein filtering the output of the first hidden layer of the neural network includes filtering a coefficient from the frequency-domain signal using the first non-integer stride to generate a filtered frequency-domain signal.

4 . The computer-implemented method of claim 3 , wherein filtering the output of the first hidden layer of the neural network includes converting the filtered frequency-domain signal into a filtered time-domain signal.

5 . The computer-implemented method of claim 1 , wherein filtering the output of the first hidden layer of the neural network using the first spectral pooling layer includes distributing a stride value across at least the first spectral pooling layers and the second spectral pooling layer.

6 . The computer-implemented method of claim 3 , wherein filtering the output of the first hidden layer of the neural network using the first spectral pooling layer includes performing spectral upsampling on the filtered frequency-domain signal.

7 . A computing system comprising:

one or more processors; and

one or more computer-readable hardware storage media having stored thereon computer-executable instructions, the computer-executable instructions being executable by the one or more processors to configure the computer system to:

insert a first spectral pooling layer into a neural network of a speech processing system directly between a first hidden layer of the neural network and a second hidden layer of the neural network;

filter an output of the first hidden layer of the neural network using the first spectral pooling layer with a first non-integer stride and a first matrix filter to generate a first filtered output;

provide the filtered output to the second hidden layer of the neural network;

insert a second spectral pooling layer into the neural network directly between the second hidden layer of the neural network and a third hidden layer of the neural network;

filter an output of the second hidden layer of the neural network using the second spectral pooling layer with a second non-integer stride to generate a second filtered output; and

provide the second filtered output to the third hidden layer of the neural network.

8 . The computing system of claim 7 , wherein filtering the output of the first hidden layer of the neural network using the spectral pooling layer includes:

generating the matrix filter.

9 . The computing system of claim 8 , wherein generating the matrix filter includes:

generating a complex matrix representation of a discrete Fourier transform.

10 . The computing system of claim 9 , wherein generating the matrix filter includes:

generating a coefficient filtering matrix.

11 . The computing system of claim 10 , wherein generating the matrix filter includes:

generating a complex matrix representation of an inverse Fourier transform.

12 . The computing system of claim 11 , wherein filtering the output of the first hidden layer of the neural network using the first spectral pooling layer includes:

generating the matrix filter by multiplying the complex matrix representation of the discrete Fourier transform, the coefficient filtering matrix, and the complex matrix representation of an inverse Fourier transform; and

generating the filtered output by multiplying the output of the first hidden layer of the neural network and the matrix filter.

13 . A hardware storage device having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising:

inserting a first spectral pooling layer into a neural network of a speech processing system directly between a first hidden layer of the neural network and a second hidden layer of the neural network;

filtering an output of the first hidden layer of the neural network using the first spectral pooling layer with a first non-integer stride and a first complex matrix filter to generate a first filtered output by multiplying the output of the first hidden layer of the neural network by a real portion of the first complex matrix filter;

providing the first filtered output to the second hidden layer of the neural network;

inserting a second spectral pooling layer into the neural network directly between the second hidden layer of the neural network and a third hidden layer of the neural network;

filtering an output of the second hidden layer using the second spectral pooling layer with a second non-integer stride and a second complex matrix filter to generate a second filtered output by multiplying the output of the second hidden layer of the neural network by a real portion of the second complex matrix filter; and

providing the second filtered output to the third hidden layer of the neural network.

14 . The hardware storage device of claim 13 , wherein filtering the output of the first hidden layer of the neural network using the spectral pooling layer includes:

generating the complex matrix filter.

15 . The hardware storage device of claim 14 , wherein generating the complex matrix filter includes:

generating a complex matrix representation of a discrete Fourier transform;

generating a coefficient filtering matrix;

generating a complex matrix representation of an inverse Fourier transform;

generating the matrix filter by multiplying the complex matrix representation of the discrete Fourier transform, the coefficient filtering matrix, and the complex matrix representation of an inverse Fourier transform; and

generating the filtered output by multiplying the output of the first hidden layer of the neural network and the matrix filter.

16 . The hardware storage device of claim 13 , wherein filtering the output of the hidden layer of the neural network using the spectral pooling layer includes:

distributing a stride value across a plurality of spectral pooling layers.

17 . The hardware storage device of claim 13 , wherein the speech processing system is a streaming automated speech recognition (ASR) system.