Apparatus and method for speech processing using a densely connected hybrid neural network
Disclosed is a speech processing apparatus and method using a densely connected hybrid neural network. The speech processing method includes inputting a time domain sample of N*1 dimension for an input speech into a densely connected hybrid network; passing the time domain sample through a plurality of dense blocks in a densely connected hybrid network; reshaping the time domain samples into M subframes by passing the time domain samples through the plurality of dense blocks; inputting the M subframes into gated recurrent unit (GRU) components of N/M-dimension; outputting clean speech from which noise is removed from the input speech by passing the M subframes through GRU components.
1. A speech processing method, comprising:
inputting a time domain sample of N*1 dimension for an input speech into a densely connected hybrid network, N being an integer, greater than one;
passing the time domain sample through a plurality of dense blocks in a densely connected hybrid network;
reshaping the time domain samples into M subframes by passing the time domain samples through the plurality of dense blocks;
inputting the M subframes into gated recurrent unit (GRU) components of N/M-dimension;
outputting clean speech from which noise is removed from the input speech by passing the M subframes through GRU components
wherein the densely connected hybrid network is combination a convolutional neural network (CNN) and a recurrent neural network (RNN).
2. The speech processing method of claim 1 , wherein the densely connected hybrid network includes a plurality of dense blocks,
wherein the each of the dense blocks comprise a plurality of convolutional layers.
3. The speech processing method of claim 1 , wherein the time domain sample of the N*1 dimension is extended to convolution layers of the N*D dimension to the N*MD dimension, and each of the convolution layers of the N*D dimension to the N*MD dimension may be connected,
wherein D is an expansion rate, and M is the number of convolutional layers constituting the dense block.
4. The speech processing method of claim 1 , wherein the each of the plurality of dense blocks included in the densely connected hybrid network is expressed as a repetitive neural network as having identical convolutional layers.
5. The speech processing method of claim 1 , wherein the output of a last dense block of the densely connected hybrid network is reshaped into M subframes,
wherein the M subframes are input to N/M-dimensional GRU components.
6. The speech processing method of claim 1 , wherein each of the dense blocks outputs a data tensor in the form of a combination of features and channels by performing a 1D convolution operation.
7. The speech processing method of claim 1 , wherein the each of the dense blocks performs a 1D convolution operation or an extended convolution operation according to an expansion rate.
8. The speech processing method of claim 1 , wherein the GRU components enhance speech in a form in which a first hidden state and a second hidden state are mixed by an update gate.
9. The speech processing method of claim 8 , wherein the second hidden state is determined using a linear combination of an input of the GRU component and the first hidden state that is a gated previous hidden state.
10. A speech processing device comprising:
a processor;
wherein the processor is configured to:
input a time domain sample of N*1 dimension for an input speech into a densely connected hybrid network, N being an integer greater than one;
pass the time domain sample through a plurality of dense blocks in a densely connected hybrid network;
reshape the time domain samples into M subframes by passing the time domain samples through the plurality of dense blocks;
input the M subframes into gated recurrent unit (GRU) components of N/M-dimension;
output clean speech from which noise is removed from the input
speech by passing the M subframes through GRU components wherein the densely connected hybrid network is combination a convolutional neural network (CNN) and a recurrent neural network (RNN).
11. The speech processing device of claim 10 , wherein the densely connected hybrid network includes a plurality of dense blocks,
wherein the each of the dense blocks comprise a plurality of convolutional layers.
12. The speech processing device of claim 10 , wherein the time domain sample of the N*1 dimension is extended to convolution layers of the N*D dimension to the N*MD dimension, and each of the convolution layers of the N*D dimension to the N*MD dimension may be connected,
wherein D is an expansion rate, and M is the number of convolutional layers constituting the dense block.
13. The speech processing device of claim 10 , wherein the each of the plurality of dense blocks included in the densely connected hybrid network is expressed as a repetitive neural network as having identical convolutional layers.
14. The speech processing device of claim 10 , wherein the output of a last dense block of the densely connected hybrid network is reshaped into M subframes,
wherein the M subframes are input to N/M-dimensional GRU components.
15. The speech processing device of claim 10 , wherein each of the dense blocks outputs a data tensor in the form of a combination of features and channels by performing a 1D convolution operation.
16. The speech processing device of claim 10 , wherein the each of the dense blocks performs a 1D convolution operation or an extended convolution operation according to an expansion rate.
17. The speech processing device of claim 10 , wherein the GRU components enhance speech in a form in which a first hidden state and a second hidden state are mixed by an update gate.
18. The speech processing device of claim 17 , wherein the second hidden state is determined using a linear combination of an input of the GRU component and the first hidden state that is a gated previous hidden state.