SEQUENCE TO SEQUENCE TRANSFORMATIONS FOR SPEECH SYNTHESIS VIA RECURRENT NEURAL NETWORKS
A system eliminates alignment processing and performs TTS functionality using a new neural architecture. The neural architecture includes an encoder and a decoder. The encoder receives an input and encodes it into vectors. The encoder applies a sequence of transformations to the input and generates a vector representing the entire sentence. The decoder takes the encoding and outputs an audio file, which can include compressed audio frames.
1 . A method for performing speech synthesis, comprising:
receiving one or more streams of input by one or more decoders implemented on a computing device;
generating a context vector by the one or more encoders;
decoding the context vector by a decoding mechanism implemented on the computing device;
feeding the decoded context vectors into a neural network implemented on the computing device; and
providing an audio file from the neural network.
2 . The method of claim 1 , wherein the streams of input include original text data and pronunciation data.
3 . The method of claim 2 , wherein one or more streams are processed simultaneously as a single process.
4 . The method of claim 1 , wherein decoding the context vector includes generating an attention vector.
5 . The method of claim 1 , wherein decoding the context vector includes computing an attention score.
6 . The method of claim 1 , wherein decoding the context vector includes computing an attention distribution.
7 . The method of claim 1 , wherein the system provides text-to-speech function to an automated assistant system.
8 . The method of claim 1 , further comprising determining to end processing of the one or more streams of input upon processing a stop frame.
9 . The method of claim 1 , wherein the audio file includes compressed audio frames.
10 . A system for performing speech synthesis, comprising:
one or more encoder modules stored in memory and executable by a processor that when executed receive one or more streams of input and generate a context vector for each stream; and
a decoder module stored in memory and executable by a processor that when executed decodes the context vector, feeds the decoded context vectors into a neural network, provides an audio file from the neural network.
11 . The system of claim 10 , wherein the streams of input include original text data and pronunciation data.
12 . The system of claim 11 , wherein one or more streams are processed simultaneously as a single process.
13 . The system of claim 10 , wherein decoding the context vector includes generating an attention vector.
14 . The system of claim 10 , wherein decoding the context vector includes computing an attention score.
15 . The system of claim 10 , wherein decoding the context vector includes computing an attention distribution.
16 . The system of claim 10 , wherein the system provides text-to-speech function to an automated assistant system.
17 . The system of claim 10 , further comprising determining to end processing of the one or more streams of input upon processing a stop frame.
18 . The system of claim 10 , wherein the audio file includes compressed audio frames.