Generating expressive speech audio from text data

1. A system for use in video game development for generating expressive speech audio, the system comprising:

a user interface configured to receive user-input text data and a user selection of a speech style; and

a machine-learned synthesizer comprising a text encoder, a speech style encoder and a decoder, the machine-learned synthesizer being configured to:

generate one or more text encodings derived from the user-input text data, using the text encoder of the machine-learned synthesizer;

generate a speech style encoding by processing a set of speech style features associated with the selected speech style using the speech style encoder of the machine-learned synthesizer;

combine the one or more text encodings and the speech style encoding to generate one or more combined encodings; and

decode the one or more combined encodings with the decoder of the machine-learned synthesizer to generate predicted spectrogram parameters for the expressive speech audio.

2. The system of claim 1 , wherein the spectrogram parameters comprise linear spectrogram parameters.

3. The system of claim 1 , wherein the spectrogram parameters comprise log mel-spectrogram parameters.

4. The system of claim 1 , wherein the spectrogram parameters comprise Mel-Frequency Cepstral Coefficients (MFCCs).

5. The system of claim 1 , wherein the set of speech style features comprises prosodic features determined from the selected speech style.

6. The system of claim 5 , wherein:

the user interface is further configured to receive a user selection of an instance of speech audio; and

the system further comprises a prosody analyzer configured to process the selected instance of speech audio to determine the prosodic features.

7. The system of claim 1 , wherein:

the user interface is further configured to receive a user selection of speaker attribute information; and

the set of speech style features further comprises the speaker attribute information.

8. The system of claim 1 further comprising one or more modules configured to process the predicted spectrogram parameters, the one or more modules comprising a machine-learned vocoder configured to generate a waveform of the expressive speech audio.

9. The system of claim 8 , wherein:

the user interface is further configured to receive a user selection of effects to be applied when generating the expressive speech audio; and

the one or more modules further comprise a special effects module, the special effects module configured to modify a predicted spectrogram parameters output by the machine-learned synthesizer to generate modified spectrogram parameters.

10. The system of claim 9 , wherein the special effects module is a machine-learned special effects module.

11. The system of claim 8 , wherein:

the user interface is further configured to receive a user selection of mastering information to be applied when generating the expressive speech audio; and

the one or more modules further comprise a mastering module, the mastering module configured to modify the waveform of the expressive speech audio by the machine-learned vocoder in accordance with the user selection of mastering information.

12. The system of claim 11 , wherein the mastering module is configured to perform one or more modifications to the waveform of:

sample rate;

bit depth;

audio format;

amplitude; and/or

silence removal.

13. The system of claim 8 wherein at least one of the machine-learned synthesizer and the machine-learned vocoder comprises recurrent neural network layers.

14. A computer-implemented method for generating spectrogram parameters from text data using a machine-learned synthesizer, the method comprising:

receiving user-input text data and a user selection of a speech style;

generating one or more text encodings derived from the user-input text data, using a text encoder of the machine-learned synthesizer;

generating a speech style encoding by processing a set of speech style features associated with the selected speech style using a speech style encoder of the machine-learned synthesizer;

combining one or more text encodings and the speech style encoding to generate one or more combined encodings; and

decoding the one or more combined encodings with a decoder of the machine-learned synthesizer to generate predicted spectrogram parameters for expressive speech audio.

15. The method of claim 12 , wherein the spectrogram parameters comprise linear spectrogram parameters.

16. The method of claim 14 , wherein the spectrogram parameters comprise log mel-spectrogram parameters.

17. The method of claim 14 , wherein the spectrogram parameters comprise Mel-Frequency Cepstral Coefficients (MFCCs).

18. A non-transitory computer readable medium storing instructions, which when executed by a processor, cause the processor to:

receive user-input text data and a user selection of a speech style;

generate one or more text encodings derived from the user-input text data, using a text encoder of a machine-learned synthesizer;

generate a speech style encoding by processing a set of speech style features associated with the selected speech style using a speech style encoder of the machine-learned synthesizer;

combine one or more text encodings and the speech style encoding to generate one or more combined encodings; and

decode the one or more combined encodings with a decoder of the machine-learned synthesizer to generate predicted spectrogram parameters for expressive speech audio.

19. The non-transitory computer readable medium of claim 18 , wherein the spectrogram parameters comprise linear spectrogram parameters.

20. The non-transitory computer readable medium of claim 18 , wherein the spectrogram parameters comprise log mel-spectrogram parameters.