Textual echo cancellation

1. A computer-implemented method when executed on data processing hardware causes the data processing hardware to perform operations comprising:

obtaining a plurality of training examples, each training example of the plurality of training examples comprising:

a ground truth audio signal corresponding to non-synthesized speech;

a training overlapped audio signal comprising the ground truth audio signal overlapping a synthesized audio signal; and

training text comprising a respective sequence of characters representing the synthesized audio signal; and

training a cancelation neural network on the plurality of training examples to teach the cancelation neural network to generate the ground truth audio signals based on the training overlapped audio signals and the training text.

2. The computer-implemented method of claim 1 , wherein training the cancelation neural network comprises, for each training example of the plurality of training examples:

encoding the respective sequence of characters that represent the synthesized audio signal into a text embedding representation;

for each character in the respective sequence of characters, generating, using the text embedding representation, a respective cancelation probability indicating a likelihood that the corresponding character is associated with a segment of the synthesized audio signal overlapped by the ground truth audio signal in the training overlapped audio signal; and

teaching, based on the cancelation neural network receiving the training overlapped audio signal and the respective cancelation probability generated for each character in the respective sequence of characters as inputs, the cancelation neural network to generate the ground truth audio signal by removing the segment of the synthesized audio signal from the training overlapped audio signal.

3. The computer-implemented method of claim 2 , wherein the text embedding representation comprises a single, fixed-dimensional text embedding vector.

4. The computer-implemented method of claim 2 , wherein the text embedding representation comprises a length corresponding to a number characters in the respective sequence of characters.

5. The computer-implemented method of claim 2 , wherein encoding the respective sequence of characters comprises encoding each character in the respective sequence of characters into a corresponding character embedding to generate a sequence of character embeddings.

6. The computer-implemented method of claim 5 , wherein:

the training overlapped audio signal comprises a sequence of frames, each frame in the sequence of frames corresponding to a portion of the ground truth audio signal that overlaps the segment of synthesized audio signal; and

generating the respective cancelation probability for each character in the respective sequence of characters comprises using an attention mechanism to apply a weight to the corresponding character embedding when the corresponding character embedding corresponds to one of the frames in the sequence of frames of the training overlapped audio signal.

7. The computer-implemented method of claim 1 , wherein a text encoder of a text encoding neural network encodes the respective sequence of characters that correspond to the synthesized audio signal into the text embedding representation.

8. The computer-implemented method of claim 7 , wherein the text encoder is shared by a text-to-speech (TTS) system, the TTS system configured to generate the synthesized audio signal from the respective sequence of characters.

9. The computer-implemented method of claim 1 , wherein the cancelation neural network comprises a Long Short Term Memory (LSTM) network with a plurality of LSTM layers.

10. The computer-implemented method of claim 1 , wherein a text-to-speech (TTS) system converts the respective sequence of characters into synthesized speech comprising the synthesized audio signal.

11. A system comprising:

data processing hardware; and

memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:

obtaining a plurality of training examples, each training example of the plurality of training examples comprising:

a ground truth audio signal corresponding to non-synthesized speech;

a training overlapped audio signal comprising the ground truth audio signal overlapping a synthesized audio signal; and

training text comprising a respective sequence of characters representing the synthesized audio signal; and

training a cancelation neural network on the plurality of training examples to teach the cancelation neural network to generate the ground truth audio signals based on the training overlapped audio signals and the training text.

12. The system of claim 11 , wherein training the cancelation neural network comprises, for each training example of the plurality of training examples:

encoding the respective sequence of characters that represent the synthesized audio signal into a text embedding representation;

for each character in the respective sequence of characters, generating, using the text embedding representation, a respective cancelation probability indicating a likelihood that the corresponding character is associated with a segment of the synthesized audio signal overlapped by the ground truth audio signal in the training overlapped audio signal; and

teaching, based on the cancelation neural network receiving the training overlapped audio signal and the respective cancelation probability generated for each character in the respective sequence of characters as inputs, the cancelation neural network to generate the ground truth audio signal by removing the segment of the synthesized audio signal from the training overlapped audio signal.

13. The system of claim 12 , wherein the text embedding representation comprises a single, fixed-dimensional text embedding vector.

14. The system of claim 12 , wherein the text embedding representation comprises a length corresponding to a number characters in the respective sequence of characters.

15. The system of claim 12 , wherein encoding the respective sequence of characters comprises encoding each character in the respective sequence of characters into a corresponding character embedding to generate a sequence of character embeddings.

16. The system of claim 15 , wherein:

the training overlapped audio signal comprises a sequence of frames, each frame in the sequence of frames corresponding to a portion of the ground truth audio signal that overlaps the segment of synthesized audio signal; and

generating the respective cancelation probability for each character in the respective sequence of characters comprises using an attention mechanism to apply a weight to the corresponding character embedding when the corresponding character embedding corresponds to one of the frames in the sequence of frames of the training overlapped audio signal.

17. The system of claim 11 , wherein a text encoder of a text encoding neural network encodes the respective sequence of characters that correspond to the synthesized audio signal into the text embedding representation.

18. The system of claim 17 , wherein the text encoder is shared by a text-to-speech (TTS) system, the TTS system configured to generate the synthesized audio signal from the respective sequence of characters.

19. The system of claim 11 , wherein the cancelation neural network comprises a Long Short Term Memory (LSTM) network with a plurality of LSTM layers.

20. The system of claim 11 , wherein a text-to-speech (TTS) system converts the respective sequence of characters into synthesized speech comprising the synthesized audio signal.