Code-mixed speech engine in a speech synthesis system

1 . A computerized system comprising:

one or more computer processors; and

computer memory storing computer-useable instructions that, when used by the one or more computer processors, cause the one or more computer processors to perform operations, the operations comprising:

accessing code-mixed training data associated with a plurality of different languages;

based on the code-mixed training data, generating a code-mixed speech model associated with a training engine and an inference engine that support generating code-mixed synthesized speech,

wherein the code-mixed speech model is trained using an orthogonal loss training constraint that separates speaker embeddings and language embeddings to reduce accent leakage;

deploying the code-mixed speech model;

receiving a request for synthesized speech, the request being associated with code-mixed input of a speech synthesis client;

using the code-mixed speech model and the inference engine, generating an instance of code-mixed synthesized speech in a single target speaker voice across two or more languages; and

communicating the instance of the code-mixed synthesized speech associated with the single target speaker voice for output on an interface associated with the speech synthesis client.

2 . The system of claim 1 , wherein the code-mixed speech model comprises a multilingual speech model generated based on training the code-mixed speech model using the code-mixed training data associated with a two or more languages and two or more scripts, wherein training the code-mixed speech model is based at least in part on speaker embeddings and language embeddings identified in the code-mixed training data.

3 . The system of claim 1 , wherein the training engine supports an orthogonal loss training constraint, the orthogonal loss training constraint comprises disentangled speaker embeddings and language embeddings in training the code-mixed speech model to reduce accent leak of a target speaker.

4 . The system of claim 1 , wherein the code-mixed speech model is deployed to support generating inferences using the inference engine, wherein the code-mixed speech model is accessible at the speech synthesis client via an Application Programming Interface (API).

5 . The system of claim 1 , wherein the speech synthesis client comprises the interface that supports receiving the request and outputting the instance of code-mixed synthesized speech, wherein the request comprises the code-mixed input including input text in two or more languages and the instance of the code-mixed synthesized speech is associated with output text in two or more languages.

6 . The system of claim 1 , wherein the inference engine supports selecting a locale identifier and prosody features of a target speaker; and executing duration scaling based on phones identified in the code-mixed input request.

7 . The system of claim 1 , wherein generating the code-mixed synthesized speech further comprises:

using a language identification model to segment text of the code-mixed input into a first text of a first language and a second text of a second language;

selecting a first language module for the first text of the first language;

selecting a second language module for the second text of the second language;

using the first language module and the second language module, generating a code-mixed phone sequence comprising a plurality of phones associated with the first text and the second text, wherein each code-mixed phone is associated with a phone identifier; and

generating the code-mixed synthesized speech based on changing a reference speaker and language identifier based on a corresponding phone identifier.

8 . The system of claim 7 , wherein the reference speaker and the language identifier are associated with a plurality of prosody features that are employed for a target speaker's voice associated with the code-mixed synthesized speech.

9 . The system of claim 1 , the operations further comprising:

communicating, from the speech synthesis client, the request for the code-mixed synthesized speech;

based on the request, receiving the code-mixed synthesized speech; and

causing output of the code-mixed synthesized speech on the interface.

10 . The system of claim 1 , the operations further comprising:

receiving, from the speech synthesis client, an indication to update to a new target speaker;

communicating a second request for a second instance of code-mixed synthesized speech;

based on the second request, receiving the second instance of code-mixed synthesized speech generated based on the new target speaker, wherein the second instance of code-mixed synthesized speech is different from the instance of code-mixed synthesized speech and generated based on the new target speaker; and

causing output of the second code-mixed synthesized speech on the interface.

11 . One or more computer-storage media having computer-executable instructions embodied thereon that, when executed by a computing system having a processor and memory, cause the processor to perform operations, the operations comprising:

communicating a request for synthesized speech, the request being associated with code-mixed input of a speech synthesis client;

based on the request, receiving an instance of code-mixed synthesized speech in a single target speaker voice across two or more languages, the code-mixed synthesized speech is generated using a code-mixed speech model that is associated with a training engine and an inference engine that support generating code-mixed synthesized speech,

wherein the code-mixed speech model is trained using an orthogonal loss training constraint that separates speaker embeddings and language embeddings to reduce accent leakage; and

causing output of the instance of code-mixed synthesized speech associated with the single target speaker voice via an interface associated with the speech synthesis client.

12 . The media of claim 11 , wherein the speech synthesis client comprises the interface that supports receiving the request and outputting the instance of code-mixed synthesized speech, wherein the request comprises the code-mixed input comprises input text in two or more languages and the instance of the code-mixed synthesized speech is associated with output text in two or more languages.

13 . The media of claim 11 , wherein the training engine supports an orthogonal loss training constraint, the orthogonal loss training constraint comprises disentangled speaker embeddings and language embeddings in training the code-mixed speech model to reduce accent leak of a target speaker.

14 . The media of claim 11 , wherein the inference engine supports selecting a locale identifier and prosody features of a target speaker; and executing duration scaling based on phones identified in the code-mixed input request.

15 . The media of claim 11 , the operations further comprising:

receiving, from the speech synthesis client, an indication to update to a new target speaker;

communicating a second request for a second instance of code-mixed synthesized speech;

based on the second request, receiving the second instance of code-mixed synthesized speech generated based on the new target speaker, wherein the second instance of code-mixed synthesized speech is different from the instance of code-mixed synthesized speech and generated based on the new target speaker; and

causing output of the second code-mixed synthesized speech on the interface.

16 . A computer-implemented method, the method comprising:

accessing code-mixed training data associated with a plurality of different languages;

based on the code-mixed training data, generating a code-mixed speech model associated with a training engine and an inference engine that support generating code-mixed synthesized speech in a single target speaker voice across two or more languages,

wherein the code-mixed speech model is trained using an orthogonal loss training constraint that separates speaker embeddings and language embeddings to reduce accent leakage; and

deploying the code-mixed speech model to cause generation of an instance of code-mixed synthesized speech in a single target speaker voice across two or more languages using the code-mixed speech model and the inference engine, generating.

17 . The method of claim 16 , wherein the training engine supports an orthogonal loss training constraint, the orthogonal loss training constraint comprises disentangled speaker embeddings and language embedding in training the code-mixed speech model to reduce accent leak of a target speaker.

18 . The method of claim 16 , wherein the inference engine supports selecting a locale identifier and prosody features of a target speaker; and executing duration scaling based on a phone code-mixed input requests.

19 . The method of claim 16 , the method further comprising:

receiving a request for synthesized speech, the request being associated with code-mixed input of a speech synthesis client;

using the code-mixed speech model and the inference engine, generating an instance of code-mixed synthesized speech; and

communicating the instance of the code-mixed synthesized speech for output on an interface associated with the speech synthesis client.

20 . The method of claim 19 , the method further comprising:

receiving an indication to update to a new target speaker;

receiving a second request for a second instance of code-mixed synthesized speech;

using the code-mixed speech model and the inference engine, generating a second instance of code-mixed synthesized speech associated with the new target speaker; and

communicating the second instance code-mixed synthesized speech for output on an interface associated with the speech synthesis client.