User-customized synthetic voice

1. A computer-implemented method, comprising:

receiving, from a first device, a first user input representing a natural language description of a desired synthetic voice;

processing, using a first encoder, the first user input to determine synthetic voice description embedding data representing the natural language description of the desired synthetic voice;

determining, using the synthetic voice description embedding data, first synthetic voice embedding data corresponding to a first proposed synthetic voice;

processing, using a decoder, the first synthetic voice embedding data to determine first synthetic voice characteristics data;

generating, using the first synthetic voice characteristics data and text data representing words, first output audio data representing first synthetic speech corresponding to the first proposed synthetic voice saying the words;

causing the first device to output the first output audio data;

receiving a second user input representing a user satisfaction corresponding to the first proposed synthetic voice;

based at least in part on the user satisfaction and the first synthetic voice embedding data, generating first data representing a first probability that second synthetic voice embedding data corresponding to a second proposed synthetic voice will result in higher user satisfaction than third synthetic voice embedding data corresponding to a third proposed synthetic voice;

based at least in part on the first data, selecting the second synthetic voice embedding data instead of the third synthetic voice embedding data;

processing, using the decoder, the second synthetic voice embedding data to determine second synthetic voice characteristics data;

generating, using the second synthetic voice characteristics data and the text data, second output audio data representing second synthetic speech corresponding to the second proposed synthetic voice saying the words; and

causing the first device to output the second output audio data.

2. The computer-implemented method of claim 1 , further comprising:

determining first profile data associated with the first user input;

processing, using a second encoder, the first profile data to determine profile embedding data;

processing, using a third encoder, the synthetic voice description embedding data and the profile embedding data to determine first synthetic voice preference embedding data representing the natural language description of the desired synthetic voice and the first profile data;

determining, using the first synthetic voice preference embedding data, from among a plurality of synthetic voice preference embedding data, second synthetic voice preference embedding data corresponding to a first user;

determining a similarity between the first synthetic voice preference embedding data and the second synthetic voice preference embedding data; and

based at least in part on the similarity, determining fourth synthetic voice embedding data corresponding to the first user, the fourth synthetic voice embedding data corresponding to a first previously configured synthetic voice,

wherein the first synthetic voice embedding data corresponds to the fourth synthetic voice embedding data.

3. The computer-implemented method of claim 2 , further comprising:

determining, using the first synthetic voice preference embedding data, from among the plurality of synthetic voice preference embedding data, third synthetic voice preference embedding data corresponding to a second user;

determining fifth synthetic voice embedding data corresponding to the second user, the fifth synthetic voice embedding data corresponding to a second previously configured synthetic voice; and

processing the fifth synthetic voice embedding data and the fourth synthetic voice embedding data to determine sixth synthetic voice embedding data, the sixth synthetic voice embedding data corresponding to an average between the fifth synthetic voice embedding data and the fourth synthetic voice embedding data,

wherein the second synthetic voice embedding data corresponds to the sixth synthetic voice embedding data.

4. The computer-implemented method of claim 1 , wherein the first data further represents a second probability that fourth synthetic voice embedding data corresponding to a fourth proposed synthetic voice will result in higher user satisfaction than fifth synthetic voice embedding data corresponding to a fifth proposed synthetic voice, and selecting the second synthetic voice embedding data comprises:

based at least in part on the first data, selecting the fourth synthetic voice embedding data instead of the fifth synthetic voice embedding data; and

processing the fourth synthetic voice embedding data and the second synthetic voice embedding data to determine sixth synthetic voice embedding data, the sixth synthetic voice embedding data corresponding to an average between the fourth synthetic voice embedding data and the second synthetic voice embedding data, the sixth synthetic voice embedding data corresponding to a sixth proposed synthetic voice.

5. A computer-implemented method comprising:

receiving a first user input representing a description of a desired synthetic voice;

processing the first user input to determine synthetic voice description embedding data;

determining, based at least in part on the synthetic voice description embedding data, first synthetic voice embedding data corresponding to a first proposed synthetic voice;

processing the first synthetic voice embedding data to determine first synthetic voice characteristics data;

performing speech synthesis processing using the first synthetic voice characteristics data to determine first output audio data representing first speech corresponding to the first proposed synthetic voice;

causing output of the first output audio data;

receiving a second user input corresponding to the first proposed synthetic voice;

based at least in part on the second user input and the first synthetic voice embedding data, determining second synthetic voice embedding data corresponding to a second proposed synthetic voice;

processing the second synthetic voice embedding data to determine second synthetic voice characteristics data;

performing speech synthesis processing using the second synthetic voice characteristics data to determine second output audio data representing second speech corresponding to the second proposed synthetic voice; and

causing output of the second output audio data.

6. The computer-implemented method of claim 5 , further comprising:

determining first profile data associated with the first user input;

processing the first profile data to determine profile embedding data;

processing the synthetic voice description embedding data and the profile embedding data to determine first synthetic voice preference embedding data; and

determining, using the first synthetic voice preference embedding data, from among a plurality of synthetic voice preference embedding data, third synthetic voice embedding data corresponding to a first user, the third synthetic voice embedding data corresponding to a previously configured synthetic voice,

wherein the first synthetic voice embedding data corresponds to the third synthetic voice embedding data.

7. The computer-implemented method of claim 6 , wherein determining, from among the plurality of synthetic voice preference embedding data, the third synthetic voice embedding data comprises:

determining a similarity between the first synthetic voice preference embedding data and second synthetic voice preference embedding data corresponding to the first user,

wherein determining the first synthetic voice embedding data is based at least in part on the similarity.

8. The computer-implemented method of claim 5 , further comprising:

storing first data corresponding to the second proposed synthetic voice;

after storing the first data, receiving a third user input corresponding to a user request;

processing the third user input to determine output data responsive to the user request;

determining the user request is associated with the first data; and

performing speech synthesis processing using the output data and the second synthetic voice characteristics data to determine third output audio data representing a synthetic speech response to the user request in the second proposed synthetic voice.

9. The computer-implemented method of claim 5 , wherein determining the second synthetic voice embedding data comprises:

based at least in part on the second user input and the first synthetic voice embedding data, determining to select third synthetic voice embedding data corresponding to a third proposed synthetic voice instead of fourth synthetic voice embedding data corresponding to a fourth proposed synthetic voice; and

processing the third synthetic voice embedding data and the second synthetic voice embedding data to determine fifth synthetic voice embedding data corresponding to a fifth proposed synthetic voice, the fifth synthetic voice embedding data corresponding to an interpolation of the third synthetic voice embedding data and the second synthetic voice embedding data,

wherein the second synthetic voice embedding data corresponds to the fifth synthetic voice embedding data.

10. The computer-implemented method of claim 5 , wherein:

the second user input represents a user satisfaction corresponding to the first proposed synthetic voice, and

determining the second synthetic voice embedding data comprises:

generating, based at least in part on the second user input and the first synthetic voice embedding data, first data representing a first probability that third synthetic voice embedding data corresponding to a third proposed synthetic voice predicted to result in higher user satisfaction than fourth synthetic voice embedding data corresponding to a fourth proposed synthetic voice; and

based at least in part on the first data, selecting the third synthetic voice embedding data instead of the fourth synthetic voice embedding data,

wherein the second synthetic voice embedding data corresponds to the third synthetic voice embedding data.

11. The computer-implemented method of claim 5 , further comprising:

prior to receiving the second user input, determining, based at least in part on the synthetic voice description embedding data, third synthetic voice embedding data corresponding to a third proposed synthetic voice;

processing the third synthetic voice embedding data to determine third synthetic voice characteristics data;

generating, using the third synthetic voice characteristics data, third output audio data representing third speech corresponding to the third proposed synthetic voice; and

causing output of the third output audio data,

wherein the second user input corresponds to the first proposed synthetic voice and not the third proposed synthetic voice.

12. The computer-implemented method of claim 5 , further comprising:

prior to determining the first synthetic voice embedding data, determining, using the synthetic voice description embedding data, third synthetic voice embedding data corresponding to a third proposed synthetic voice;

determining a first user associated with the first user input; and

determining that the third synthetic voice embedding data is associated with a profile, the profile being associated with a second user different from the first user,

wherein the first synthetic voice embedding data is determined based at least in part on the second user being different from the first user.

13. A computing system comprising:

at least one processor; and

at least one memory comprising instructions that, when executed by the at least one processor, cause the computing system to:

receive a first user input representing a description of a desired synthetic voice;

process the first user input to determine synthetic voice description embedding data;

determine, based at least in part on the synthetic voice description embedding data, first synthetic voice embedding data corresponding to a first proposed synthetic voice;

process the first synthetic voice embedding data to determine first synthetic voice characteristics data;

perform speech synthesis processing using the first synthetic voice characteristics data to determine first output audio data representing first speech corresponding to the first proposed synthetic voice;

cause output of the first output audio data;

receive a second user input corresponding to the first proposed synthetic voice;

based at least in part on the second user input and the first synthetic voice embedding data, determine second synthetic voice embedding data corresponding to a second proposed synthetic voice;

process the second synthetic voice embedding data to determine second synthetic voice characteristics data;

generate, using the second synthetic voice characteristics data, second output audio data representing second speech corresponding to the second proposed synthetic voice; and

cause output of the second output audio data.

14. The computing system of claim 13 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to:

determine first profile data associated with the first user input;

process the first profile data to determine profile embedding data;

process the synthetic voice description embedding data and the profile embedding data to determine first synthetic voice preference embedding data; and

determine, using the first synthetic voice preference embedding data, from among a plurality of synthetic voice preference embedding data, third synthetic voice embedding data corresponding to a first user, the third synthetic voice embedding data corresponding to a previously configured synthetic voice,

wherein the first synthetic voice embedding data corresponds to the third synthetic voice embedding data.

15. The computing system of claim 14 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, cause the computing system to determine, from among the plurality of synthetic voice preference embedding data, the third synthetic voice embedding data further cause the computing system to:

determine a similarity between the first synthetic voice preference embedding data and second synthetic voice preference embedding data corresponding to the first user,

wherein determining the first synthetic voice embedding data is based at least in part on the similarity.

16. The computing system of claim 14 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to:

determine, using the first synthetic voice preference embedding data, from among the plurality of synthetic voice preference embedding data, fourth synthetic voice embedding data corresponding to a second user, the fourth synthetic voice embedding data corresponding to a second previously configured synthetic voice; and

process the third synthetic voice embedding data and the fourth synthetic voice embedding data to determine fifth synthetic voice embedding data corresponding to a third proposed synthetic voice, the fifth synthetic voice embedding data corresponding to an interpolation of the third synthetic voice embedding data and the fourth synthetic voice embedding data,

wherein the first synthetic voice embedding data corresponds to the fifth synthetic voice embedding data.

17. The computing system of claim 13 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, cause the computing system to:

store first data corresponding to the second proposed synthetic voice;

after storing the first data, receive a third user input corresponding to a user request;

process the third user input to determine output data responsive to the user request;

determine the user request is associated with the first data; and

perform speech synthesis processing using the output data and the second synthetic voice characteristics data to determine third output audio data representing a synthetic speech response to the user request in the second proposed synthetic voice.

18. The computing system of claim 13 , wherein:

the second user input represents a user satisfaction corresponding to the first proposed synthetic voice, and

wherein the at least one memory further comprises instructions that, when executed by the at least one processor, cause the computing system to generate the second synthetic voice embedding data further cause the computing system to:

determine, based at least in part on the second user input and the first synthetic voice embedding data, first data representing a first probability that third synthetic voice embedding data corresponding to a third proposed synthetic voice predicted to result in higher user satisfaction than fourth synthetic voice embedding data corresponding to a fourth proposed synthetic voice; and

based at least in part on the first data, select the third synthetic voice embedding data instead of the fourth synthetic voice embedding data,

wherein the second synthetic voice embedding data corresponds to the third synthetic voice embedding data.

19. The computing system of claim 13 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to:

prior to receiving the second user input, determine, based at least in part on the synthetic voice description embedding data, third synthetic voice embedding data corresponding to a third proposed synthetic voice;

process the third synthetic voice embedding data to determine third synthetic voice characteristics data;

generate, using the third synthetic voice characteristics data, third output audio data representing third speech corresponding to the third proposed synthetic voice; and

cause output of the third output audio data,

wherein the second user input corresponds to the first proposed synthetic voice and not the third proposed synthetic voice.

20. The computing system of claim 13 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to:

prior to determining the first synthetic voice embedding data, determine, using the synthetic voice description embedding data, third synthetic voice embedding data corresponding to a third proposed synthetic voice;

determine a first user associated with the first user input; and

determine that the third synthetic voice embedding data is associated with a profile, the profile being associated with a second user different from the first user,

wherein the first synthetic voice embedding data is determined based at least in part on the second user being different from the first user.