System and method for synthetic voice generation and modification
View Patent ↗Disclosed herein are systems, methods, and non-transitory computer-readable storage media for generating a synthetic voice. A system configured to practice the method combines a first database of a first text-to-speech voice and a second database of a second text-to-speech voice to generate a combined database, selects from the combined database, based on a policy, voice units of a phonetic category for the synthetic voice to yield selected voice units, and synthesizes speech based on the selected voice units. The system can synthesize speech without parameterizing the first text-to-speech voice and the second text-to-speech voice. A policy can define, for a particular phonetic category, from which text-to-speech voice to select voice units. The combined database can include multiple text-to-speech voices from different speakers. The combined database can include voices of a single speaker speaking in different styles. The combined database can include voices of different languages.
1. A method comprising:
storing, in a database, voice data, wherein the voice data is associated with a plurality of voices, wherein the plurality of voices are stored within libraries according to emotions;
identifying, using user speech exhibited by a user, a user emotion;
identifying, via a processor and according to the user emotion, a first text-to-speech voice of the plurality of voices which are in the database, wherein the first text-to-speech voice has a first emotional content from a first speaker;
identifying, via the processor and according to the user emotion, a second text-to-speech voice of the plurality of voices which are in the database, wherein the second text-to-speech voice has a second emotional content from a second speaker, and wherein the second emotional content is distinct from the first emotional content; and
synthesizing synthesized speech using the first text-to-speech voice and the second text-to-speech voice, wherein the synthesized speech mimics the user emotion.
2. The method of claim 1 , further comprising generating a plurality of synthetic voices from the database, wherein each synthetic voice in the plurality of synthetic voices is generated according to a respective selection policy.
3. The method of claim 1 , wherein the first emotional content is determined according to a speech act associated with the user speech.
4. The method of claim 1 , wherein the synthesized speech is synthesized using the first emotional content.
5. The method of claim 1 , wherein the synthesized speech is synthesized using selected voice units from the database, wherein the selected voice units comprise a first voice unit from the first text-to-speech voice, and comprise a second voice unit from the second text-to-speech voice.
6. The method of claim 5 , wherein the synthesized speech is synthesized without parameterizing the first text-to-speech voice and without parameterizing the second text-to-speech voice.
7. The method of claim 1 , wherein a selection policy defines, for a particular phonetic category, from which text-to-speech voice to synthesize the synthesized speech.
8. The method of claim 1 , wherein the first text-to-speech voice and the second text-to-speech voice are in distinct styles of speaking.
9. The method of claim 1 , wherein the first text-to-speech voice and the second text-to-speech voice have a similar pitch range and fundamental frequency with respect to one another.
10. A system comprising:
a processor; and
a computer-readable storage medium having instructions stored which, when executed by the processor, result in the processor performing operations comprising:
storing, in a database, voice data, wherein the voice data is associated with a plurality of voices, wherein the plurality of voices are stored within libraries according to emotions;
identifying, using user speech exhibited by a user, a user emotion;
identifying, according to the user emotion, a first text-to-speech voice of the plurality of voices which are in the database, wherein the first text-to-speech voice has a first emotional content from a first speaker;
identifying, according to the user emotion, a second text-to-speech voice of the plurality of voices which are in the database, wherein the second text-to-speech voice has a second emotional content from a second speaker, and wherein the second emotional content is distinct from the first emotional content; and
synthesizing synthesized speech using the first text-to-speech voice and the second text-to-speech voice, wherein the synthesized speech mimics the user emotion.
11. The system of claim 10 , the computer-readable storage medium having additional instructions stored which, when executed by the processor, result in operations comprising generating a plurality of synthetic voices from the database, wherein each synthetic voice in the plurality of synthetic voices is generated according to a respective selection policy.
12. The system of claim 10 , wherein the first emotional content is determined according to a speech act associated with the user speech.
13. The system of claim 10 , wherein the synthesized speech is synthesized using the first emotional content.
14. The system of claim 10 , wherein the synthesized speech is synthesized using selected voice units from the database, wherein the selected voice units comprise a first voice unit from the first text-to-speech voice, and comprise a second voice unit from the second text-to-speech voice.
15. The system of claim 14 , wherein the synthesized speech is synthesized without parameterizing the first text-to-speech voice and without parameterizing the second text-to-speech voice.
16. The system of claim 10 , wherein a selection policy defines, for a particular phonetic category, from which text-to-speech voice to synthesize the synthesized speech.
17. The system of claim 10 , wherein the first text-to-speech voice and the second text-to-speech voice are in distinct styles of speaking.
18. A device having instructions stored which, when executed by a computing device, result in the computing device performing operations comprising:
storing, in a database, voice data, wherein the voice data is associated with a plurality of voices, wherein the plurality of voices are stored within libraries according to emotions;
identifying, using user speech exhibited by a user, a user emotion; identifying, according to the user emotion, a first text-to-speech voice of the plurality of voices which are in the database, wherein the first text-to-speech voice has a first emotional content from a first speaker;
identifying, according to the user emotion, a second text-to-speech voice of the plurality of voices which are in the database, wherein the second text-to-speech voice has a second emotional content from a second speaker, and wherein the second emotional content is distinct from the first emotional content; and
synthesizing synthesized speech using the first text-to-speech voice and the second text-to-speech voice, wherein the synthesized speech mimics the user emotion.
19. The device of claim 18 , having additional instructions stored which, when executed by the processor, result in operations comprising generating a plurality of synthetic voices from the database, wherein each synthetic voice in the plurality of synthetic voices is generated according to a respective selection policy.
20. The device of claim 18 , wherein the first emotional content is determined according to a speech act associated with the user speech.