Coarticulation method for audio-visual text-to-speech synthesis
View Patent ↗A method for generating animated sequences of talking heads in text-to-speech applications wherein a processor samples a plurality of frames comprising image samples. The processor reads first data comprising one or more parameters associated with noise-producing orifice images of sequences of at least three concatenated phonemes which correspond to an input stimulus. The processor reads, based on the first data. second data comprising images of a noise-producing entity. The processor generates an animated sequence of the noise-producing entity.
1. A method of generating a noise-producing entity, the method comprising:
receiving a stimulus representing data that the noise-producing entity will track;
associating the stimulus with at least one phoneme or phoneme sequence recalling all mouth parameters corresponding to the associated at least one phoneme or phoneme sequence;
selecting a parameter set from an animation library, the parameter set representing frame segments, the selected parameter set corresponding to the recalled mouth parameters; and
outputting speech associated with the stimulus in synchronization with outputting frame segments associated with the parameter set.
2. The method of claim 1 , wherein outputting frame segment further comprises overlaying frame segments on a larger entity to synthesize a whole animated image.
3. The method of claim 1 , wherein the stimulus is text.
4. The method of claim 1 , wherein the speech is output using a phoneme transcript stored in a coarticulation library.
5. The method of claim 1 , wherein the method is iteratively applied to phoneme sequences in the stimulus to form a complete animation.
6. The method of claim 1 , wherein the parameter set is associated with images of at least three concatenated phonemes which correspond to the stimulus.
7. A system for generating a noise-producing entity, the system comprising:
a processor;
a module configured to control the processor to receive a stimulus representing data that the noise-producing entity will track;
a module configured to control the processor to associate the stimulus with at least one phoneme or phoneme sequence;
a module configured to control the processor to recall all mouth parameters corresponding to the associated at least one phoneme or phoneme sequence;
a module configured to control the processor to select a parameter set from an animation library, the parameter set representing frame segments, the selected parameter set corresponding to the recalled mouth parameters; and
a module configured to control the processor to output speech associated with the stimulus in synchronization with outputting frame segments associated with the parameter set.
8. The system of claim 7 , wherein the module configured to control the processor to output frame segments further overlays frame segments on a larger entity to synthesize a whole animated image.
9. The system of claim 7 , wherein the stimulus is text.
10. The system of claim 7 , wherein the speech is output using a phoneme transcript stored in a coarticulation library.
11. The system of claim 7 , wherein the modules configured to control the processor iteratively operate on phoneme sequences in the stimulus to form a complete animation.
12. The system of claim 7 , wherein the parameter set is associated with images of at least three concatenated phonemes which correspond to the stimulus.
13. A computer-readable medium storing instructions for controlling a computing device to generate a noise-producing entity, the instructions comprising:
receiving a stimulus representing data that the noise-producing entity will track;
associating the stimulus with at least one phoneme or phoneme sequence recalling all mouth parameters corresponding to the associated at least one phoneme or phoneme sequence;
selecting a parameter set from an animation library, the parameter set representing frame segments, the selected parameter set corresponding to the recalled mouth parameters; and
outputting speech associated with the stimulus in synchronization with outputting frame segments associated with the parameter set.
14. The computer-readable medium of claim 13 , wherein outputting frame segment further comprises overlaying frame segments on a larger entity to synthesize a whole animated image.
15. The computer-readable medium of claim 13 , wherein the stimulus is text.
16. The computer-readable medium of claim 13 , wherein the speech is output using a phoneme transcript stored in a coarticulation library.
17. The computer-readable medium of claim 13 , wherein the instructions are iteratively applied to phoneme sequences in the stimulus to form a complete animation.
18. The computer-readable medium of claim 13 , wherein the parameter set is associated with images of at least three concatenated phonemes which correspond to the stimulus.