Coarticulation method for audio-visual text-to-speech synthesis
View Patent ↗A method for generating animated sequences of talking heads in text-to-speech applications wherein a processor samples a plurality of frames comprising image samples. The processor reads first data comprising one or more parameters associated with noise-producing orifice images of sequences of at least three concatenated phonemes which correspond to an input stimulus. The processor reads, based on the first data, second data comprising images of a noise-producing entity. The processor generates an animated sequence of the noise-producing entity.
1. A computing device for generating a noise-producing entity, comprising:
a module configured to read first data comprising one or more parameters associated with noise-producing orifice images of sequences of at least three concatenated phonemes which correspond to an input stimulus;
a module configured to read, based on the first data, corresponding second data comprising images of a noise-producing entity; and
a module configured to generate, using the second data, an animated sequence of the noise-producing entity tracking the input stimulus.
2. The computing device of claim 1 , further comprising:
a module configured to read acoustic data associated with the second data;
a module configured to convert the acoustic data into sound; and
a module configured to output the sound synchronously with the animated sequence of the noise-producing entity.
3. The computing device of claim 1 , wherein the first data comprises one or more equations characterizing noise-producing orifice shapes.
4. The computing device of claim 3 , wherein the first data comprises one or more equations characterizing noise-producing orifice shapes.
5. The computing device of claim 2 , wherein the module configured to convert further uses a data-to-sound converter.
6. The computing device of claim 2 , wherein the first data comprises segments of sampled images of a noise-producing subject.
7. The computing device of claim 2 , wherein the second data comprises parameters associated with a noise-producing orifice degree of opening.
8. The computing device of claim 6 , wherein the first data comprises animation data, and the second data comprises coarticulation data.
9. The computing device of claim 6 , wherein the module configured to generate further overlays the segments onto a common interface to create frames comprising the animation sequence.
10. A method of generating an animated entity, the method comprising:
reading first data associated with noise-producing orifice images of sequences of at least three concatenated phonemes;
reading second data comprising images associated with an animated entity; and
generating the animated entity based on the first data and the second data.
11. The method of claim 10 , wherein the first data corresponds to an input stimulus and the animated entity is generated to track the input stimulus.
12. The method of claim 10 , wherein the animated entity is a noise producing entity.
13. The method of claim 10 , further comprising:
reading acoustic data associated with the sound data;
converting the acoustic data into second; and
outputting the sound synchronously with the generation of the animated entity.
14. A computer-readable medium storing instructions for controlling a computing device to generate an animated entity, the instructions comprising:
reading first data associated with noise-producing orifice images of sequences of at least three concatenated phonemes;
reading second data comprising images associated with animated entity; and
generating the animated entity based on the first data and the second data.
15. The computer-readable medium of claims 14 , wherein the first data corresponds to an input stimulus and the animated entity is generated to track the input stimulus.
16. The computer-readable medium of claim 14 , wherein the animated entity is a noise producing entity.
17. The computer-readable medium of claim 14 , the instructions further comprising:
reading acoustic data associated with the second data;
converting the acoustic data into sound; and
outputting the sound synchronously with the generation of the animated entity.