Method and apparatus for providing an animated display with translated speech
View Patent ↗A first language to second language translator ( 11 ) converts text comprising a message in a first language into text representing that message in a second language. A text to speech transducer ( 12 ) then synthesizes an audiblized version of the text in the second language. An animated display driver ( 14 ) creates a corresponding display ( 15 ) of an image ( 16 ) that includes a mouth ( 17 ) that moves appropriately and synchronously with respect to the audiblized second language. In other embodiments, other facial features can be animated as well. Such animation can be random and unrelated to the appearance of the original speaker. Or, such animation can accurately represent at least some alterations to facial features of the original speaker while speaking. In some embodiments, the facial features can be altered by correlating specific detected alterations with specific corresponding substantive content in the original message and the translated text.
1. A method comprising:
capturing speech in a first language being uttered by a person having a face;
detecting alteration of at least a part of the face, other than a movement of a mouth, and correlating such alteration to specific words in the speech to provide detected alterations converting the speech into text;
translating the text into a second language that is different from the first language to provide translated speech;
rendering the translated speech audible to provide audiblized translated speech;
while rendering the translated speech audible, substantially synchronously providing a display of at least an animated representation of the face wherein:
the face has a mouth that moves synchronously with the audiblized translated speech; and
the face has at least one other part that is altered synchronously with the detected alteration;
such that the at least animated representation of the face presents a mouth that moves in substantial accord with the audiblized translated speech and at least one part of the face that is altered in substantive accord with facial alterations as evinced by the person when uttering the speech.
2. The method of claim 1 wherein synchronously altering the at least one other part includes altering the at least one other part during a particular part of the audiblized translated speech in a way that is at least substantially similar to a detected alteration that correlates to the specific words having substantive content that substantially matches the particular part of the audiblized translated speech.
3. The method of claim 2 and further including identifying at least one category of facial alteration and wherein substantially synchronously providing a display of at least an animated representation of the face includes not altering the animated representation of the face to mimic the at least one category of facial alteration.
4. The method of claim 1 wherein detecting alteration of at least a part of the face includes detecting alteration of at least one of an eye, a part of an eye, an eyelid, an eyebrow, a cheek, a nose, a part of a nose, a forehead, an ear, and a chin.
5. The method of claim 1 wherein providing a display of at least an animated representation of the face includes providing a display of at least an animated three-dimensional representation of the face.
6. The method of claim 1 wherein rendering the translated speech audible includes selecting from amongst a plurality of synthesized speech voices.
7. The method of claim 1 wherein providing a display of at least an animated representation of the face wherein the face has a mouth that moves synchronously with the translated speech includes moving the mouth, at least from time to time, on a phoneme by phoneme basis with the audiblized translated speech.
8. A method comprising:
capturing speech in a first language being uttered by a person having a face;
detecting alteration of at least a part of the face, other than a movement of a mouth, and correlating such alteration to specific words in the speech to provide detected alterations converting the speech into text;
translating the text into a second language that is different from the first language to provide translated speech;
rendering the translated speech audible to provide audiblized translated speech;
while rendering the translated speech audible, providing a display of at least an animated representation of a second face wherein:
the second face has a mouth that moves synchronously with the audiblized translated speech; and
the second face has at least one other part that is altered synchronously with the detected alteration;
such that the animated representation of the second face presents a mouth that moves in substantial accord with the audiblized translated speech and the at least one other part of the second face is altered in substantive accord with facial alterations as evinced by the person when uttering the speech and in correlation to specific words in the audiblized translated speech having substantive content that substantially matches the specific words in the speech to which the facial alterations are correlated.
9. The method of claim 8 and further including identifying at least one category of facial alteration and wherein providing a display of at least an animated representation of the second face includes not altering the animated representation of the face to mimic the at least one category of facial alteration.
10. The method of claim 8 wherein detecting alteration of at least a part of the face includes detecting alteration of at least one of an eye, a part of an eye, an eyelid, an eyebrow, a cheek, a nose, a part of a nose, a forehead, an ear, and a chin.
11. The method of claim 8 wherein the display of at least the animated representation of the second face does not correspond to the face of the person uttering the speech in the first language.
12. The method of claim 8 wherein the display of at least the animated representation of the second face does correspond to the face of the person uttering the speech in the first language.
13. The method of claim 8 wherein providing a display of at least an animated representation of the second face includes providing a display of at least an animated three-dimensional representation of the face.
14. The method of claim wherein rendering the translated speech audible includes selecting from amongst a plurality of synthesized speech voices.
15. The method of claim 8 wherein providing a display of at least an animated representation of the second face wherein the second face has a mouth that moves with the translated speech includes moving the mouth, at least from time to time, on a phoneme by phoneme basis with the audiblized translated speech.