Sentiment-based interactive avatar system for sign language

1. A method comprising:

receiving a content item comprising a video and an audio, wherein the audio comprises a first plurality of spoken words in a first language, the video comprises a character speaking the first plurality of spoken words in the first language;

translating the first plurality of spoken words of the first language into a first sign of a first sign language;

determining an emotional state expressed by the character based on sentiment analysis; generating an avatar that performs the first sign of the first sign language, the avatar exhibiting the determined emotional state;

querying a sign language database for a word of the first plurality of spoken words to identify the first sign of the first sign language, wherein the sign language database comprises visual content showing the first sign;

based on the visual content of the first sign, determining one or more skeleton models that apply to the first sign for configuring the avatar to perform the first sign; and

performing a transformation of the avatar using the determined one or more skeleton models, wherein performing the transformation comprises modifying one or more joints of the avatar based on the visual content of the first sign; and

causing the content item and the avatar for display on a first device.

2. The method of claim 1 , wherein the transformation of the avatar is based on a movement of at least one of a hand, a finger, an arm, or a face of the avatar.

3. The method of claim 1 , further comprising:

converting the first plurality of spoken words of the first language into a first text using one or more speech recognition algorithms, the first text comprises one or more words corresponding to the first plurality of spoken words.

4. The method of claim 1 , wherein the sentiment analysis is performed by:

determining an emotion identifier contained in the first plurality of spoken words; determining a facial expression or a body expression of the character using one or more expression recognition algorithms; and

determining a vocal tone of the character using one or more voice recognition algorithms.

5. The method of claim 1 , further comprising:

receiving user input specifying a visual characteristic of the avatar, wherein an appearance of the avatar is modified based on the specified visual characteristic.

6. The method of claim 1 , further comprising:

identifying a visual characteristic of the character in the video based on image analysis, wherein an appearance of the avatar is modified based on the identified visual characteristic of the character in the video.

7. The method of claim 1 , further comprising:

receiving a user request to transmit the avatar from the first device to a second device;

transmitting a configuration file that includes a visual characteristic of the avatar to the second device different from the first device; and

causing the avatar for display based on the transmitted configuration file on the second device.

8. The method of claim 1 , further comprising:

receiving a user input specifying a command for the avatar to perform from the first device;

determining a second sign corresponding to the command; and

generating the avatar performing the second sign of the first sign language.

9. A method comprising:

receiving user input comprising audio input and video input from a first device, the audio input comprising a first plurality of spoken words in a first language in proximity to the first device, the video input comprising an image of a user while speaking the first plurality of spoken words;

translating the first plurality of spoken words of the first language into a first sign of a first sign language;

determining an emotional state expressed by the user based on sentiment analysis;

generating an avatar that performs the first sign of the first sign language, the avatar exhibiting the determined emotional state of the user;

querying a sign language database for a word of the first plurality of spoken words to identify the first sign of the first sign language, wherein the sign language database comprises visual content showing the first sign;

based on the visual content of the first sign, determining one or more skeleton models that apply to the first sign for configuring the avatar to perform the first sign; and

performing a transformation of the avatar using the determined one or more skeleton models, wherein performing the transformation comprises modifying one or more joints of the avatar based on the visual content of the first sign; and

causing the avatar for display on the first device.

10. The method of claim 9 , wherein the transformation of the avatar is based on a movement of at least one of a hand, a finger, an arm, or a face of the avatar.

11. The method of claim 9 , further comprising:

converting the first plurality of spoken words of the first language into a first text using one or more speech recognition algorithms, the first text comprises one or more words corresponding to the first plurality of spoken words.

12. The method of claim 9 , wherein the sentiment analysis is performed by:

determining an emotion identifier contained in the first plurality of spoken words; determining a facial expression or a body expression of the user using one or more expression recognition algorithms; and

determining a vocal tone of the user using one or more voice recognition algorithms.

13. The method of claim 9 , further comprising:

receiving user input specifying a visual characteristic of the avatar, wherein an appearance of the avatar is modified based on the specified visual characteristic.

14. The method of claim 9 , further comprising:

receiving a user request to transmit the avatar from the first device to a second device; transmitting a configuration file that includes a visual characteristic of the avatar to the second device different from the first device; and

causing the avatar for display based on the transmitted configuration file on the second device.

15. The method of claim 9 , wherein the audio input is received at the first device via a microphone of the first device and the video input is received at the first device via a camera of the first device.

16. The method of claim 9 , wherein the avatar is automatically generated in real time.