Method for Providing a Sign-Language Avatar Video for a Primary Video
An embodiment provides a software system capable of reading an audio file or a transcript and converting it into a sequence of sign language movements. A 3D or 2D avatar animation may be generated from the sequence of sign language movements in a primary window or in a secondary window on a user's computing device (or in a virtual reality or augmented reality space) using accelerated graphical APIs To make movements appear more natural, the sequence of gestures/movements may be generated through an AI model, or a combination of natural language analysis and an AI model, to smooth out any possible transition across the gestures and adapt it to the viewing condition.
1 . A method for providing a sign-language avatar video for a primary video, comprising:
providing a video and a transcription of spoken audio from the video;
converting the transcription into a sequence of sign language instructions including one or both of gesture or movement instructions associated with word-for-word translation of the transcription into sign language and any associated sign language grammar, including sign order;
transmitting the video and sign language instructions to a user's computing device;
displaying the video by the user's computing device in a primary video window;
generating an avatar animation from the sign language instructions in the primary window or in a secondary window by the user's computing device using accelerated graphical APIs.
2 . The method of claim 1 , wherein the sequence of sign language instructions is generated by an AI model.
3 . The method of claim 1 , wherein the sequence of sign language instructions is generated by static conversion or translation.
4 . The method of claim 1 , wherein the avatar animation is on a 3D humanoid model.
5 . The method of claim 1 , wherein an avatar used in the avatar animation is customizable on the user's computing device.
6 . The method of claim 5 , wherein the user's computing device provides the user with a selection of available avatars from which to select from.
7 . The method of claim 1 , wherein an animation format for the avatar animation is based on a list of qualifying properties that describe a 3D scene and a humanoid avatar, including textures, meshes, materials, expressions, and armature.
8 . The method of claim 1 , wherein:
the video is processed on transcoding pipeline of a server; and
the converting step is performed on a parallel computing path to the transcoding pipeline.
9 . A method for providing a sign-language avatar video for a primary video, comprising:
receiving on a user's computing device a primary video and secondary sign-language instructions associated with spoken audio in the primary video, the secondary sign-language instructions including one or both of gesture or movement instructions;
displaying the primary video on the user's computing device in a primary video window;
generating an avatar animation from the secondary sign-language instructions in either the primary video window or in a secondary window on the user's computing device using GPU processing calls.
10 . A method for providing a sign-language avatar in a display, comprising:
receiving on a user's computing device or system an audio input;
converting speech in the audio input into a transcription of speech;
converting the transcription into a sequence of sign language instructions, the sign-language instructions including one or both of gesture or movement instructions associated with word-for-word translation of the transcription into sign language and any associated sign language grammar, including sign order; and
generating an animation from the sequence of sign-language instructions in a display portion of the user's computing device.
11 . The method of claim 10 , wherein the animation is generated using accelerated graphical APIs.
12 . The method of claim 10 , wherein the sequence of sign language instructions is generated by an AI model.
13 . The method of claim 10 , wherein the animation is either a 3D humanoid model or a 2D humanoid model.
14 . The method of claim 13 , wherein the animation is an avatar animation, which is customizable on at least one of the user's computing device or on a video-creator's configuration options.
15 . The method of claim 14 , wherein the avatar animation is customizable on the user's computing device, which provides a user with a selection of available avatars from which to select from.
16 . The method of claim 10 , wherein the audio input is extracted from a video input.
17 . The method of claim 16 , wherein the steps occur in real time or near real time.
18 . The method of claim 10 , wherein the steps occur in real time or near real time.
19 . The method of claim 10 , wherein the user's computing device or system comprises an augmented reality device.
20 . The method of claim 10 , wherein the user's computing device or system comprises a virtual reality device.
21 . One or more non-transitory memory devices including computer instructions for controlling one or more computer processors to perform the steps of:
receiving on a user's computing device a primary video and secondary sign-language instructions associated with spoken audio in the primary video, the secondary sign-language instructions including one or both of gesture or movement instructions;
displaying the primary video on the user's computing device in a primary video window;
generating an avatar animation from the secondary sign-language instructions in either the primary video window or in a secondary window on the user's computing device using GPU processing calls.
22 . One or more non-transitory memory devices including computer instructions for controlling one or more computer processors to perform the steps of:
receiving on a user's computing device or system an audio input;
converting speech in the audio input into a transcription of speech;
converting the transcription into a sequence of sign language instructions, the sign-language instructions including one or both of gesture or movement instructions; and
generating an animation from the sequence of sign-language instructions in a display portion of the user's computing device.
23 . The one or more non-transitory memory devices of claim 22 , wherein the animation is generated using accelerated graphical APIs.
24 . The one or more non-transitory memory devices of claim 22 , wherein the sequence of sign language instructions is generated by an AI model.
25 . The one or more non-transitory memory devices of claim 22 , wherein the animation is either a 3D humanoid model or a 2D humanoid model.
26 . The one or more non-transitory memory devices of claim 25 , wherein the animation is an avatar animation, which is customizable on the user's computing device or system.
27 . The one or more non-transitory memory devices of claim 26 , wherein the avatar animation is customizable on the user's computing device, which provides a user with a selection of available avatars from which to select from.
28 . The one or more non-transitory memory devices of claim 22 , wherein the audio input is extracted from a video input.
29 . The one or more non-transitory memory devices of claim 22 , wherein the processing steps occur in real time or near real time.