System and method for generating video in target language
One or more computing devices, systems, and/or methods for generating a video in a target language are provided. In an example, a first video, in which a first speaker speaks in a first language, is identified. A translated transcript in a second language is determined. The translated transcript is indicative of a translation of speech spoken by the first speaker in the first video. Based upon the translated transcript and a speaker profile associated with a second speaker, first audio, including an auditory representation of the translated transcript being spoken in a voice of the second speaker, is generated. Based upon the first video and the first audio, a second video, in which mouth movements of the first speaker are aligned with speech of the auditory representation of the first audio, is generated.
1 . A method, comprising:
identifying a first video;
determining a transcript indicative of speech spoken by a speaker in the first video;
translating the transcript from a first language to a second language to generate a translated transcript in the second language;
determining a speaker profile associated with the speaker, the speaker profile generated based upon second speech of the speaker from a first source of audio identified as being associated with the speaker during analysis of a first internet resource and third speech of the speaker from a second source of audio identified as being associated with the speaker during analysis of a second internet resource;
generating, based upon the translated transcript and the speaker profile associated with the speaker, first audio comprising an auditory representation of the translated transcript being spoken in a voice of the speaker; and
generating, based upon the first video and the first audio, a second video in which mouth movements of the speaker are aligned with speech of the auditory representation of the first audio, wherein generating the second video comprises aligning each audio segment of a plurality of audio segments of the second video with a corresponding video segment of the second video.
2 . The method of claim 1 , comprising:
presenting the second video via a client device, wherein the presenting the second video comprises:
displaying the second video via a graphical user interface; and
outputting the first audio via a loudspeaker.
3 . The method of claim 1 , wherein:
the transcript comprises a plurality of text segments;
a first text segment of the plurality of text segments is associated with a first video segment of the first video;
a second text segment of the plurality of text segments is associated with a second video segment of the first video;
the translated transcript comprises a plurality of translated text segments;
a first translated text segment of the plurality of translated text segments is a translation of the first text segment; and
a second translated text segment of the plurality of translated text segments is a translation of the second text segment.
4 . The method of claim 3 , wherein:
the first text segment corresponds to one or more first sentences; and
the second text segment corresponds to one or more second sentences.
5 . The method of claim 3 , wherein:
the first audio comprises a plurality of audio segments; and
the generating the first audio comprises:
generating a first audio segment of the plurality of audio segments of the first audio based upon the first translated text segment and a duration of time of the first video segment associated with the first text segment, wherein:
the first audio segment comprises an auditory representation, of the first translated text segment, in the voice of the speaker; and
a duration of time of the first audio segment matches the duration of time of the first video segment associated with the first text segment; and
generating a second audio segment of the plurality of audio segments of the first audio based upon the second translated text segment and a duration of time of the second video segment associated with the second text segment, wherein:
the second audio segment comprises an auditory representation, of the second translated text segment, in the voice of the speaker; and
a duration of time of the second audio segment matches the duration of time of the second video segment associated with the second text segment.
6 . The method of claim 5 , wherein at least one of:
the generating the first audio segment comprises:
generating, based upon the first translated text segment, a third audio segment comprising an auditory representation, of the first translated text segment, in the voice of the speaker, wherein a duration of time of the third audio segment is longer than the duration of time of the first video segment associated with the first text segment; and
shortening the third audio segment to generate the first audio segment of the plurality of audio segments of the first audio; or
the generating the second audio segment comprises:
generating, based upon the second translated text segment, a fourth audio segment comprising an auditory representation, of the second translated text segment, in the voice of the speaker, wherein a duration of time of the fourth audio segment is shorter than the duration of time of the second video segment associated with the second text segment; and
lengthening the fourth audio segment to generate the second audio segment of the plurality of audio segments of the first audio.
7 . The method of claim 6 , wherein at least one of:
the shortening the third audio segment comprises increasing a speed of audio of the third audio segment; or
the lengthening the fourth audio segment comprises padding the fourth audio segment.
8 . The method of claim 1 , wherein the generating the second video comprises:
identifying a face of the speaker in the first video; and
modifying, based upon the first audio, pixels of the first video to generate the second video in which mouth movements, of the speaker, are aligned with the speech of the first audio, wherein the pixels of the first video that are modified correspond to a portion, of the face, comprising a mouth of the face.
9 . The method of claim 8 , wherein:
the generating the second video comprises aligning a first audio segment of the plurality of audio segments of the second video with a corresponding first video segment of the second video by lengthening the first audio segment by padding the first audio segment with silence that was not in a corresponding first audio segment in the first video.
10 . The method of claim 8 , comprising:
generating a face profile associated with the speaker based upon one or more images comprising a face of the speaker, wherein the identifying the face of the speaker in the first video comprises performing, based upon the face profile, facial recognition on the first video.
11 . The method of claim 1 , comprising:
generating the speaker profile associated with the speaker based upon audio of the first video.
12 . The method of claim 1 , wherein:
the speaker profile comprises a vector representation generated based upon the second speech of the speaker.
13 . A computing device comprising:
a processor; and
memory comprising processor-executable instructions that when executed by the processor cause performance of operations, the operations comprising:
identifying a first video;
determining a transcript indicative of speech spoken by a speaker in the first video;
translating the transcript from a first language to a second language to generate a translated transcript in the second language;
determining a speaker profile associated with the speaker, the speaker profile generated based upon second speech of the speaker from a first source of audio identified as being associated with the speaker during analysis of a first internet resource and third speech of the speaker from a second source of audio identified as being associated with the speaker during analysis of a second internet resource;
generating, based upon the translated transcript and the speaker profile associated with the speaker, first audio comprising an auditory representation of the translated transcript being spoken in a voice of the speaker; and
generating, based upon the first video and the first audio, a second video in which mouth movements of the speaker are aligned with speech of the auditory representation of the first audio, wherein generating the second video comprises aligning each audio segment of a plurality of audio segments of the second video with a corresponding video segment of the second video.
14 . The computing device of claim 13 , the operations comprising:
presenting the second video via a client device, wherein the presenting the second video comprises:
displaying the second video via a graphical user interface; and
outputting the first audio via a loudspeaker.
15 . The computing device of claim 13 , wherein:
the transcript comprises a plurality of text segments;
a first text segment of the plurality of text segments is associated with a first video segment of the first video;
a second text segment of the plurality of text segments is associated with a second video segment of the first video;
the translated transcript comprises a plurality of translated text segments;
a first translated text segment of the plurality of translated text segments is a translation of the first text segment; and
a second translated text segment of the plurality of translated text segments is a translation of the second text segment.
16 . The computing device of claim 15 , wherein:
the first audio comprises a plurality of audio segments; and
the generating the first audio comprises:
generating a first audio segment of the plurality of audio segments of the first audio based upon the first translated text segment and a duration of time of the first video segment associated with the first text segment, wherein:
the first audio segment comprises an auditory representation, of the first translated text segment, in the voice of the speaker; and
a duration of time of the first audio segment matches the duration of time of the first video segment associated with the first text segment; and
generating a second audio segment of the plurality of audio segments of the first audio based upon the second translated text segment and a duration of time of the second video segment associated with the second text segment, wherein:
the second audio segment comprises an auditory representation, of the second translated text segment, in the voice of the speaker; and
a duration of time of the second audio segment matches the duration of time of the second video segment associated with the second text segment.
17 . The computing device of claim 16 , wherein at least one of:
the generating the first audio segment comprises:
generating, based upon the first translated text segment, a third audio segment comprising an auditory representation, of the first translated text segment, in the voice of the speaker, wherein a duration of time of the third audio segment is longer than the duration of time of the first video segment associated with the first text segment; and
shortening the third audio segment to generate the first audio segment of the plurality of audio segments of the first audio; or
the generating the second audio segment comprises:
generating, based upon the second translated text segment, a fourth audio segment comprising an auditory representation, of the second translated text segment, in the voice of the speaker, wherein a duration of time of the fourth audio segment is shorter than the duration of time of the second video segment associated with the second text segment; and
lengthening the fourth audio segment to generate the second audio segment of the plurality of audio segments of the first audio.
18 . The computing device of claim 16 , wherein the generating the second video comprises:
generating a third video segment, of the second video, based upon the first audio segment of the plurality of audio segments of the first audio and the first video segment of the first video; and
generating a fourth video segment, of the second video, based upon the second audio segment of the plurality of audio segments of the first audio and the second video segment of the first video.
19 . A non-transitory machine readable medium having stored thereon processor-executable instructions that when executed cause performance of operations, the operations comprising:
identifying a first video in which a first speaker speaks in a first language;
determining a translated transcript, in a second language, indicative of a translation of speech spoken by the first speaker in the first video;
determining a speaker profile associated with a second speaker, the speaker profile generated based upon second speech of the second speaker from a first source of audio identified as being associated with the second speaker during analysis of a first internet resource and third speech of the second speaker from a second source of audio identified as being associated with the second speaker during analysis of a second internet resource;
generating, based upon the translated transcript and the speaker profile associated with the second speaker, first audio comprising an auditory representation of the translated transcript being spoken in a voice of the second speaker; and
generating, based upon the first video and the first audio, a second video in which mouth movements of the first speaker are aligned with speech of the auditory representation of the first audio.
20 . The non-transitory machine readable medium of claim 19 , wherein:
the second speaker is the same as the first speaker.