Systems and methods for generating translated media streams with synthesized laughter
View Patent ↗Systems, methods and non-transitory computer readable media for generating media streams are provided. The generation of the media streams involves receiving an input media stream including an individual speaking in a first language; determining from the input media stream a set of speech-related characteristics associated with the individual; generating, using the set of speech-related characteristics associated with the individual, a synthesized voice for the individual; and producing an output media stream comprising for each set of words spoken by the individual in the first language in the input media stream a translation set of words in a target language using the synthesized voice.
1 . A method for generating media streams, the method comprising:
receiving an input media stream of an individual speaking in a first language;
determining from the input media stream a set of speech-related characteristics associated with the individual;
identifying in the input media stream a set of words spoken by the individual in the first language;
identifying in the input media stream non-verbal vocalizations articulated by the individual;
generating, using the set of speech-related characteristics associated with the individual, a synthesized voice for the individual; and
generating an output media stream that includes, for the identified set of words spoken by the individual in the first language a corresponding translated set of words in a target language using the synthesized voice, and for the identified non-verbal vocalizations articulated by the individual corresponding non-verbal vocalizations using the synthesized voice, wherein the non-verbal vocalizations generated using the synthesized voice include laughter.
2 . The method of claim 1 , wherein the set of speech-related characteristics includes voice characteristics of the individual while speaking the set of words in the first language.
3 . The method of claim 2 , wherein the voice characteristics includes at least one of prosodic characteristics of a voice of the individual, characteristics of a pitch of the voice of the individual, characteristics of a loudness of the voice of the individual, characteristics of an intonation of the voice of the individual, or characteristics of a stress of the voice of the individual.
4 . The method of claim 1 , wherein the set of speech-related characteristics includes articulation characteristics of the individual while speaking the set of words in the first language.
5 . The method of claim 4 , wherein the articulation characteristics includes at least one of characteristics of speech rhythm, characteristics of speech tempo, characteristics of a linguistic tone of a speech, characteristics of pauses within the speech, characteristics of an accent of the speech, characteristics of a language register of the speech, characteristics of a language of the speech, or a form of the speech.
6 . The method of claim 1 , wherein the set of speech-related characteristics includes characteristics of emotional states of the individual while speaking the set of words in the first language.
7 . The method of claim 1 , further comprising determining a desired level of accent to introduce in the synthesized voice, and wherein the output media stream includes articulations in the target language using the synthesized voice with the desired level of accent.
8 . The method of claim 1 , further comprising: processing visual data in the input media stream to identify text written in the first language and including in the output media stream a translation in the target language of the identified text.
9 . The method of claim 1 , further comprising:
causing a display in a graphical user interface (GUI) of information indicative of a plurality of available target languages; and
receiving, via the GUI, a selection of a preferred target language.
10 . The method of claim 1 , further comprising: receiving a user selection to enable selective manipulation of at least one voice characteristic of the synthesized voice.
11 . The method of claim 1 , wherein the non-verbal vocalizations generated using the synthesized voice further include cheering.
12 . The method of claim 1 , wherein the non-verbal vocalizations generated using the synthesized voice further include crying.
13 . The method of claim 1 , further comprising storing data associated with the determined set of speech-related characteristics for future generation of other media streams using the synthesized voice.
14 . The method of claim 1 , wherein the input media stream further includes speech in a second language, and the method further includes analyzing the input media stream to identify first words in the first language and second words in the second language, wherein the output media stream includes a first plurality of words corresponding to the first words articulated using the synthesized voice and a second plurality of words corresponding to the second words articulated using a second synthesized voice.
15 . The method of claim 14 , wherein, in the output media stream, both the first plurality of words and the second plurality of words are spoken in the target language, and the second plurality the second plurality of words is spoken in the target language with an accent of the second language.
16 . The method of claim 1 , wherein the input media stream is associated with a real-time conversation between the individual and at least one other individual, and the method further includes determining the target language based on an identity of the at least one other individual.
17 . The method of claim 1 , wherein the input media stream is associated with a real-time conversation between the individual and at least one other individual, and the method further includes receiving user selection indicative of the target language.
18 . The method of claim 1 , wherein the input media stream is associated with a real-time conversation between the individual and at least one other individual, and the method further includes changing the synthesized voice as the real-time conversation progresses to improve how the individual sounds when speaking the target language.
19 . The method of claim 1 , wherein the input media stream is associated with a real-time conversation between the individual and at least one other individual, and the output media stream is generated in a manner that takes into account a gender of the at least one other individual.
20 . A system for generating media streams, comprising:
a microphone for recording an input media stream of an individual speaking in a first language;
at least one processing device configured to:
determine from the input media stream a set of speech-related characteristics associated with the individual;
identify in the input media stream a set of words spoken by the individual in the first language;
identify in the input media stream non-verbal vocalizations articulated by the individual;
generate, using the set of speech-related characteristics associated with the individual, a synthesized voice for the individual; and
generate an output media stream that includes, for the identified set of words spoken by the individual in the first language a corresponding translated set of words in a target language using the synthesized voice; and
a transmitter for transmitting the output media stream as part of a real-time conversation, and for the identified non-verbal vocalizations articulated by the individual corresponding non-verbal vocalizations using the synthesized voice, wherein the non-verbal vocalizations generated using the synthesized voice include laughter.
21 . A computer program product for generating media streams, the computer program product embodied in a non-transitory computer-readable medium and including instructions that when executed by at least one processor cause the at least one processor to execute a method, the method comprising:
receiving an input media stream of an individual speaking in a first language;
determining from the input media stream a set of speech-related characteristics associated with the individual;
identifying in the input media stream a set of words spoken by the individual in the first language;
identifying in the input media stream non-verbal vocalizations articulated by the individual;
generating, using the set of speech-related characteristics associated with the individual, a synthesized voice for the individual; and
producing an output media stream that includes, for the identified set of words spoken by the individual in the first language a corresponding translated set of words in a target language using the synthesized voice, and for the identified non-verbal vocalizations articulated by the individual corresponding non-verbal vocalizations using the synthesized voice, wherein the non-verbal vocalizations generated using the synthesized voice include laughter.