Method and a Device For Performing an Automatic Dubbing on a Multimedia Signal
A method and a device for performing an automatic dubbing on a multimedia signal This invention relates to a method and a system for performing automatic dubbing on a multimedia signal, such as a TV or a DVD signal, where the multimedia signal comprises information relating to video and speech and further comprises textual information corresponding to the speech. Initially the multimedia signal is received by a receiver. The speech and the textual information are then, respectively, extracted which results in said speech and textual information. The speech is analyzed resulting in at least one voice characteristic parameter, and based on the at least one voice characteristic parameter the textual information is converted to a new speech.
1 . A method of performing automatic dubbing on a multimedia signal ( 100 ), such as a TV or a DVD signal, where said multimedia signal ( 100 ) comprises information relating to video ( 108 ) and speech ( 102 ) and further comprises textual information ( 103 ) corresponding to said speech ( 102 ); said method comprises the steps of:
receiving said multimedia signal ( 100 ),
extracting respectively the speech ( 102 ) and the textual information ( 103 ) from said multimedia signal ( 100 ),
analyzing said speech to obtain at least one voice characteristic parameter, and based on said at least one voice characteristic parameter,
converting said textual information ( 103 ) to a new speech ( 207 ).
2 . A method according to claim 1 , wherein said at least one voice characteristic parameter comprises one or more parameters from the group consisting of: pitch, melody, duration, phoneme reproduction speed, loudness, timbre.
3 . A method according to claim 1 , wherein said textual information ( 103 ) comprises subtitle information on a DVD, teletext subtitles, or closed captioning subtitles.
4 . A method according to claim 3 , wherein said textual information ( 103 ) comprises information which is extracted from the multimedia ( 100 ) signal by means of text detection and optical character recognition.
5 . A method according to claim 1 , wherein said original speech is removed and replaced by said new speech ( 207 ) which is inserted into a new multimedia signal ( 109 ), said new multimedia signal ( 109 ) comprising said new speech ( 207 ) and said video ( 108 ) information.
6 . A method according to claim 5 , where said new speech ( 207 ) is inserted into said new multi media signal ( 109 ) at a predetermined time delay ( 308 ).
7 . A method according to claim 5 , wherein the timing of inserting said new speech into said new multimedia signal ( 109 ) corresponds to the timing of displaying said textual information ( 103 ) on said video ( 108 ) in the received multimedia signal ( 100 ).
8 . A method according to claim 5 , wherein the timing of inserting said new speech into said new multimedia signal ( 109 ) is based on sentence boundaries identified by capital letters and punctuation within the textual information.
9 . A method according to claim 5 , wherein the timing of inserting said new speech into said new multimedia signal ( 109 ) is based on speech boundaries identified by silences within the received speech information.
10 . A computer readable medium having stored therein instructions for causing a processing unit to execute a method according to claim 1 .
11 . A device for performing automatic dubbing on a multimedia signal ( 100 ), such as a TV or a DVD signal, where said multimedia signal ( 100 ) comprises information relating to video ( 108 ) and speech ( 102 ) and further comprises textual information ( 103 ) corresponding to said speech ( 102 ), wherein said device comprises:
a receiver ( 208 ) for receiving said multimedia signal ( 100 ),
a processor ( 206 ) for extracting respectively the speech and the textual information from said multimedia signal ( 100 ),
a voice analyzer ( 203 ) for analyzing said speech ( 102 ) to obtain at least one voice characteristic parameter,
a speech synthesizer ( 204 ) for, based on said at least one voice characteristic parameter, converting said textual information ( 103 ) to a new speech ( 207 ).