Characterizing content for audio-video dubbing and other transformations
A computer-implemented method for transforming audio-video data includes automatically detecting substantially all discrete human-perceivable messages encoded in the audio-video data, determining a semantic encoding for each of the detected messages, assigning a time code to each of the encodings correlated to specific frames of the audio-video data, and recording a data structure relating each time code to a corresponding one of the semantic encodings in a recording medium. The method may further include converting extracted recorded vocal instances from the audio-video data into a text data, generating a dubbing list comprising the text data and the time code, assigning a set of annotations corresponding to the one or more vocal instances specifying one or more creative intents, generating the scripting data comprising the dubbing list and the set of annotations, and other optional operations. An apparatus may be programmed to perform the method by executable instructions for the foregoing operations.
1. A computer-implemented method for creating a scripting data for dubbed audio of a media data, the method comprising:
receiving, by one or more processors, the media data comprising one or more recorded vocal instances;
extracting, by the one or more processors, the one or more recorded vocal instances from an audio portion of the media data;
determining, by the one or more processors, a semantic encoding for a human-perceivable message encoded in the media data based on an analysis of a scene or a character represented in the one or more recorded vocal instances;
assigning, by the one or more processors, a time code to each of the extracted vocal instances and to the determined semantic encoding, correlated to a specific frame of the media data;
converting, by the one or more processors, the extracted recorded vocal instances into a text data;
generating, by the one or more processors, a dubbing list comprising the text data and the time code;
displaying, by the one or more processors, a selected portion of the dubbing list corresponding to the one or more vocal instances;
assigning, by the one or more processors, a set of annotations corresponding to the one or more vocal instances based on the determined semantic encoding; and
generating, by the one or more processors, the scripting data comprising the dubbing list and the set of annotations.
2. The method of claim 1 , wherein the time code includes a time-in and a time-out.
3. The method of claim 1 , wherein the dubbing list further includes an identity information of each of the extracted vocal instances.
4. The method of claim 1 , wherein the dubbing list is selected from a group consisting of a dialogue list, a spotting list, or both.
5. The method of claim 1 , further comprising generating, by the one or more processors, the set of annotations including at least one of: an explanation of context specific to a language used in the one or more extracted vocal instances, a description of a scene in which the one or more extracted vocal instances occur, or annotations for a rotoscoped content.
6. The method of claim 1 , further comprising translating, by the one or more processors, the text data in a first language to a second language.
7. The method of claim 1 , further comprising grouping, by the one or more processors, the one or more extracted recorded voice instances based on at least one of: a closed caption, a subtitle, a script, a transcript, a scene position, a main title, an insert, a location card, a background title, a song type, a lyric of a song, a character that sings the song, an intellectual property ownership, a censorship, and a delivery format.
8. The method of claim 1 , further comprising identifying, by the one or more processors, the one or more extracted record voice instances as corresponding to a role played by a child.
9. An apparatus for creating a scripting data for dubbed audio of a media data, the apparatus comprising:
at least one memory storing processor-readable instructions;
one or more processors configured to access the at least one memory and execute the processor-readable instructions to perform operations, the operations comprising:
receiving the media data comprising one or more recorded vocal instances;
determining a semantic encoding for a human-perceivable message encoded in the media data based on an analysis of a scene or a character associated with the one or more recorded vocal instances;
extracting the one or more recorded vocal instances from an audio portion of the media data;
assigning a time code to each of the extracted vocal instances and to the determined semantic encoding, correlated to a specific frame of the media data;
converting the extracted recorded vocal instances into a text data;
generating a dubbing list comprising the text data and the time code;
displaying a selected portion of the dubbing list corresponding to the one or more vocal instances;
assigning a set of annotations corresponding to the one or more vocal instances based on the determined semantic encoding; and
generating the scripting data comprising the dubbing list and the set of annotations.
10. The method of claim 1 , further comprising:
correlating, by the one or more processors, the determined semantic encoding to one or more emotional indicators based on a machine learning algorithm.
11. The method of claim 1 , wherein the analysis of the scene or the character includes one or more of a facial expression analysis, an action/gesture/posture analysis, a mood analysis, or a perspective analysis.
12. The apparatus of claim 9 , wherein the operations further comprise:
correlating the determined semantic encoding to one or more emotional indicators based on a machine learning algorithm.
13. The apparatus of claim 9 , wherein the analysis of the scene or the character includes one or more of a facial expression analysis, an action/gesture/posture analysis, a mood analysis, or a perspective analysis.
14. A non-transitory computer-readable medium containing instructions for creating a scripting data for dubbed audio of a media data, the instructions comprising:
receiving the media data comprising one or more recorded vocal instances;
extracting the one or more recorded vocal instances from an audio portion of the media data;
determining a semantic encoding for a human-perceivable message encoded in the media data based on an analysis of a scene or a character represented in the one or more recorded vocal instances;
assigning a time code to each of the extracted vocal instances and to the determined semantic encoding, correlated to a specific frame of the media data;
converting the extracted recorded vocal instances into a text data;
generating a dubbing list comprising the text data and the time code;
displaying a selected portion of the dubbing list corresponding to the one or more vocal instances;
assigning a set of annotations corresponding to the one or more vocal instances based on the determined semantic encoding; and
generating the scripting data comprising the dubbing list and the set of annotations.
15. The non-transitory computer-readable medium of claim 14 , wherein the time code includes a time-in and a time-out.
16. The non-transitory computer-readable medium of claim 14 , wherein the dubbing list further includes an identity information of each of the extracted vocal instances.
17. The non-transitory computer-readable medium of claim 14 , wherein the dubbing list is selected from a group consisting of a dialogue list, a spotting list, or both.
18. The non-transitory computer-readable medium of claim 14 , further comprising:
correlating the determined semantic encoding to one or more emotional indicators based on a machine learning algorithm.
19. The non-transitory computer-readable medium of claim 14 , wherein the analysis of the scene or the character includes one or more of a facial expression analysis, an action/gesture/posture analysis, a mood analysis, or a perspective analysis.
20. The non-transitory computer-readable medium of claim 14 , further comprising identifying the one or more extracted record voice instances as corresponding to a role played by a child.