Comparing frame data to generate a textless version of a multimedia production
A media frame alignment system aligns textless media clips with associated texted media frames in a multimedia production, such as film or video. Textless frames in a film clip are aligned with frames containing text (e.g., in the final version of a film) based on similar frame data. Masking may be applied to both the textless clip and to the texted frames to mask areas within the frames that differ, such as the text in the multimedia production and the associated areas in the textless clip. The frame data surrounding the masks can be analyzed and the frame data from the textless frames and from the texted frames in the multimedia production can be compared to determine matching frames. Once the textless frames are matched with texted frames in the multimedia production, an edit decision list (EDL) and/or master textless version may be created.
1 . A computer-implemented media frame alignment system comprising
a storage device configured to ingest and store one or more media files thereon; and
one or more processors configured with instructions to
receive a texted version of a multimedia production and a textless media clip associated with the texted version of the multimedia production, wherein the texted version of the multimedia production comprises one or more texted frames and the textless media clip comprises one or more textless frames;
mask text in the one or more texted frames;
mask a same area in the one or more textless frames as the text in the one or more texted frames;
analyze frame data surrounding the masks;
compare the analyzed frame data between the one or more texted frames and the one or more textless frames to determine frames with similar frame data; and
align the one or more textless frames with the one or more texted frames based on frames with similar frame data.
2 . The computer-implemented media frame alignment system of claim 1 , wherein the one or more processors are further configured with instructions to determine one or more frame locations for the one or more textless frames based on the alignment of the one or more textless frames with the one or more texted frames, wherein the texted frames include at least one of frame numbering or timing information.
3 . The computer-implemented media frame alignment system of claim 2 , wherein the one or more processors are further configured with instructions to generate at least one of an edit decision list (EDL) or a textless master copy based on the determined one or more frame locations for the one or more textless frames.
4 . The computer-implemented media frame alignment system of claim 3 , wherein the one or more processors are further configured with instructions to insert the one or more textless frames into a copy of the multimedia production based on the determined one or more frame locations to generate the textless master copy.
5 . The computer-implemented media frame alignment system of claim 3 , wherein the one or more processors are further configured to store at least one of the edit decision list (EDL) or textless master copy on the storage device.
6 . The computer-implemented media frame alignment system of claim 1 , wherein the instructions to analyze frame data surrounding the masks comprises instructions to perform a perceptual hash algorithm on the image areas surrounding the masks to produce a hash value for each frame.
7 . The computer-implemented media frame alignment system of claim 6 , wherein the instructions to compare the analyzed frame data comprises instructions to compare hash values.
8 . The computer-implemented media frame alignment system of claim 7 , wherein the instructions to compare hash values comprises instructions to compare bit positions and determine a number of bit positions that are different.
9 . A method implemented on a computer system for aligning media frames, wherein one or more processors in the computer system is particularly configured to perform a number of processing steps comprising
receiving a texted version of a multimedia production and a textless media clip associated with the texted version of the multimedia production, wherein the texted version of the multimedia production comprises one or more texted frames and the textless media clip comprises one or more textless frames;
masking text in the one or more texted frames;
masking a same area as the text in the one or more texted frames in the one or more textless frames;
analyzing frame data surrounding the masks;
comparing the analyzed frame data between the one or more texted frames and the one or more textless frames to determine frames with similar frame data; and
aligning the one or more textless frames with the one or more texted frames based on frames with similar frame data.
10 . The method of claim 9 , comprising a further step of determining one or more frame locations for the one or more textless frames based on the alignment of the one or more textless frames with the one or more texted frames, wherein the texted frames include at least one of frame numbering or timing information.
11 . The method of claim 10 , comprising a further step of generating at least one of an edit decision list (EDL) or a textless master copy based on the determined one or more frame locations for the one or more textless frames.
12 . The method of claim 11 , comprising a further step of inserting the one or more textless frames into a copy of the multimedia production based on the determined one or more frame locations to generate the textless master copy.
13 . The method of claim 11 , comprising a further step of storing at least one of the edit decision list (EDL) or textless master copy on a storage device communicatively coupled to the one or more processors in the computer system.
14 . The method of claim 9 , wherein the analyzing step comprises performing a perceptual hash algorithm on the images surrounding the masks to produce a hash value for each frame.
15 . The method of claim 14 , wherein the comparing step comprises comparing hash values.
16 . The method of claim 15 , wherein comparing hash values comprises comparing bit positions and determining a number of bit positions that are different.
17 . A non-transitory computer readable storage medium containing instructions for instantiating a special purpose computer to align media frames, wherein the instructions implement a computer process comprising the steps of
receiving a texted version of a multimedia production and a textless media clip associated with the texted version of the multimedia production, wherein the texted version of the multimedia production comprises one or more texted frames and the textless media clip comprises one or more textless frames;
masking text in the one or more texted frames;
masking a same area as the text in the one or more texted frames in the one or more textless frames;
analyzing frame data surrounding the masks;
comparing the analyzed frame data between the one or more texted frames and the one or more textless frames to determine frames with similar frame data; and
aligning the one or more textless frames with the one or more texted frames based on frames with similar frame data.
18 . The non-transitory computer readable storage medium of claim 17 , wherein the instructions implement a further process step comprising determining one or more frame locations for the one or more textless frames based on the alignment of the one or more textless frames with the one or more texted frames, wherein the texted frames include at least one of frame numbering or timing information.
19 . The non-transitory computer readable storage medium of claim 18 , wherein the instructions implement a further process step comprising generating at least one of an edit decision list (EDL) or a textless master copy based on the determined one or more frame locations for the one or more textless frames.
20 . The non-transitory computer readable storage medium of claim 19 , wherein the instructions implement a further process step comprising inserting the one or more textless frames into a copy of the multimedia production based on the determined one or more frame locations to generate the textless master copy.
21 . The non-transitory computer readable storage medium of claim 19 , wherein the instructions implement a further process step comprising storing at least one of the edit decision list (EDL) or textless master copy in the non-transitory computer readable storage medium.
22 . The non-transitory computer readable storage medium of claim 17 , wherein the analyzing step comprises performing a perceptual hash algorithm on the images surrounding the masks to produce a hash value for each frame.
23 . The non-transitory computer readable storage medium of claim 22 , wherein the comparing step comprises comparing hash values.
24 . The non-transitory computer readable storage medium of claim 23 , wherein comparing hash values comprises comparing bit positions and determining a number of bit positions that are different.