Conformance of media content to original camera source using optical character recognition
View Patent ↗A clip of shots is uploaded to a conformance platform. The conformance platform evaluates the clip type and initiates shot boundary evaluation and detection. The identified shot boundaries are then seeded for OCR evaluation and the burned in metadata is extracted into categories using a custom OCR module based on the location of the burn-ins within the frame. The extracted metadata is then error corrected based on OCR evaluation of the neighboring frame and arbitrary frames at pre-computed timecode offsets from the frame boundary. The error corrected metadata and categories are then packaged into a metadata package and returned back to a conform editor. The application then presents the metadata package as an edit decision list with associated pictures and confidence level to the user. The user can further validate and override the edit decision list if necessary and then use it to directly to conform to the online content.
1. A method for conforming offline video content to original source frames performed in a computer system having a processor and a memory, the method comprising
ingesting a video clip within the memory of the computer system, the video clip comprising a series of frames and having a plurality of shots and shot boundaries therein and further including frame identification information burned into a perimeter of a plurality of the series of frames;
identifying via operations within the processor the shot boundaries between adjacent frames within the video clip;
locating via operations within the processor burn-in areas in the frames adjacent to the shot boundaries that contain burn-in information;
perceiving via operations within the processor character strings within the burn-in areas in the frames adjacent to the shot boundaries;
recognizing via operations within the processor visually perceptible characters within the character strings;
separating via operations within the processor the visually perceptible characters;
performing via operations within the processor an optical character recognition using a trained classifier on the separated visually perceptible characters to identify the frame identification information represented by the characters;
correcting errors in the identified frame identification information via operations within the processor by
determining cadence, consistency, or both, between timecodes in the frame identification information in adjacent frames; and
adjusting timecodes of inconsistent frames or out of cadence frames to conform with the consistency or cadence of other frames within a particular shot in the video clip; and
storing the frame identification information as a metadata file in a relational database within the memory for use with an edit decision list application to identify source image frames corresponding to the frame identification information to create a high resolution instantiation of the video clip.
2. The method of claim 1 further comprising
correcting errors in the identified frame identification information via operations within the processor by
determining consistency of a file name in the frame identification information in adjacent frames; and
adjusting the file names of inconsistent frames to conform with consistency of other file names of other frames within a particular shot in the video clip.
3. The method of claim 1 , wherein the step of identifying the shot boundaries further comprises
masking an area of the frames of the video clip without covering bands along a top edge and a bottom edge of each of the frames;
computing a perceptual fingerprint of each frame outside of the area covered by the mask; and
comparing the perceptual fingerprint values between frames to determine difference values between frames, wherein
if the difference value between frames is below a threshold value, categorizing the compared frames as being within a same shot; and
if the difference value between frames is above the threshold value, categorizing the compared frames as being within different shots.
4. The method of claim 3 , wherein the step of identifying the shot boundaries further comprises
locating a darkest frame in the video clip; and
identifying a presence or absence of a watermark in the darkest frame; wherein
if a watermark is identified,
defining a boundary area of the watermark; and
fitting the masked area to the boundary area of the watermark.
5. The method of claim 3 , wherein
the computation of the perceptual hash is based upon color values of pixels in the frames; and
the step of identifying the shot boundaries further comprises identifying whether the shot boundary is a cut or dissolve based upon the perceptual hash of the color values.
6. The method of claim 3 , wherein
the computation of the perceptual hash is based upon luminance values of pixels in the frames; and
the step of identifying the shot boundaries further comprises identifying whether the shot boundary is a fade based upon the perceptual hash of the luminance values.
7. The method of claim 1 , wherein the step of locating burn-in areas further comprises masking an area of each frame without covering bands along a top edge and a bottom edge of each frame.
8. The method of claim 7 , wherein the step of perceiving character strings further comprises
converting color information in each frame to hue/saturation/value information;
discarding the hue and saturation information;
performing morphological operations on the bands to increase a contrast between characters in the bands and a background of the bands.
9. The method of claim 8 , wherein the step of recognizing visually perceptible characters further comprises
pyramid downsampling the bands to identify connected characters; and
identifying contours of connected characters.
10. The method of claim 9 , wherein the step of separating visually perceptible characters further comprises
segmenting connected characters;
identifying contours of individual characters previously segmented; and
resizing the segmented characters into a uniform character image size.
11. The method of claim 1 , wherein the step of performing optical character recognition includes using a trained nearest neighbor classifier.
12. A non-transitory computer readable medium containing instructions for instantiating a computer system having a processor and a memory to conform offline video content to original source frames, wherein the instructions configure the processor to implement a computer process comprising the steps of
ingesting a video clip within the memory of the computer system, the video clip comprising a series of frames and having a plurality of shots and shot boundaries therein and further including frame identification information burned into a perimeter of a plurality of the series of frames;
identifying via operations within the processor the shot boundaries between adjacent frames within the video clip;
locating via operations within the processor burn-in areas in the frames adjacent to the shot boundaries that contain burn-in information;
perceiving via operations within the processor character strings within the burn-in areas in the frames adjacent to the shot boundaries;
recognizing via operations within the processor visually perceptible characters within the character strings;
separating via operations within the processor the visually perceptible characters;
performing via operations within the processor an optical character recognition using a trained classifier on the separated visually perceptible characters to identify the frame identification information represented by the characters;
correcting errors in the identified frame identification information by
determining cadence, consistency, or both, between timecodes in the frame identification information in adjacent frames; and
adjusting timecodes of inconsistent frames or out of cadence frames to conform with the consistency or cadence of other frames within a particular shot in the video clip; and
storing the frame identification information as a metadata file in a relational database within the memory for use with an edit decision list application to identify source image frames corresponding to the frame identification information to create a high resolution instantiation of the video clip.
13. The non-transitory computer readable storage medium of claim 12 , wherein the instructions configure the processor to implement a further processing step comprising
correcting errors in the identified frame identification information by
determining consistency of a file name in the frame identification information in adjacent frames; and
adjusting the file names of inconsistent frames to conform with consistency of other file names of other frames within a particular shot in the video clip.
14. The non-transitory computer readable storage medium of claim 12 , wherein the instructions further implement the step of identifying the shot boundaries by configuring the processor to
mask an area of the frames of the video clip without covering bands along a top edge and a bottom edge of each of the frames;
compute a perceptual fingerprint of each frame outside of the area covered by the mask; and
compare the perceptual fingerprint values between frames to determine difference values between frames, wherein
if the difference value between frames is below a threshold value, categorize the compared frames as being within a same shot; and
if the difference value between frames is above the threshold value, categorize the compared frames as being within different shots.
15. The non-transitory computer readable storage medium of claim 14 , wherein the instructions further implement the step of identifying the shot boundaries by configuring the processor to
locate a darkest frame in the video clip; and
identify a presence or absence of a watermark in the darkest frame; wherein
if a watermark is identified,
define a boundary area of the watermark; and
fit the masked area to the boundary area of the watermark.
16. The non-transitory computer readable storage medium of claim 14 , wherein
the computation of the perceptual hash is based upon color values of pixels in the frames; and
the instructions further implement the step of identifying the shot boundaries by configuring the processor to identify whether the shot boundary is a cut or dissolve based upon the perceptual hash of the color values.
17. The non-transitory computer readable storage medium of claim 14 , wherein
the computation of the perceptual hash is based upon luminance values of pixels in the frames; and
the instructions further implement the step of identifying the shot boundaries by configuring the processor to identify whether the shot boundary is a fade based upon the perceptual hash of the luminance values.
18. The non-transitory computer readable storage medium of claim 12 , wherein the step of locating burn-in areas further comprises masking an area of each frame without covering bands along a top edge and a bottom edge of each frame.
19. The non-transitory computer readable storage medium of claim 18 , wherein the instructions further implement the step of perceiving character strings by configuring the processor to
convert color information in each frame to hue/saturation/value information;
discard the hue and saturation information;
perform morphological operations on the bands to increase a contrast between characters in the bands and a background of the bands.
20. The non-transitory computer readable storage medium of claim 19 , wherein the instructions further implement the step of recognizing visually perceptible characters by configuring the processor to
pyramid downsample the bands to identify connected characters; and
identify contours of connected characters.
21. The non-transitory computer readable storage medium of claim 20 , wherein the instructions further implement the step of separating visually perceptible characters by configuring the processor to
segment connected characters;
identify contours of individual characters previously segmented; and
resize the segmented characters into a uniform character image size.
22. The non-transitory computer readable storage medium of claim 12 , wherein the step of performing optical character recognition includes using a trained nearest neighbor classifier.