Quality measurement between mismatched videos
In some embodiments, a method receives a first video and processes frames of the first video to generate a second video. A frame matching process is performed to determine whether frames of the second video and frames of the first video meet a condition that determines whether frames match to generate a new version of the second video and a new version of the first video. A sequence of frames in the new version of the first video is different from the first video. The method compares frames of the new version of the second video and frames of the new version of the first video to generate a score based on a quality metric.
1 . A method comprising:
receiving, by a computing device, a first video;
processing, by the computing device, frames of the first video to generate a second video;
performing, by the computing device, a frame matching process to determine whether frames of the second video and frames of the first video meet a condition that determines whether frames match to generate a new version of the second video and a new version of the first video, wherein a sequence of frames in the new version of the first video is different from the first video;
comparing, by the computing device, frames of the new version of the second video and frames of the new version of the first video to generate a plurality of frame level scores based on a quality metric;
outputting a video level score based on the plurality of frame level scores; and
adjusting the video level score based on statistics from the frame matching process that determine whether frames match in the first video and the second video to determine an adjusted video level score, wherein the statistics include a mis-matched frame count, a duplicated frame count, and a number of reordered frame count.
2 . The method of claim 1 , wherein a first set of frames in the second video includes a variation from a second set of frames in the first video.
3 . The method of claim 2 , wherein the variation is in a sequence of frames in the first set of frames compared to the second set of frames.
4 . The method of claim 2 , wherein the variation comprises at least one of a dropped frame, a duplicated frame, and an out of order frame in the first set of frames.
5 . The method of claim 2 , wherein the variation is created when generating the second video using a degradation process, wherein the degradation process changes a characteristic of the first video to generate the second video.
6 . The method of claim 1 , wherein the second video is a degraded version of the first video.
7 . The method of claim 1 , wherein performing the frame matching process comprises:
selecting a first frame in the second video; and
searching for a corresponding first frame in the first video that is determined to meet a matching condition.
8 . The method of claim 7 , wherein performing the frame matching process comprises:
when the matching condition is met, outputting the corresponding first frame in the first video and the first frame in the second video as a matching pair.
9 . The method of claim 7 , wherein performing the frame matching process comprises:
when the matching condition is met, storing information for the corresponding first frame in the first video in the new version of the first video and storing information for the first frame in the second video in the new version of the second video.
10 . The method of claim 1 , wherein the new version of the first video and the new version of the second video includes matching frames from the frame matching process.
11 . The method of claim 1 , wherein new version of the first video does not include a frame that is determined to be dropped from the second video.
12 . The method of claim 1 , wherein new version of the first video includes a duplicate frame that is determined to be duplicated in the second video.
13 . The method of claim 1 , wherein new version of the first video includes an out of order frame that is determined to be out of order in the second video.
14 . The method of claim 1 , wherein performing the frame matching process comprises:
determining a first frame from the second video;
determining a second frame from the first video;
determining a plurality of similarity metric values based on comparing a plurality of similarity metrics from the first frame and the second frame; and
using the plurality of similarity metric values to determine whether the first frame and the second frame meet a condition.
15 . The method of claim 1 , wherein performing the frame matching process comprises:
comparing a first frame from the second video and a second frame from the first video;
outputting a frame level score based on the comparing; and
comparing the frame level score to a threshold to determine whether the first frame and the second frame meet a condition to determine that the first frame and the second frame match.
16 . The method of claim 15 , wherein the threshold changes as different frames are compared from the second video based on feedback from previous comparisons of frames.
17 . The method of claim 1 , wherein adjusting the video level score based on statistics from the frame matching process comprises:
adjusting the video level score based on a first function of the mis-matched frame count, the duplicated frame count, and the number of reordered frame count and a second function of the mis-matched frame count, the duplicated frame count and the number of reordered frame count, wherein the first function and the second function are different.
18 . The method of claim 17 , wherein:
the adjusted video level score is the first function times the video level score minus the second function.
19 . A non-transitory computer-readable storage medium having stored thereon computer executable instructions, which when executed by a computing device, cause the computing device to be operable for:
receiving a first video;
processing frames of the first video to generate a second video;
performing a frame matching process to determine whether frames of the second video and frames of the first video meet a condition that determines whether frames match to generate a new version of the second video and a new version of the first video, wherein a sequence of frames in the new version of the first video is different from the first video;
comparing frames of the new version of the second video and frames of the new version of the first video to generate a plurality of frame level scores based on a quality metric;
outputting a video level score based on the plurality of frame level scores; and
adjusting the video level score based on statistics from the frame matching process that determine whether frames match in the first video and the second video to determine an adjusted video level score, wherein the statistics include a mis-matched frame count, a duplicated frame count, and a number of reordered frame count.
20 . An apparatus comprising:
one or more computer processors; and
a computer-readable storage medium comprising instructions for controlling the one or more computer processors to be operable for:
receiving a first video;
processing frames of the first video to generate a second video;
performing a frame matching process to determine whether frames of the second video and frames of the first video meet a condition that determines whether frames match to generate a new version of the second video and a new version of the first video, wherein a sequence of frames in the new version of the first video is different from the first video;
comparing frames of the new version of the second video and frames of the new version of the first video to generate a plurality of frame level scores based on a quality metric;
outputting a video level score based on the plurality of frame level scores; and
adjusting the video level score based on statistics from the frame matching process that determine whether frames match in the first video and the second video to determine an adjusted video level score, wherein the statistics include a mis-matched frame count, a duplicated frame count, and a number of reordered frame count.