Using motion compensated temporal filter (MCTF) statistics for scene change detection when a fade, dissolve or cut occurs
A method is provided to better detect a scene change to provide a prediction to an encoder to enable more efficient encoding. The method uses a Motion Compensated Temporal Filter (MCTF) that provides motion estimation and is located prior to an encoder. The MCTF provides a Motion Compensated Residual (MCR) used to detect the scene change transition. When a scene is relatively stable, the MCR score is also relatively stable. However, when a scene transition is in process, the MCR score behavior changes, Algorithmically, the MCR score is used by comparing the sliding mean of the MCR score to the sliding median. This comparison highlights the transition points. In the case of a scene cut, the MCR score exhibits a distinct spike. In the case of a fade or dissolve, the MCR score exhibits a transitional period of degradation followed by recovery. By implementing the above detection using the MCR, the location of the I-pictures in the downstream encoding process can be accurately determined for the encoder.
1. A method for encoding video using scene change detection, the method for determining an initial frame of a new scene following a gradual scene transition and comprising:
obtaining video frames provided to an encoder;
determining a sliding MCR score for individual ones of the video frames, the MCR score based on a motion compensated residual (MCR) for the video frames;
determining a first set of statistics using respective MCR scores for a plurality of the video frames leading up to a current frame;
determining a second set of statistics using respective MCR scores for a plurality of the video frames leading up to the current frame, the second set of statistics different than the first set of statistics;
classifying the current frame as the initial frame of the new scene following a gradual scene transition based on a comparison of each of the first set of statistics to respective thresholds and based on a comparison of each of the second set of statistics to respective thresholds; wherein the first set of statistics comprises a linear regression of the MCR and a variance of the MCR, and the comparison of the first set of statistics to respective thresholds determines whether the MCR linear regression is flat and the MCR variance is low.
2. The method of claim 1 , wherein the first set of statistics are used to determine scene stability and the second set of statistics are used to verify that the current frame is a first frame following a gradual scene transition.
3. The method of claim 1 , wherein the second set of statistics comprises an MCR score of a current frame relative to the mean MCR score of previous frames, a comparison of the MCR score of the current frame to that of the frame prior to the current frame, and a regression value of the MCR score of the frame prior to the current frame.
4. The method of claim 3 , wherein the first set of statistics comprises a linear regression of the MCR and a variance of the MCR.
5. The method of claim 1 further comprising placing an I-frame during the encoding process at a point where the scene change occurs.
6. An apparatus to encode video frames, the apparatus comprising:
an encoder having a first input for receiving video frames to be processed and a second input for receiving parameter data to enable the encoder to allocate bits for frames for encoding;
a pre-filter with Motion Compensated Temporal Filtering (MCTF) frame buffer having an input receiving the video frames and an output providing the first input to the encoder;
a MCTF statistical analysis module processor that provides a Motion Compensated Residual (MCR) for receiving the video frames from the pre-filter with MCTF and having an output providing the second input to the encoder;
a preprocessor memory connected to the MCTF statistical analysis processor for storing code that is executable by the preprocessor to determine the parameter data to enable the encoder to allocate bits, the code causing the preprocessor to perform the following steps:
obtaining a MCR for the video frames;
determining a sliding MCR score for individual ones of the video frames, the MCR score based on the MCR for the video frames;
determining a first set of statistics using respective MCR scores for a plurality of the video frames leading up to a current frame that follows a scene transition;
determining a second set of statistics using respective MCR scores for a plurality of the video frames leading up to the current frame, the second set of statistics different than the first set of statistics; and
classifying the current frame as the initial frame of a new scene following a gradual scene transition based on a comparison of each of the first set of statistics to respective thresholds and based on a comparison of each of the second set of statistics to respective thresholds; wherein the first set of statistics comprises a linear regression of the MCR and a variance of the MCR, and the comparison of the first set of statistics to respective thresholds determines whether the MCR linear regression is flat and the MCR variance is low.
7. The apparatus of claim 6 , wherein the first set of statistics are used to determine scene stability and the second set of statistics are used to verify that the current frame is a first frame following a gradual scene transition.
8. The apparatus of claim 6 , wherein the second set of statistics comprises an MCR score of a current frame relative to the mean MCR score of previous frames, a comparison of the MCR score of the current frame to that of the frame prior to the current frame, and a regression value of the MCR score of the frame prior to the current frame.
9. The apparatus of claim 8 , wherein the first set of statistics comprises a linear regression of the MCR and a variance of the MCR.
10. The apparatus of claim 6 further comprising placing an I-frame during the encoding process at a point where the scene change occurs.
11. The apparatus of claim 6 , wherein the encoder is a two path device with the MCTF statistical analysis module processor provided in a first path and the encoder provided in the second path.