Content adaptive video encoder and coding method
A method of pre-processing a sequence of video picture data comprising I pictures, P pictures and B pictures, where the pictures are arranged as a plurality of Group Of Pictures (GOP) and each of the Group Of Pictures has a first picture and a plurality of Sub Group of Pictures (Sub-GOP). The method uses an analysis of the sequence of video picture data to determine a position of a scene change relative to a position of a start of a new GOP within the sequence of video picture data and then moves the start of the new GOP to coincide with the scene change by extending or decreasing a current GOP length according to whether the scene change and the start of a new GOP are within a predetermined number of pictures of each other. An apparatus for carrying out the method is also provided.
1. A method of pre-processing a sequence of video picture data, said sequence of video picture data comprising I pictures, P pictures and B pictures arranged as a plurality of Group Of Pictures, GOPs, and each of said plurality of Group Of Pictures having a first picture and a plurality of Sub Group of Pictures, Sub-GOPs, the method comprising:
analysing the sequence of video picture data;
determining a position of a scene change relative to a position of a start of a new GOP within the sequence of video picture data; and
providing information to a video encoder indicative of moving the start of the new GOP to coincide with the scene change by extending or decreasing a current GOP length, if the scene change and the start of a new GOP are within a predetermined number of pictures,
wherein, if the information provided indicates to move the start of the new GOP by extending the current GOP length, the step of extending comprises:
creating one or more new Sub-GOPs out of P and B pictures; and
inserting the newly created one or more Sub-GOPs into the current GOP until the current GOP is of a sufficient length;
wherein the step of creating the one or more new Sub-GOP comprises:
providing a first P picture as the last picture of the one or more sub-GOP; and
inserting B or P pictures previous to the first P picture up to a desired length of Sub-GOP,
wherein a picture type content of the pictures inserted in to a newly-created Sub-GOP is B type pictures, unless an analysis of the sequence of video picture data finds the following Boolean equation to be true:
(Temporal displacement of the sequence of video picture data≧first predetermined value) OR (Temporal distortion of the sequence of video picture data≧second predetermined value) OR (Temporal displacement≦third predetermined value AND Spatial complexity≦fourth predetermined value AND bitrate<a predetermined rate) AND (no flashes occur OR no fades occur);
wherein if found true, then P pictures are used in the newly created sub-GOP; wherein;
temporal displacement and temporal distortion are temporal complexity metrics;
flash activity metrics comprise detecting whether flashes occur in a Sub-GOP under test, to provide the ‘no flashes occur’ Boolean value; and
fade activity metrics comprise detecting whether a fade occurs in the Sub-GOP under test, to provide the ‘no fades occur’ Boolean value.
2. The method of claim 1 , wherein, if the information provided indicates to move the start of the new GOP by decreasing the current GOP length, the step of decreasing the current GOP length comprises:
inserting an I picture as a first picture of a new GOP at the scene change to start a new GOP;
creating one or more new Sub-GOPs out of P and B pictures; and
inserting the newly created one or more Sub-GOPs into the new GOP until the new GOP is of a sufficient length;
wherein the step of creating the one or more new Sub-GOP comprises;
providing a first P picture as the last picture of the one or more sub-GOP; and
inserting B or P pictures previous to the first P picture up to a desired length of Sub-GOP.
3. A method of pre-processing a sequence of video picture data, said sequence of video picture, data comprising I pictures, P pictures and pictures arranged as a plurality of Group Of Pictures, GOPs, and each of said plurality of Group Of Pictures having a first picture and a plurality of Sub Group of Pictures, Sub-GOPs, the method comprising:
analysing the sequence of video picture data;
determining a position of a scene change relative to a position of a start of a new GOP within the sequence of video picture data; and
providing information to a video encoder indicative of moving the start of the new GOP to coincide with the scene change by extending or decreasing a current GOP length, if the scene chance and the start of a new GOP are within a predetermined number of pictures,
wherein, if the information provided indicates to move the start of the new GOP by extending the current GOP length, the step of extending comprises:
creating one or more new Sub-GOPs out of P and B pictures; and
inserting the newly created one or more Sub-GOPs into the current GOP until the current GOP is of a sufficient length;
wherein the step of creating the one or more new Sub-GOP comprises:
providing a first P picture as the last picture of the one or more sub-GOP, and
inserting B or P pictures previous to the first P picture up to a desired length of Sub-GOP,
wherein a picture type of the pictures inserted into a newly created Sub-GOP is dependent on metrics of the video picture data,
including:
Spatial complexity metrics;
Temporal complexity metrics;
Fade activity metrics;
Flash activity metrics;
Scene change metrics; and
Bit rate metrics.
4. The method of claim 1 , wherein the first predetermined value is 16, the second predetermined value is (256*16), the third predetermined value is 5, the fourth predetermined value is 2560, and the predetermined bitrate is 2 megabits per second.
5. The method of claim 1 , wherein a scene change comprises a scene cut or splice point in an edited sequence of video data.
6. The method of claim 5 , wherein if the scene change is a splice point, the method further comprises providing information to the encoder indicative of:
inserting an I picture at the splice point; and
converting all pictures in the Sub-GOP preceding the splice point into P pictures.
7. The method of claim 5 , wherein the sequence of video picture data comprises interlaced video data such that each picture comprises a top and bottom field, and if the scene change is a scene cut, the method further comprises providing information to the encoder indicative of:
inserting an I picture on a reference picture of the next sub-GOP if the scene cut occurs on a bottom field of a last picture of the sub-GOP under test.
8. The method of claim 1 , wherein the predetermined number of pictures is 6.
9. The method of claim 1 , wherein the method is applied to each Sub-GOP in a sequence of video picture data in turn.
10. A video pre-processor apparatus for pre-processing a sequence of video picture data, said sequence of video picture data comprising I pictures, P pictures and B pictures arranged as a plurality of Group Of Pictures (GOP) and each said Group Of Pictures having a first picture and a plurality of Sub Group of Pictures (SubGOP), the apparatus comprising:
a video sequence analyser configured to analyse the sequence of video picture data, including a position determination processor configured to determining a position of a scene change relative to a position of a start of a new GOP within the sequence of video picture data; and
a GOP adaptation circuit configured to provide information to a video encoder such that the video encoder moves the start of the new GOP to coincide with the scene change by extending or decreasing a current GOP length dependent upon output metrics of the video sequence analyser, if the scene change and the start of a new GOP are within a predetermined number of pictures,
wherein, if the information provided indicates to move the start of the new GOP by extending the current GOP length, the video encoder is configured to extend a current GOP length by:
creating one or more new Sub-GOPs out of P and B pictures; and
inserting the newly created one or more Sub-GOPs into the current GOP until the current GOP is of a sufficient length;
wherein creating the one or more new Sub-GOP includes:
providing a first P picture as the last picture of the one or more sub-GOP; and
inserting B or P pictures previous to the first P picture up to a desired length of Sub-GOP,
wherein a picture type content of the pictures inserted in to a newly-created Sub-GOP is B type pictures, unless an analysis of the sequence of video picture data finds the following Boolean equation to be true:
(Temporal displacement of the sequence of video picture data≧first predetermined value) OR (Temporal distortion of the sequence of video picture data≧second predetermined value) OR (Temporal displacement≦third predetermined value AND Spatial complexity≦fourth predetermined value AND bitrate<a predetermined rate AND (no flashes occur OR no fades occur);
wherein if found true, then P pictures are used in the newly created sub-GOP; wherein:
temporal displacement and temporal distortion are temporal complexity metrics;
flash activity metrics comprise detecting whether flashes occur in a Sub-GOP under test, to provide the ‘no flashes occurs’ Boolean value; and
fade activity metrics comprise detecting whether a fade occurs in the Sub-GOP under test, to provide the ‘no fades occur’ Boolean value.
11. The apparatus of claim 10 , wherein the video sequence analyser further comprises one or more of:
a scene change detector configured to detect a scene change in the sequence of video picture data;
a fade detector configured to detect a fade in the sequence of video picture data;
a flash detector adapted to detect a flash in the sequence of video picture data;
a hierarchical motion estimator configured to provide temporal metrics of the sequence of video picture data;
a spatial complexity estimator configured to provide spatial complexity metrics of the sequence of video picture data;
a first buffer for buffering the sequence of video picture data for use in any of the detector or estimators; and
a delay for delaying the output of the second buffer.
12. The apparatus of claim 10 , further comprising processors.
13. A non-transitory computer-readable medium, carrying instructions, which, when executed, causes one or more computers to carry out the method of claim 1 .