Method of background residual prediction for video coding
A method and apparatus for video encoding or decoding utilizing adaptive background residual prediction is disclosed. The present invention adaptively applies background residual prediction to a current block based on a selection decision. The coding block is split into one or more coding sub-blocks. A reference sub-block in a reference picture is located for a current coding sub-block of the current coding block according to a motion vector associated with the current coding block. A background reference sub-block in a background picture is located for the reference sub-block, where the background reference sub-block is at a first co-located location as the reference sub-block. The method then selects a first predictor or a second predictor to encode or decode the current sub-block based on a selection decision. The first predictor corresponds to the reference sub-block, and the second predictor is derived according to the reference sub-block and the background picture.
1. A method of video encoding or decoding utilizing adaptive background residual prediction, the method comprising:
receiving input data associated with a current block in a current picture;
locating a reference block in a reference picture for the current block according to a motion vector associated with the current block;
locating a background reference block in a background picture for the reference block, wherein the background reference block is positioned at a first set of location coordinates (x1, y1) in the background picture, and the reference block is positioned at the same first set of location coordinates (x1, y1) in the reference picture; and
selecting a first predictor or a second predictor to encode or decode the current block based on a selection decision, wherein the first predictor corresponds to the reference block and the second predictor is derived according to at least one of the reference block and the background picture.
2. The method of claim 1 , wherein the background picture is generated based on one or more decoded pictures.
3. The method of claim 1 , wherein the selection decision is derived based on the reference block and the background reference block.
4. The method of claim 3 , wherein a sum of absolute differences (SAD) or a mean squared error (MSE) between the reference block and the background reference block is derived, and the first predictor is selected if the SAD or the MSE is greater than a threshold and the second predictor is selected if the SAD or the MSE is smaller than the threshold.
5. The method of claim 3 , wherein absolute differences between the reference block and the background reference block are derived, the first predictor is selected if a number of absolute differences exceeding a threshold is larger than a selected number and the second predictor is selected otherwise, and wherein the selected number corresponds to a non-negative integer.
6. The method of claim 1 , wherein the selection decision is indicated by a flag.
7. The method of claim 1 , further comprising incorporating a syntax element in an encoder side or parsing the syntax element in a decoder side, wherein the syntax element indicates whether to enable adaptive background residual prediction.
8. The method of claim 7 , wherein the syntax element is incorporated in a picture level or a slice header level of a video bitstream to control selection of the adaptive background residual prediction for a respective picture or slice.
9. The method of claim 7 , wherein the syntax element is incorporated in coded data for the current coding block to control selection of the adaptive background residual prediction for the current coding block.
10. The method of claim 9 , wherein said incorporating the syntax element in the encoder side and said parsing the syntax element in the decoder side are skipped for the current coding block if the current coding block is coded using Merge mode or Skip mode.
11. The method of claim 9 , wherein said incorporating the syntax element in the encoder side and said parsing the syntax element in the decoder side are skipped for the current coding block if the current coding block is predicted using prediction units smaller than the current coding block.
12. The method of claim 9 , wherein said incorporating the syntax element in the encoder side and said parsing the syntax element in the decoder side are skipped for the current coding block if width or height of the current coding block is equal to or smaller than a selected size, wherein the selected size corresponds to a positive integer.
13. The method of claim 12 , wherein the selected size is eight or sixteen.
14. The method of claim 1 , wherein width of the current block is from 1 to picture width and height of the current block is from 1 to picture height.
15. The method of claim 1 , wherein the second predictor corresponds to a linear combination of the reference block, the background reference block and a background current block in the background picture, and wherein the background current block is positioned at a second set of location coordinates (x2, y2) in the background picture, and the current block is positioned at the same second set of location coordinates (x2, y2) in the current picture.
16. The method of claim 15 , wherein the second predictor is set to a sum of a reference residual and the background current block, and wherein the reference residual is derived by subtracting the background reference block from the reference block.
17. The method of claim 1 , wherein the motion vector associated with the current coding block is derived in a picture domain or a background residual domain, wherein the current picture and the reference picture are used to derive the motion vector for the picture domain, and current background residual data and reference background residual data are used to derive the motion vector for the background residual domain, the current background residual data corresponds to first differences between the current picture and the background picture, and the reference background residual data corresponds to second differences between the reference picture and the background picture.
18. An apparatus for video encoding or decoding utilizing adaptive background residual prediction, the apparatus comprising one or more electronic circuits configured to:
receive input data associated with a current block in a current picture;
locate a reference block in a reference picture for the current block according to a motion vector associated with the current coding block;
locate a background reference block in a background picture for the reference block, wherein the background reference block is positioned at a first set of location coordinates (x1, y1) in the background picture, and the reference block is positioned at the same first set of location coordinates (x1, y1) in the reference picture; and
select a first predictor or a second predictor to encode or decode the current block based on a selection decision, wherein the first predictor corresponds to the reference block and the second predictor is derived according to at least one of the reference block and the background picture.
19. A non-transitory computer readable medium storing a computer-executable program, the computer-executable program, when executed, causing a decoder to perform the following steps:
receiving input data associated with a current block in a current picture;
locating a reference block in a reference picture for the current block according to a motion vector associated with the current block;
locating a background reference block in a background picture for the reference block, wherein the background reference block is positioned at a first set of location coordinates (x1, y1) in the background picture, and the reference block is positioned at the same first set of location coordinates (x1, y1) in the reference picture; and
selecting a first predictor or a second predictor to decode the current block based on a selection decision, wherein the first predictor corresponds to the reference block and the second predictor is derived according to at least one of the reference block and the background picture.