Motion compensated bi-prediction based on local illumination compensation
Systems, methods, and instrumentalities are described herein for calculating local illumination compensation (LIC) parameters for bi-predicted coding unit (CU). The LIC parameters may be used to generate adjusted samples for the current CU and to address local illumination changes that may exist among temporal neighboring pictures. LIC parameters may be calculated based on bi-predicted reference template samples and template samples for a current CU. Bi-predicted reference template samples may be generated based on reference template samples neighboring temporal reference CUs. For example, the bi-predicted reference template samples may be generated based on averaging the reference template samples. The reference template samples may correspond to template samples for the current CU. A CU may be or may include a coding block and/or a sub-block that may be derived by dividing the coding block.
1. A method for video decoding, the method comprising:
obtaining a set of template samples neighboring a video block;
obtaining a first set of reference template samples neighboring a first reference block of the video block;
obtaining a second set of reference template samples neighboring a second reference block of the video block;
calculating a first local illumination compensation (LIC) parameter set and a second LIC parameter set jointly based on the first set of reference template samples, the second set of reference template samples for the video block, and the set of template samples; and
decoding the video block based on the first and the second LIC parameter sets.
2. The method of claim 1 , wherein the first LIC parameter set and the second LIC parameter set are calculated jointly based on jointly minimizing differences between the first set of reference template samples and the set of template samples, and minimizing differences between the second set of reference template samples and the set of template samples.
3. The method of claim 1 , wherein the first set of reference template samples corresponds to the set of template samples neighboring the video block, and wherein the second set of reference template samples corresponds to the set of template samples neighboring the video block.
4. The method of claim 1 , wherein the first LIC parameter set comprises a first LIC scaling parameter and a first LIC offset parameter, and wherein the second LIC parameter set comprises a second LIC scaling parameter and a second LIC offset parameter.
5. The method of claim 1 , wherein the method further comprises:
applying the first LIC parameter set and the second LIC parameter set to generate an adjusted prediction sample for the video block.
6. The method of claim 5 , wherein the adjusted prediction sample comprises a first adjusted prediction sample and a second adjusted prediction sample, and wherein applying the first LIC parameter set and the second LIC parameter set to generate an adjusted prediction sample for the video block comprises:
applying the first LIC parameter set to a first sample in the first reference block to generate a first adjusted prediction sample for the video block; and
applying the second LIC parameter set to a second sample in the second reference block to generate a second adjusted prediction sample for the video block.
7. The method of claim 1 , wherein the video block comprises a video sub-block.
8. A method for video encoding, the method comprising:
obtaining a set of template samples neighboring a video block;
obtaining a first set of reference template samples neighboring a first reference block of the video block;
obtaining a second set of reference template samples neighboring a second reference block of the video block;
calculating a first local illumination compensation (LIC) parameter set and a second LIC parameter set jointly based on the first set of reference template samples, the second set of reference template samples for the video block, and the set of template samples; and
encoding the video block based on the first and the second LIC parameter sets.
9. The method of claim 8 , wherein the first LIC parameter set and the second LIC parameter set are calculated jointly based on jointly minimizing differences between the first set of reference template samples and the set of template samples, and minimizing differences between the second set of reference template samples and the set of template samples.
10. The method of claim 8 , wherein the first set of reference template samples corresponds to the set of template samples neighboring the video block, and wherein the second set of reference template samples corresponds to the set of template samples neighboring the video block.
11. The method of claim 8 , wherein the first LIC parameter set comprises a first LIC scaling parameter and a first LIC offset parameter, and wherein the second LIC parameter set comprises a second LIC scaling parameter and a second LIC offset parameter.
12. The method of claim 8 , wherein the method further comprises:
applying the first LIC parameter set and the second LIC parameter set to generate an adjusted prediction sample for the video block.
13. The method of claim 12 , wherein the adjusted prediction sample comprises a first adjusted prediction sample and a second adjusted prediction sample, and wherein applying the first LIC parameter set and the second LIC parameter set to generate an adjusted prediction sample for the video block comprises:
applying the first LIC parameter set to a first sample in the first reference block to generate a first adjusted prediction sample for the video block; and
applying the second LIC parameter set to a second sample in the second reference block to generate a second adjusted prediction sample for the video block.
14. The method of claim 8 , wherein the video block comprises a video sub-block.
15. A video decoding device, comprising:
a processor configured to:
obtain a set of template samples neighboring a video block;
obtain a first set of reference template samples neighboring a first reference block of the video block;
obtain a second set of reference template samples neighboring a second reference block of the video block;
calculate a first local illumination compensation (LIC) parameter set and a second LIC parameter set jointly based on the first set of reference template samples, the second set of reference template samples for the video block, and the set of template samples; and
decode the video block based on the first and the second LIC parameter sets.
16. The device of claim 15 , wherein the first LIC parameter set and the second LIC parameter set are calculated jointly based on jointly minimizing differences between the first set of reference template samples and the set of template samples, and minimizing differences between the second set of reference template samples and the set of template samples.
17. The device of claim 15 , wherein the first set of reference template samples corresponds to the set of template samples neighboring the video block, and wherein the second set of reference template samples corresponds to the set of template samples neighboring the video block.
18. The device of claim 15 , wherein the first LIC parameter set comprises a first LIC scaling parameter and a first LIC offset parameter, and wherein the second LIC parameter set comprises a second LIC scaling parameter and a second LIC offset parameter.
19. The device of claim 15 , wherein the processor is further configured to:
apply the first LIC parameter set and the second LIC parameter set to generate an adjusted prediction sample for the video block.
20. The device of claim 19 , wherein the adjusted prediction sample comprises a first adjusted prediction sample and a second adjusted prediction sample, and wherein the processor configured to apply the first LIC parameter set and the second LIC parameter set to generate an adjusted prediction sample for the video block comprises the processor being configured to:
apply the first LIC parameter set to a first sample in the first reference block to generate a first adjusted prediction sample for the video block; and
apply the second LIC parameter set to a second sample in the second reference block to generate a second adjusted prediction sample for the video block.