Motion compensated bi-prediction based on local illumination compensation
Systems, methods, and instrumentalities are described herein for calculating local illumination compensation (LIC) parameters for bi-predicted coding unit (CU). The LIC parameters may be used to generate adjusted samples for the current CU and to address local illumination changes that may exist among temporal neighboring pictures. LIC parameters may be calculated based on bi-predicted reference template samples and template samples for a current CU. Bi-predicted reference template samples may be generated based on reference template samples neighboring temporal reference CUs. For example, the bi-predicted reference template samples may be generated based on averaging the reference template samples. The reference template samples may correspond to template samples for the current CU. A CU may be or may include a coding block and/or a sub-block that may be derived by dividing the coding block.
1. A method for video decoding, the method comprising:
obtaining a current coding block;
obtaining a set of template samples neighboring the current coding block;
obtaining a set of reference template samples associated with a temporal reference block of the current coding block, the set of reference template samples corresponding to the set of template samples neighboring the current coding block;
calculating a local illumination compensation (LIC) scaling parameter and a LIC offset parameter jointly based on the set of reference template samples and the set of template samples for the current coding block; and
applying the LIC scaling parameter and the LIC offset parameter to generate an adjusted prediction sample for the current coding block.
2. The method of claim 1 , wherein the LIC scaling parameter and the LIC offset parameter are calculated jointly based on minimizing differences between the set of reference template samples and the set of template samples neighboring the current coding block.
3. The method of claim 1 , wherein the temporal reference block is a first temporal reference block of the current coding block, the set reference template samples are a first set of reference template samples neighboring the first temporal reference block, the LIC scaling parameter is a first LIC scaling parameter, the LIC offset is a first LIC scaling offset parameter, and the method further comprises:
determining that the current coding block is bi-predicted; and
obtaining a second set of reference template samples neighboring a second temporal reference block of the current coding block, the second set of reference template samples corresponding to the set of template samples neighboring the current coding block, wherein calculating the first LIC scaling parameter and first LIC offset parameter jointly further comprises:
calculating the first LIC scaling parameter, the first LIC offset parameter, a second LIC scaling parameter and a second LIC offset parameter jointly based on the first set of reference template samples, the second set of reference template samples, and the set of template samples for the current coding block.
4. The method of claim 3 , wherein the first LIC scaling parameter, the first LIC offset parameter, the second LIC scaling parameter, and the second LIC offset parameter are calculated jointly based on minimizing differences between the first set of reference template samples and the set of template samples neighboring the current coding block, and minimizing the differences between the second set of reference template samples and the set of template samples neighboring the current coding block.
5. The method of claim 3 , further comprising:
applying the first LIC scaling parameter and the first LIC offset parameter to a sample in the first temporal reference block to generate a first adjusted prediction sample for the current coding block; and
applying the second LIC scaling parameter and the second LIC offset parameter to a sample in the second temporal reference block to generate a second adjusted prediction sample for the current coding block.
6. The method of claim 1 , further comprising:
determining that the current coding block is bi-predicted;
identifying a first set of reference template samples neighboring a first temporal reference block and a second set of reference template samples neighboring a second temporal reference block, the first set of reference template samples and the second set of reference template samples corresponding to the set of template samples neighboring the current coding block; and
generating a set of bi-predicted reference template samples based on the identified first set of reference template samples and the identified second set of reference template samples, wherein the set of reference template samples are the set of bi-predicted reference template samples.
7. The method of claim 1 , wherein the current block comprises a current sub-block.
8. A device for video decoding, comprising:
a processor configured to:
obtain a current coding block;
obtain a set of template samples neighboring the current coding block;
obtain a set of reference template samples associated with a temporal reference block of the current coding block, the set of reference template samples corresponding to the set of template samples neighboring the current coding block;
calculate a local illumination compensation (LIC) scaling parameter and a LIC offset parameter jointly based on the set of reference template samples and the set of template samples for the current coding block; and
apply the LIC scaling parameter and the LIC offset parameter to generate an adjusted prediction sample for the current coding block.
9. The device of claim 8 , wherein the LIC scaling parameter and the LIC offset parameter are calculated jointly based on minimizing differences between the set of reference template samples and the set of template samples neighboring the current coding block.
10. The device of claim 8 , wherein the temporal reference block is a first temporal reference block of the current coding block, the set reference template samples are a first set of reference template samples neighboring the first temporal reference block, the LIC scaling parameter is a first LIC scaling parameter, the LIC offset is a first LIC scaling offset parameter, and the processor is further configured to:
determine that the current coding block is bi-predicted; and
obtain a second set of reference template samples neighboring a second temporal reference block of the current coding block, the second set of reference template samples corresponding to the set of template samples neighboring the current coding block, wherein calculating the first LIC scaling parameter and first LIC offset parameter jointly further comprises:
calculating the first LIC scaling parameter, the first LIC offset parameter, a second LIC scaling parameter, and a second LIC offset parameter jointly based on the first set of reference template samples, the second set of reference template samples, and the set of template samples for the current coding block.
11. The device of claim 10 , wherein the first LIC scaling parameter, the first LIC offset parameter, the second LIC scaling parameter, and the second LIC offset parameter are calculated jointly based on minimizing differences between the first set of reference template samples and the set of template samples neighboring the current coding block, and minimizing differences between the second set of reference template samples and the set of template samples neighboring the current coding block.
12. The device of claim 10 , wherein the first LIC scaling parameter, the first LIC offset parameter, the second LIC scaling parameter, and the second LIC offset parameter are calculated jointly based on minimizing differences between the first set of reference template samples and the set of template samples neighboring the current coding block, and simultaneously minimizing differences between the second set of reference template samples and the set of template samples neighboring the current coding block.
13. The device of claim 10 , wherein the processor is further configured to:
apply the first LIC scaling parameter and the first LIC offset parameter to a sample in the first temporal reference block to generate a first adjusted prediction sample for the current coding block; and
apply the second LIC scaling parameter and the second LIC offset parameter to a sample in the second temporal reference block to generate a second adjusted prediction sample for the current coding block.
14. The device of claim 8 , wherein the processor is further configured to:
reconstruct the current coding block based on the adjusted prediction sample.
15. The device of claim 10 , further comprising a memory.
16. A method for video encoding, the method comprising:
obtaining a current coding block;
obtaining a set of template samples neighboring the current coding block;
obtaining a set of reference template samples associated with a temporal reference block of the current coding block, the set of reference template samples corresponding to the set of template samples neighboring the current coding block;
calculating a local illumination compensation (LIC) scaling parameter and a LIC offset parameter jointly based on the set of reference template samples and the set of template samples for the current coding block; and
using the LIC scaling parameter and the LIC offset parameter to generate a residual block for the current coding block.
17. The method of claim 16 , wherein the LIC scaling parameter and the LIC offset parameter are calculated jointly based on minimizing differences between the set of reference template samples and the set of template samples neighboring the current coding block.
18. The method of claim 16 , wherein the temporal reference block is a first temporal reference block of the current coding block, the set of reference template samples are a first set of reference template samples neighboring the first temporal reference block, the LIC scaling parameter is a first LIC scaling parameter, the LIC offset is a first LIC scaling offset parameter, and the method further comprises:
determining that the current coding block is bi-predicted; and
obtaining a second set of reference template samples neighboring a second temporal reference block of the current coding block, the second set of reference template samples corresponding to the set of template samples neighboring the current coding block, wherein calculating the first LIC scaling parameter and first LIC offset parameter jointly further comprises:
calculating the first LIC scaling parameter, the first LIC offset parameter, a second LIC scaling parameter, and a second LIC offset parameter jointly based on the first set of reference template samples, the second set of reference template samples, and the set of template samples for the current coding block.
19. The method of claim 18 , wherein the first LIC scaling parameter, the first LIC offset parameter, the second LIC scaling parameter, and the second LIC offset parameter are calculated jointly based on minimizing differences between the first set of reference template samples and the set of template samples neighboring the current coding block, and minimizing the differences between the second set of reference template samples and the set of template samples neighboring the current coding block.
20. The method of claim 18 , wherein the method further comprises:
applying the first LIC scaling parameter and the first LIC offset parameter to a sample in the first temporal reference block to generate a first adjusted prediction sample for the current coding block; and
applying the second LIC scaling parameter and the second LIC offset parameter to a sample in the second temporal reference block to generate a second adjusted prediction sample for the current coding block.