Local illumination compensation with multiple linear models
Systems, methods, and instrumentalities are disclosed for performing local illumination compensation (LIC) with multiple linear models. In an example, a device, such as a video decoding device, or a video encoding device, may determine that sub-partition-based local illumination compensation (sub-LIC) is enabled for a current block. The device may divide the current block into a plurality of sub-partitions. The device may identify a plurality of templates associated with the plurality of sub-partitions. The device may derive a plurality of local illumination compensation (LIC) parameter sets associated with the plurality of sub-partitions based on the plurality of templates associated with the plurality of sub-partitions. The device may process (e.g., encode and/or decode) the current block based on the plurality of LIC parameter sets.
1 . A video decoding device, comprising:
a processor configured to:
determine that sub-partition-based local illumination compensation (sub-LIC) is enabled for a current block;
divide the current block into a first sub-partition and a second sub-partition;
identify a first template associated with the first sub-partition;
derive a first local illumination compensation (LIC) parameter set based on the first template associated with the first sub-partition;
identify a second template associated with the second sub-partition;
derive a second LIC parameter set based on the second template associated with the second sub-partition;
apply the first LIC parameter set on a reference block of the current block to obtain a first prediction block of the current block;
apply the second LIC parameter set on the reference block of the current block to obtain a second prediction block of the current block; and
decode the current block based on the first prediction block and the second prediction block.
2 . The device of claim 1 , the processor further configured to:
obtain a sub-LIC enablement indicator configured to indicate whether sub-LIC is enabled for the current block, wherein that sub-LIC is enabled for the current block is determined based on the sub-LIC enablement indicator indicating that sub-LIC is enabled for the current block.
3 . The device of claim 1 , the processor further configured to:
determine whether a sub-LIC direction indication is included in video data based on a sub-LIC enablement indicator; and
based on the sub-LIC enablement indicator indicating that sub-LIC is enabled, obtain the sub-LIC direction indication from the video data, wherein the current block is divided into the first sub-partition and the second sub-partition based on the sub-LIC direction indication.
4 . The device of claim 1 , wherein the first template corresponds to a left sub-partition of the current block, and the first template comprises reconstructed samples neighboring a left border and a top border of the left sub-partition of the current block, and the second template corresponds to a right sub-partition of the current block, and the second template comprises reconstructed samples neighboring a top border of the right sub-partition of the current block.
5 . The device of claim 1 , wherein the first template corresponds to a top sub-partition of the current block, and the first template comprises reconstructed samples neighboring a left border and a top border of the top sub-partition of the current block, and the second template corresponds to a bottom sub-partition of the current block, and the second template comprises reconstructed samples neighboring a left border of the bottom sub-partition of the current block.
6 . The device of claim 1 , wherein the processor is further configured to:
apply the first LIC parameter set on a plurality of reference samples of the first sub-partition to obtain a plurality of prediction samples of the first sub-partition;
apply the second LIC parameter set on a plurality of reference samples of the second sub-partition to obtain a plurality of prediction samples of the second sub-partition; and
obtain a prediction block of the current block by combining the plurality of prediction samples of the first sub-partition and the plurality of prediction samples of the second sub-partition, the current block being decoded based on the prediction block.
7 . A method for video decoding, the method comprising:
determining that sub-partition-based local illumination compensation (sub-LIC) is enabled for a current block;
dividing the current block into a first sub-partition and a second sub-partition;
identifying a first template associated with the first sub-partition;
deriving a first local illumination compensation (LIC) parameter set based on the first template associated with the first sub-partition;
identifying a second template associated with the second sub-partition;
deriving a second LIC parameter set based on the second template associated with the second sub-partition;
applying the first LIC parameter set on a reference block of the current block to obtain a first prediction block of the current block;
applying the second LIC parameter set on the reference block of the current block to obtain a second prediction block of the current block; and
decoding the current block based on the first prediction block and the second prediction block.
8 . The method of claim 7 , the method further comprising:
obtaining a sub-LIC enablement indicator configured to indicate whether sub-LIC is enabled for the current block, wherein that sub-LIC is enabled for the current block is determined based on the sub-LIC enablement indicator indicating that sub-LIC is enabled for the current block.
9 . The method of claim 7 , the method further comprising:
determining whether a sub-LIC direction indication is included in video data based on a sub-LIC enablement indicator; and
based on the sub-LIC enablement indicator indicating that sub-LIC is enabled, obtaining the sub-LIC direction indication from the video data, wherein the current block is divided into the first sub-partition and the second sub-partition based on the sub-LIC direction indication.
10 . The method of claim 7 , wherein the first template corresponds to a left sub-partition of the current block, and the first template comprises reconstructed samples neighboring a left border and a top border of the left sub-partition of the current block, and the second template corresponds to a right sub-partition of the current block, and the second template comprises reconstructed samples neighboring a top border of the right sub-partition of the current block.
11 . The method of claim 7 , wherein the first template corresponds to a top sub-partition of the current block, and the first template comprises reconstructed samples neighboring a left border and a top border of the top sub-partition of the current block, and the second template corresponds to a bottom sub-partition of the current block, and the second template comprises reconstructed samples neighboring a left border of the bottom sub-partition of the current block.
12 . The method of claim 7 , wherein the method further comprises:
applying the first LIC parameter set on a plurality of reference samples of the first sub-partition to obtain a plurality of prediction samples of the first sub-partition;
applying the second LIC parameter set on a plurality of reference samples of the second sub-partition to obtain a plurality of prediction samples of the second sub-partition; and
obtaining a prediction block of the current block by combining the plurality of prediction samples of the first sub-partition and the plurality of prediction samples of the second sub-partition, the current block being decoded based on the prediction block.
13 . A video encoding device, comprising:
a processor configured to:
determine to enable sub-partition-based local illumination compensation (sub-LIC) for a current block;
divide the current block into a first sub-partition and a second sub-partition;
identify a first template associated with the first sub-partition;
derive a first local illumination compensation (LIC) parameter set based on the first template associated with the first sub-partition;
identify a second template associated with the second sub-partition;
derive a second LIC parameter set based on the second template associated with the second sub-partition;
apply the first LIC parameter set on a reference block of the current block to obtain a first prediction block of the current block;
apply the second LIC parameter set on the reference block of the current block to obtain a second prediction block of the current block; and
encode the current block based on the first prediction block and the second prediction block.
14 . The device of claim 13 , the processor further configured to:
based on determining to enable sub-LIC for the current block, include, in video data, a sub-LIC enablement indicator configured to indicate that sub-LIC is enabled for the current block.
15 . The device of claim 13 , the processor further configured to:
based on determining to enable sub-LIC for the current block, determine a sub-LIC direction for dividing the current block, wherein the current block is divided into the first sub-partition and the second sub-partition based on the determined sub-LIC direction; and
include a sub-LIC direction indication configured to indicate the determined sub-LIC direction in video data.
16 . The device of claim 13 , wherein a first template of the plurality of templates corresponds to a left sub-partition of the current block, and the first template comprises samples neighboring a left border and a top border of the left sub-partition of the current block, and a second template of the plurality of templates corresponds to a right sub-partition of the current block, and the second template comprises samples neighboring a top border of the right sub-partition of the current block.
17 . A method for video encoding, the method comprising:
determining to enable sub-partition-based local illumination compensation (sub-LIC) for a current block;
dividing the current block into a first sub-partition and a second sub-partition;
identifying a first template associated with the first sub-partition;
deriving a first local illumination compensation (LIC) parameter set based on the first template associated with the first sub-partition;
identifying a second template associated with the second sub-partition;
deriving a second LIC parameter set based on the second template associated with the second sub-partition;
applying the first LIC parameter set on a reference block of the current block to obtain a first prediction block of the current block;
applying the second LIC parameter set on the reference block of the current block to obtain a second prediction block of the current block; and
encoding the current block based on the first prediction block and the second prediction block.
18 . The method of claim 17 , the method further comprising:
based on determining to enable sub-LIC for the current block, including, in video data, a sub-LIC enablement indicator configured to indicate that sub-LIC is enabled for the current block.
19 . The method of claim 17 , the method further comprising:
based on determining to enable sub-LIC for the current block, determining a sub-LIC direction for dividing the current block, wherein the current block is divided into the first sub-partition and the second sub-partition based on the determined sub-LIC direction; and
including a sub-LIC direction indication configured to indicate the determined sub-LIC direction in video data.
20 . The method of claim 17 , wherein a first template of the plurality of templates corresponds to a left sub-partition of the current block, and the first template comprises samples neighboring a left border and a top border of the left sub-partition of the current block, and a second template of the plurality of templates corresponds to a right sub-partition of the current block, and the second template comprises samples neighboring a top border of the right sub-partition of the current block.