Method, device, and medium for video processing

1 . A method for video processing, comprising:

applying, during a conversion between a target video block of a video and a bitstream of the video, at least one refinement process to the target video block based on a cost, the cost depending on a plurality of reference samples corresponding to the target video block and a plurality of further samples corresponding to a template of the target video block; and

performing the conversion between the target video block and the bitstream based on the applying of the at least one refinement process,

wherein the method further comprises:

enabling a Local Illumination Compensation (LIC) for a first reference picture list for the target video block;

disabling the LIC for a second reference picture list for the target video block;

determining an LIC prediction from the first reference picture list;

determining a regular prediction from the second reference picture list; and

calculating a weighted average of the LIC prediction and the regular prediction to obtain a final prediction for the target video block.

2 . The method of claim 1 , wherein applying the at least one refinement process comprises:

applying a first refinement process to the target video block based on a first template to obtain first refined information; and

applying a second refinement process based on the first refined information to the target video block based on a second template to obtain second refined information, the second template being different from the first template.

3 . The method of claim 2 , wherein the number of samples contained in the first template is greater than the number of samples contained in the second template; or

wherein the first template contains at least one sample different from samples contained in the second template; or

wherein a first cost metric or a first error metric is used in the first refinement process, a second cost metric or a second error metric is used in the second refinement process, the second cost metric being different from the first cost metric, and the second error metric being different from the first error metric.

4 . The method of claim 1 , wherein at least one of whether to use multiple refinement processes, a selection of one or more error metrics, a definition of a cost function, or samples involved in an error metric based on at least one of:

decoded information of the target video block,

decoded information of one or more neighboring blocks of the target video block,

a block dimension,

a temporal layer,

a low delay check flag,

a similarity between motion information of the target video block and motion information of a neighboring block of the target video block, or

a reference picture list or reference picture information for the target video block.

5 . The method of claim 4 , wherein the selection of the one or more error metrics or the definition of the cost metric depends on at least one of:

a coding tool applied to the target video block, or

a coding tool applied to the neighboring block.

6 . The method of claim 5 , wherein the coding tool applied to the target video block or the coding tool applied to the neighboring block comprises a Local Illumination Compensation (LIC).

7 . The method of claim 6 , wherein the selection of the one or more error metrics is based on a LIC flag of the target video block,

wherein if the LIC flag is false, the error metric used in a template matching cost measurement or a bilateral matching cost measurement is determined as one of the following:

a sum of absolute differences (SAD),

a sum of absolute transformed differences (SATD), or

a sum of squared differences (SSD); and

if the LIC flag is true, the error metric used in a template matching cost measurement or a bilateral matching cost measurement is determined as one of the followings:

a mean-removed sum of absolute differences (MR-SAD),

a mean-removed sum of absolute transformed differences (MR-SATD), or

a mean-removed sum of squared differences (MR-SSD).

8 . The method of claim 6 , further comprising:

in response to a mean-removed sum of absolute differences (MR-SAD), a mean-removed sum of absolute transformed differences (MR-SATD), or a mean-removed sum of squared differences (MR-SSD) being used in a template matching cost measurement, determining a reference template for the template matching without using LIC process; and

in response to a MR-SAD, a MR-SATD or a MR-SSD being used in a bilateral matching cost measurement, determining a reference block for the bilateral matching without using LIC process.

9 . The method of claim 4 , further comprising:

determining a first cost for the first reference picture list based on a first error metric;

determining a second cost for the second reference picture list based on a second error metric; and

determining the cost based on the first cost and the second cost.

10 . The method of claim 9 , wherein during the selection of the one or more error metrics, the first error metric for a first reference picture list is determined to be a sum of absolute differences (SAD) or a sum of squared error (SSE), and

the second error metric for a second reference picture list is determined to be a mean-removed sum of absolute differences (MR-SAD) or a mean-removed sum of squared error (MR-SSE).

11 . The method of claim 4 , wherein the neighboring block of the target video block comprises at least one of: an adjacent neighboring block or non-adjacent neighboring block.

12 . The method of claim 1 , wherein the reference sample corresponding to the target video block comprises a prediction sample of the target video block, the prediction sample comprising a uni-predicted sample or a bi-predicted sample.

13 . The method of claim 1 , wherein the further sample corresponding to the template of the target video block comprises at least one of:

a reference or prediction sample corresponding to the template of the target video block, or

a reconstructed sample in the template of the target video block.

14 . The method of claim 1 , wherein the target video block comprises one of:

a template-based-coded block, or

a bilateral-based-coded block.

15 . The method of claim 14 , wherein the target video block comprises a bilateral-based-coded block, and the at least one refinement process comprises a decoded motion vector refinement (DMVR) process.

16 . The method of claim 1 , wherein the conversion includes encoding the target video block into the bitstream, or

wherein the conversion includes decoding the target video block from the bitstream.

17 . An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

apply, during a conversion between a target video block of a video and a bitstream of the video, at least one refinement process to the target video block based on a cost, the cost depending on a plurality of reference samples corresponding to the target video block and a plurality of further samples corresponding to a template of the target video block; and

perform the conversion based on the applying,

wherein the processor is further caused to:

enable a Local Illumination Compensation (LIC) for a first reference picture list for the target video block;

disable the LIC for a second reference picture list for the target video block;

determine an LIC prediction from the first reference picture list;

determine a regular prediction from the second reference picture list; and

calculate a weighted average of the LIC prediction and the regular prediction to obtain a final prediction for the target video block.

18 . A non-transitory computer-readable storage medium storing instructions that cause a processor to perform a method for video processing comprising:

applying, during a conversion between a target video block of a video and a bitstream of the video, at least one refinement process to the target video block based on a cost, the cost depending on a plurality of reference samples corresponding to the target video block and a plurality of further samples corresponding to a template of the target video block; and

performing the conversion based on the applying,

wherein the method further comprises:

enabling a Local Illumination Compensation (LIC) for a first reference picture list for the target video block;

disabling the LIC for a second reference picture list for the target video block;

determining an LIC prediction from the first reference picture list;

determining a regular prediction from the second reference picture list; and

calculating a weighted average of the LIC prediction and the regular prediction to obtain a final prediction for the target video block.

19 . The method of claim 1 , further comprising:

storing the bitstream in a non-transitory computer-readable recording medium.