Template matching patterns and fusion for template matching related video coding

1 . A method of encoding or decoding video data, the method comprising:

constructing a template matching candidate list for a current block of the video data based on a plurality of template patterns comprising:

including a first set of reference block candidates in the template matching candidate list based on a first template pattern of the plurality of template patterns; and

including a second set of reference block candidates in the template matching candidate list based on a second template pattern of the plurality of template patterns; and

encoding or decoding the current block based on the template matching candidate list.

2 . The method of claim 1 , wherein constructing the template matching candidate list comprises:

constructing a first list of reference block candidates based on template matching a first template of the current block, the first template having a first template pattern of the plurality of template patterns, and respective first templates of each of a plurality of reference blocks, the respective first templates having the first template pattern;

constructing a second list of reference block candidates based on template matching a second template of the current block, the second template having a second template pattern of the plurality of template patterns, and respective second templates of each of the plurality of reference blocks, the respective second templates having the second template pattern; and

constructing the template matching candidate list based on the first list of reference block candidates and the second list of reference block candidates.

3 . The method of claim 2 , further comprising:

constructing a third list of reference block candidates based on template matching a third template, having a third template pattern of the plurality of template patterns, of the current block and respective third templates, having the third template pattern, of each of the plurality of reference blocks,

wherein constructing the template matching candidate list comprises constructing the template matching candidate list based on the first list of reference block candidates, the second list of reference block candidates, and the third list of reference block candidates.

4 . The method of claim 2 , wherein the first template, having the first template pattern, comprises samples in one or more rows above and one or more columns left of the current block, and wherein the second template, having the second template pattern, comprises one of:

only samples in the one or more rows above the current block, or

only samples in the one or more columns left of the current block.

5 . The method of claim 2 , wherein constructing the template matching candidate list comprises:

determining that a reference block candidate in the second list of reference block candidates is not present in the first list of reference block candidates; and

inserting the reference block candidate in the second list of reference block candidates into the template matching candidate list.

6 . The method of claim 2 , wherein constructing the template matching candidate list comprises:

determining that a reference block candidate in the second list of reference block candidates is present in the first list of reference block candidates; and

avoiding inserting the reference block candidate in the second list of reference block candidates into the template matching candidate list.

7 . The method of claim 2 , wherein constructing the first list of reference block candidates comprises:

determining first respective template matching cost values between the first template and respective templates of each of the plurality of reference blocks; and

constructing the first list of reference block candidates based on the first respective template matching cost values, wherein constructing the second list of reference block candidates comprises:

determining second respective template matching cost values between the second template and respective templates of each of the plurality of reference blocks; and

constructing the second list of reference block candidates based on the second respective template matching cost values.

8 . The method of claim 7 , wherein determining the first respective template matching cost values comprises determining the first respective template matching cost values based at least in part on the determined second respective template matching cost values.

9 . The method of claim 1 , wherein the current block comprises a first block, and wherein the template matching candidate list is a first template matching candidate list, the method further comprising:

determining that a second block is along a boundary of a picture;

constructing a second template matching candidate list based on a subset of the plurality of template patterns, wherein the subset of the plurality of template patterns comprise template patterns for which samples outside the boundary are not needed; and

encoding or decoding the second block based on the second template matching candidate list.

10 . The method of claim 1 , further comprising:

determining a plurality of reference block candidates; and

fusing the plurality of reference block candidates to generate a prediction block, wherein encoding or decoding the current block comprises encoding or decoding the current block based on the prediction block.

11 . The method of claim 10 , wherein fusing the plurality of reference block candidates comprises:

determining a respective weight for each of the plurality of reference block candidates; and

fusing the plurality of reference block candidates based on the respective weight for each of the plurality of reference block candidates.

12 . The method of claim 11 , wherein determining the respective weight comprises determining the respective weight for each of the plurality of reference block candidates based on a respective template matching cost value of each of the plurality of reference block candidates.

13 . The method of claim 11 , wherein determining the respective weight comprises determining the respective weight for each of the plurality of reference block candidates based on a mean-squared-error minimization.

14 . The method of claim 1 , wherein encoding or decoding the current block based on the template matching candidate list comprises decoding the current block based on the template matching candidate list, wherein decoding the current block comprises:

receiving an index into the template matching candidate list;

determining a prediction block based on the index;

receiving residual information indicative of a difference between the prediction block and the current block; and

reconstructing the current block based on the prediction block and the residual information.

15 . The method of claim 1 , wherein encoding or decoding the current block based on the template matching candidate list comprises encoding the current block based on the template matching candidate list, wherein encoding the current block comprises:

signaling an index into the template matching candidate list that a video decoder uses for determining a prediction block; and

signaling residual information indicative of a difference between the prediction block and the current block.

16 . A device for encoding or decoding video data, the device comprising:

one or more memories configured to store the video data; and

processing circuitry coupled to the one or more memories, wherein the processing circuitry is configured to:

construct a template matching candidate list for a current block of the video data based on a plurality of template patterns, wherein to construct the template matching candidate list, the processing circuitry is configured to:

include a first set of reference block candidates in the template matching candidate list based on a first template pattern of the plurality of template patterns; and

include a second set of reference block candidates in the template matching candidate list based on a second template pattern of the plurality of template patterns; and

encode or decode the current block based on the template matching candidate list.

17 . The device of claim 16 , wherein to construct the template matching candidate list, the processing circuitry is configured to:

construct a first list of reference block candidates based on template matching a first template of the current block, the first template having a first template pattern of the plurality of template patterns, and respective first templates of each of a plurality of reference blocks, the respective first templates having the first template pattern;

construct a second list of reference block candidates based on template matching a second template of the current block, the second template having a second template pattern of the plurality of template patterns, and respective second templates of each of the plurality of reference blocks, the respective second templates having the second template pattern; and

construct the template matching candidate list based on the first list of reference block candidates and the second list of reference block candidates.

18 . The device of claim 16 , wherein the processing circuitry is configured to:

determine a plurality of reference block candidates; and

fuse the plurality of reference block candidates to generate a prediction block, wherein to encode or decode the current block, the processing circuitry is configured to encode or decode the current block based on the prediction block.

19 . A non-transitory computer-readable storage medium storing instructions thereon that when executed cause one or more processors to:

construct a template matching candidate list for a current block of video data based on a plurality of template patterns, wherein to construct the template matching candidate list, the instructions further cause the one or more processors to:

include a first set of reference block candidates in the template matching candidate list based on a first template pattern of the plurality of template patterns; and

include a second set of reference block candidates in the template matching candidate list based on a second template pattern of the plurality of template patterns; and

encode or decode the current block based on the template matching candidate list.