Intra block copy with template matching for video encoding and decoding
In a video coding system, it is proposed to improve the intra prediction based on intra-block copy by allowing a decoder-side refinement allowing further refinement of the block vector obtained from IBC using a template matching process. In this template matching, the decoder searches in an iterative process the best match for the current block (L-shaped) template within the reconstructed area, adjacent to the block vector determined using the IBC 10 prediction. This technique allows to obtain a finer precision that what is possible with IBC while minimizing the amount of coding information. In at least one embodiment where a coarse precision is used for IBC, it allows at the same time to benefit from significant coding gain provided by IBC while providing a much finer precision with reduced signaling of the block vector.
1 . A method comprising:
determining a template based on L-shaped samples surrounding a block of a picture of a video;
determining an initial location for an initial iteration;
selecting a matching template among a set of template candidates in reconstructed blocks of the picture, wherein the set of template candidates is determined based on a determined location and wherein the matching template is selected based on a similarity measure between samples of the template and samples of the template candidates;
refining the selection by iterating over the selection step while the similarity of a template candidate is better than the similarity of the other template candidates and, on a condition that the similarity of the template candidate at a current location is better than the similarity of the other template candidates of the set of template candidates, refining the selection by iterating over a further set of template candidates for which the location is determined with a finer precision; and
reconstructing the block by copying samples of the selected matching template into the block,
wherein, for the initial iteration over the selection step, the location is determined by the initial location and wherein, for further iterations, the location is determined as a refinement based on the location of the selected matching template of the previous iteration.
2 . The method of claim 1 , wherein the set of template candidates are selected based on a precision.
3 . The method of claim 2 , wherein the initial location for the initial iteration of the selection step is determined using a coarse precision of 8 or 16 pixels.
4 . The method of claim 1 , wherein the initial location for the initial iteration of the selection step is determined by techniques of motion coding modes such as merge mode or advanced motion vector prediction and wherein the motion coding modes apply to reconstructed blocks of the picture.
5 . The method of claim 2 , wherein the initial precision for the initial iteration of the selection step is obtained from information in the bitstream.
6 . The method of claim 2 , wherein the set of template candidates is determined by selecting candidates according to a sampling of a search region according to the precision.
7 . The method of claim 2 , wherein the set of template candidates is determined by selecting candidates according to a location and precision in both horizontal and vertical directions.
8 . The method of claim 1 , further comprising:
decoding picture data using the reconstructed block.
9 . The method of claim 1 , further comprising:
encoding picture data using the reconstructed block.
10 . An apparatus for video decoding, the apparatus comprising one or more processors configured to:
determine a template based on L-shaped samples surrounding a block of a picture of a video;
determine an initial location for an initial iteration;
select a matching template among a set of template candidates in reconstructed blocks of the picture, wherein the set of template candidates is determined based on a determined location and wherein the matching template is selected based on a similarity measure between samples of the template and samples of the template candidates;
refine the selection by iterating over the selection step while the similarity of a template candidate is better than the similarity of the other template candidates and, on a condition that the similarity of the template candidate at a current location is better than the similarity of the other template candidates of the set of template candidates, refine the selection by iterating over a further set of template candidates for which the location is determined with a finer precision;
reconstruct the block by copying samples of the selected matching template into the block, wherein, for the initial iteration over the selection step, the location is determined by the initial location and wherein, for further iterations, the location is determined as a refinement based on the location of the selected matching template of the previous iteration; and
decode picture data using the reconstructed block.
11 . The apparatus of claim 10 , wherein the set of template candidates are selected based on a precision.
12 . The apparatus of claim 11 , wherein the initial location for the initial iteration of the selection step is determined using a coarse precision of 8 or 16 pixels.
13 . The apparatus of claim 11 , wherein the initial precision for the initial iteration of the selection step is obtained from information in the bitstream.
14 . The apparatus of claim 11 , wherein the set of template candidates is determined by selecting candidates according to a sampling of a search region according to the precision.
15 . The apparatus of claim 11 , wherein the set of template candidates is determined by selecting candidates according to a location and precision in both horizontal and vertical directions.
16 . An apparatus for video encoding, the apparatus comprising one or more processors configured to:
determine a template based on L-shaped samples surrounding a block of a picture of a video;
determine an initial location for an initial iteration;
select a matching template among a set of template candidates in reconstructed blocks of the picture, wherein the set of template candidates is determined based on a determined location and wherein the matching template is selected based on a similarity measure between samples of the template and samples of the template candidates;
refine the selection by iterating over the selection step while the similarity of a template candidate is better than the similarity of the other template candidates and, on a condition that the similarity of the template candidate at a current location is better than the similarity of the other template candidates of the set of template candidates, refine the selection by iterating over a further set of template candidates for which the location is determined with a finer precision;
reconstruct the block by copying samples of the selected matching template into the block, wherein, for the initial iteration over the selection step, the location is determined by the initial location and wherein, for further iterations, the location is determined as a refinement based on the location of the selected matching template of the previous iteration; and
encode picture data using the reconstructed block.
17 . A computer program comprising program code instructions for implementing the steps of the method according to claim 1 when executed by a processor.
18 . A non-transitory computer readable medium comprising program code instructions for implementing the steps of the method according to claim 1 when executed by a processor.