Clustering BVP candidates
A flipping or non-flipping type of each BVP candidate in a candidate list may require correct classification/identification, for example, for correct reordering and accurate refinement A block vector predictor (BVP) candidate list may be adjusted to provide a more accurate prediction of a block vector (BV). A coder (e.g., encoder or decoder) may determine a final candidate list, for example, by pruning invalid candidates.
1 . A method comprising:
determining, by a computing device and based on one or more indications of prediction modes for a plurality of block vector predictor (BVP) candidates, flipping types for the plurality of BVP candidates;
determining, for the plurality of BVP candidates, template matching costs between:
a template of a current block (CB) of content; and
templates, of reference blocks (RBs), based on the flipping types for the plurality of BVP candidates;
determining, based on the template matching costs, a list of BVP candidates;
receiving, in a bitstream, an index indicating a BVP candidate, from the list of BVP candidates, for decoding the CB;
determining, based on the BVP candidate indicated by the index, a block vector (BV) indicating a displacement from the CB to a reference block (RB); and
decoding the CB based on:
the BVP; and
combining the RB with a residual associated with the CB.
2 . The method of claim 1 , further comprising:
based on the template matching costs of BVP candidates, of the plurality of BVP candidates, in a first cluster, selecting a first BVP candidate from the first cluster;
based on the template matching costs of the BVP candidates, of the plurality of BVP candidates, in a second cluster, selecting a second BVP candidate from the second cluster; and
inserting, into the list of BVP candidates for a prediction mode, the first BVP candidate and the second BVP candidate.
3 . The method of claim 2 , wherein the inserting the first BVP candidate and the second BVP candidate further comprises at least one of:
inserting indications of the flipping types of the first BVP candidate and the second BVP candidate into the list of BVP candidates; and
inserting an indication of the prediction mode of the first BVP candidate and the second BVP candidate into the list of BVP candidates.
4 . The method of claim 2 , further comprising:
determining that the list of BVP candidates has a size less than a target number of BVP candidates; and
adding, based on the template matching costs, one or more BVP candidates from the plurality of BVP candidates, other than the selected first BVP candidate and the selected second BVP candidate, to the list of BVP candidates such that the size is equal to the target number of BVP candidates.
5 . The method of claim 1 , further comprising:
determining, based on the one or more indications of the prediction modes, a respective prediction mode for each of the plurality of BVP candidates,
wherein the respective prediction mode comprises at least one of:
intra block copy (IBC) merge mode,
IBC template matching (TM) merge mode,
IBC motion block vector difference (MBVD) merge code, or
advanced motion vector prediction (AMVP) for IBC mode, and
wherein the determining the respective prediction mode for each of the plurality of BVP candidates is further based on the flipping types of each of the plurality of BVP candidates.
6 . The method of claim 1 , wherein each template matching cost, of the template matching costs, for a corresponding BVP candidate of the plurality of BVP candidates is based on a difference between:
a template of the CB, and
a template, of a RB of the RBs, flipped according to a flipping type for the corresponding BVP candidate, and displaced from the CB by the corresponding BVP candidate, and
wherein the method further comprises:
reordering the plurality of BVP candidates based on the template matching costs.
7 . The method of claim 1 , further comprising:
selecting, based on flipping types for BVP candidates in the list of BVP candidates, a BVP candidate from the list of BVP candidates for prediction of the CB.
8 . The method of claim 1 , wherein the residual is received in the bitstream and indicates a difference between the CB and the RB.
9 . A method comprising:
determining, by a computing device and based on one or more indications of prediction modes for a plurality of block vector predictor (BVP) candidates, flipping types for the plurality of BVP candidates;
determining, for the plurality of BVP candidates, template matching costs between:
a template of a current block (CB) of content, and
templates, of reference blocks (RBs) of the content, based on the flipping types for the plurality of BVP candidates;
determining a first cluster comprising each BVP candidate, of the plurality of BVP candidates, with a distance from a first reference BVP candidate that is less than or equal to a threshold;
determining a second cluster comprising one or more BVP candidates, of the plurality of BVP candidates;
selecting, based on the template matching costs of the BVP candidates in the first cluster, a first BVP candidate from the first cluster;
selecting, based on the template matching costs of the BVP candidates in the second cluster, a second BVP candidate from the second cluster; and
inserting, into a list of BVP candidates for a prediction mode, the first BVP candidate and the second BVP candidate.
10 . The method of claim 9 , further comprising:
determining that the first reference BVP candidate has a lowest template matching cost of the template matching costs for the plurality of BVP candidates.
11 . The method of claim 9 , wherein:
the threshold is a radius value of a circle centered at an endpoint of the first reference BVP candidate; and
each BVP candidate of the first cluster has an endpoint inside the circle.
12 . The method of claim 9 , further comprising:
determining a third cluster comprising each BVP candidate, of one or more BVP candidates not within the first cluster and not within the second cluster, having a third distance from a third reference BVP candidate that is less than or equal to a third threshold, wherein the determining the third cluster further comprises determining that the third reference BVP candidate has a lowest template matching cost of one or more template matching costs of one or more BVP candidates within the third cluster.
13 . The method of claim 9 , further comprising:
determining that the list of BVP candidates has a size less than a target number of BVP candidates; and
adding, based on the template matching costs, one or more BVP candidates from the plurality of BVP candidates, other than the selected first BVP candidate and the selected second BVP candidate, to the list of BVP candidates such that the size is equal to the target number of BVP candidates.
14 . The method of claim 9 , further comprising
selecting, from the list of BVP candidates, a BVP candidate, for prediction of a current block (CB) of content, based on:
a prediction cost, the prediction mode, or flipping types for the BVP candidates within the list of BVP candidates; and
encoding, in a bitstream, an index to the BVP candidate selected for prediction and an indication of the prediction mode.
15 . The method of claim 9 , further comprising:
receiving, in a bitstream, an indication of the prediction mode and an index indicating a BVP candidate, from the list of BVP candidates, for decoding a current block (CB) of content;
determining, based on the BVP candidate, a block vector (BV) indicating a displacement from the CB to a reference block (RB); and
decoding the CB based on the BVP candidate indicated by the index and based on combining the RB with a residual received from the bitstream for the CB.
16 . A method comprising:
determining, by a computing device and for a plurality of block vector predictor (BVP) candidates, template matching costs between:
a template of a current block (CB) of content, and
templates, of reference blocks (RBs), based on flipping types for the plurality of BVP candidates;
selecting a first BVP candidate based on the template matching costs of the BVP candidates in a first cluster;
selecting a second BVP candidate based on the template matching costs of the BVP candidates in a second cluster;
inserting, into a list of BVP candidates for a prediction mode, the first BVP candidate and the second BVP candidate;
selecting, from the list of BVP candidates, a BVP candidate for prediction of the CB, wherein the selecting the BVP candidate is based on a prediction cost, the prediction mode, or flipping types for BVP candidates within the list of BVP candidates; and
encoding, in a bitstream, an index to the BVP candidate selected for prediction and an indication of the prediction mode.
17 . The method of claim 16 , further comprising:
determining, based on the indication of the prediction mode, the flipping types for the plurality of BVP candidates;
determining the first cluster comprising each BVP candidate, of the plurality of BVP candidates, with a distance from a first reference BVP candidate that is less than or equal to a threshold; and
determining the second cluster comprising one or more BVP candidates, of the plurality of BVP candidates, with a difference from the first reference BVP candidate that is greater than the threshold.
18 . The method of claim 16 , wherein the inserting, into the list of BVP candidates, further comprises:
inserting one or more indications of a flipping type of for the first BVP candidate and of a flipping type for the second BVP candidate into the list of BVP candidates; and
inserting an indication of the prediction mode of the first BVP candidate and the second BVP candidate into the list of BVP candidates.
19 . The method of claim 16 , wherein each template matching cost, of the template matching costs, for a corresponding BVP candidate, of the plurality of BVP candidates, is based on a difference between:
a template of the CB, and
a template of a RB, of the RBs, flipped according to a flipping type for the corresponding BVP candidate, and displaced from the CB by the corresponding BVP candidate, and
wherein the method further comprises:
reordering the plurality of BVP candidates based on the template matching costs.
20 . The method of claim 16 , wherein the list of BVP candidates comprises an intra block copy (IBC) merge list or an advanced motion vector prediction (AMVP) list.