Template matching based affine prediction for video coding

1. A method of decoding video data, the method comprising:

determining that a current block in a current picture of the video data is coded in an affine prediction mode, wherein the current block comprises a coding unit including a first plurality of sub-blocks;

determining one or more control-point motion vectors (CPMVs) for the current block;

determining for each of the first plurality of sub-blocks, a respective motion vector based on the CPMVs;

identifying an initial prediction block for the current block in a reference picture using the one or more CPMVs, wherein the initial prediction block comprises a plurality of prediction sub-blocks, and wherein identifying the initial prediction block comprises identifying, for each sub-block of the first plurality of sub-blocks, a corresponding prediction sub-block of the plurality of prediction sub-blocks based on the respective motion vector of the sub-block;

determining a current template for the current block in the current picture, wherein the current template corresponds to a second plurality of sub-blocks that are neighboring blocks of the first plurality of sub-blocks;

determining an initial reference template for the initial prediction block in the reference picture, wherein the initial reference template corresponds to a third plurality of sub-blocks that are neighboring blocks of the corresponding prediction sub-blocks; and

performing a motion vector refinement process to determine a modified prediction block based on a comparison of the current template to the initial reference template.

2. The method of claim 1 , wherein performing the motion vector refinement process to determine the modified prediction block further comprises:

searching within a search area around the initial reference template for a subsequent reference template that matches the current template more closely than the initial reference template.

3. The method of claim 1 , wherein the comparison of the current template to the initial reference template comprises a template matching cost.

4. The method of claim 3 , further comprising:

determining the template matching cost based on a weighted per-sample comparison of samples in the current template to samples in the initial reference template.

5. The method of claim 1 , wherein the initial reference template comprises a plurality of sub-blocks located above the initial prediction block or to the left of the initial prediction block.

6. The method of claim 1 , wherein the affine prediction mode comprises a 4-parameter affine prediction mode.

7. The method of claim 1 , wherein the affine prediction mode comprises a 6-parameter affine prediction mode.

8. The method of claim 1 , further comprising:

determining a prediction block based on the modified prediction block;

adding the prediction block to a residual block to determine a reconstructed block;

applying one or more filtering operations to the reconstructed block; and

outputting a picture of decoded video data that includes the filtered reconstructed block.

9. The method of claim 1 , wherein the method of decoding is performed as part of a video encoding process.

10. A device for decoding video data, the device comprising:

a memory; and

one or more processors implemented in circuitry, coupled to the memory, and configured to:

determine that a current block in a current picture of the video data is coded in an affine prediction mode, wherein the current block comprises a coding unit including a first plurality of sub-blocks;

determine one or more control-point motion vectors (CPMVs) for the current block;

determine for each of the first plurality of sub-blocks, a respective motion vector based on the CPMVs;

identify an initial prediction block for the current block in a reference picture using the one or more CPMVs, wherein the initial prediction block comprises a plurality of prediction sub-blocks, and wherein identifying the initial prediction block comprises identifying, for each sub-block of the first plurality of sub-blocks, a corresponding prediction sub-block of the plurality of prediction sub-blocks based on the respective motion vector of the sub-block;

determine a current template for the current block in the current picture, wherein the current template corresponds to a second plurality of sub-blocks that are neighboring blocks of the first plurality of sub-blocks;

determine an initial reference template for the initial prediction block in the reference picture, wherein the initial reference template corresponds to a third plurality of sub-blocks that are neighboring blocks of the corresponding prediction sub-blocks; and

perform a motion vector refinement process to determine a modified prediction block based on a comparison of the current template to the initial reference template.

11. The device of claim 10 , wherein to perform the motion vector refinement process to determine the modified prediction block further, the one or more processors are further configured to:

search within a search area around the initial reference template for a subsequent reference template that matches the current template more closely than the initial reference template.

12. The device of claim 10 , wherein the comparison of the current template to the initial reference template comprises a template matching cost.

13. The device of claim 12 , wherein the one or more processors are further configured to:

determine the template matching cost based on a weighted per-sample comparison of samples in the current template to samples in the initial reference template.

14. The device of claim 10 , wherein the initial reference template comprises a plurality of sub-blocks located above the initial prediction block or to the left of the initial prediction block.

15. The device of claim 10 , wherein the affine prediction mode comprises a 4-parameter affine prediction mode.

16. The device of claim 10 , wherein the affine prediction mode comprises a 6-parameter affine prediction mode.

17. The device of claim 10 , wherein the one or more processors are further configured to:

determine a prediction block based on the modified prediction block;

add the prediction block to a residual block to determine a reconstructed block;

apply one or more filtering operations to the reconstructed block; and

output a picture of decoded video data that includes the filtered reconstructed block.

18. The device of claim 10 , wherein the device comprises a wireless communication device, further comprising a receiver configured to receive encoded video data.

19. The device of claim 18 , wherein the wireless communication device comprises a telephone handset and wherein the receiver is configured to demodulate, according to a wireless communication standard, a signal comprising the encoded video data.

20. The device of claim 10 , further comprising:

a display configured to display decoded video data.

21. The device of claim 10 , wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

22. The device of claim 10 , wherein the device comprises a video encoding devices.

23. A non-transitory computer-readable storage medium storing instructions that when executed by one or more processors cause the one or more processors to:

determine that a current block in a current picture of the video data is coded in an affine prediction mode, wherein the current block comprises a coding unit including a first plurality of sub-blocks;

determine one or more control-point motion vectors (CPMVs) for the current block;

determine for each of the first plurality of sub-blocks, a respective motion vector based on the CPMVs;

identify an initial prediction block for the current block in a reference picture using the one or more CPMVs, wherein the initial prediction block comprises a plurality of prediction sub-blocks, and wherein identifying the initial prediction block comprises identifying, for each sub-block of the first plurality of sub-blocks, a corresponding prediction sub-block of the plurality of prediction sub-blocks based on the respective motion vector of the sub-block;

determine a current template for the current block in the current picture, wherein the current template corresponds to a second plurality of sub-blocks that are neighboring blocks of the first plurality of sub-blocks;

determine an initial reference template for the initial prediction block in the reference picture, wherein the initial reference template corresponds to a third plurality of sub-blocks that are neighboring blocks of the corresponding prediction sub-blocks; and

perform a motion vector refinement process to determine a modified prediction block based on a comparison of the current template to the initial reference template.

24. The non-transitory computer-readable storage medium of claim 23 , wherein to perform the motion vector refinement process to determine the modified prediction block further, the instructions cause the one or more processors to:

search within a search area around the initial reference template for a subsequent reference template that matches the current template more closely than the initial reference template.

25. The non-transitory computer-readable storage medium of claim 23 , wherein the comparison of the current template to the initial reference template comprises a template matching cost.

26. The non-transitory computer-readable storage medium of claim 25 , wherein the instructions cause the one or more processors to:

determine the template matching cost based on a weighted per-sample comparison of samples in the current template to samples in the initial reference template.

27. The non-transitory computer-readable storage medium of claim 23 , wherein the initial reference template comprises a plurality of sub-blocks located above the initial prediction block or to the left of the initial prediction block.

28. The non-transitory computer-readable storage medium of claim 23 , wherein the instructions cause the one or more processors to:

determine a prediction block based on the modified prediction block;

add the prediction block to a residual block to determine a reconstructed block;

apply one or more filtering operations to the reconstructed block; and

output a picture of decoded video data that includes the filtered reconstructed block.

29. An apparatus for decoding video data, the apparatus comprising:

means for determining that a current block in a current picture of the video data is coded in an affine prediction mode, wherein the current block comprises a coding unit including a first plurality of sub-blocks;

means for determining one or more control-point motion vectors (CPMVs) for the current block;

means for determining for each of the first plurality of sub-blocks, a respective motion vector based on the CPMVs,

means for identifying an initial prediction block for the current block in a reference picture using the one or more CPMVs, wherein the initial prediction block comprises a plurality of prediction sub-blocks, and wherein identifying the initial prediction block comprises identifying, for each sub-block of the first plurality of sub-blocks, a corresponding prediction sub-block of the plurality of prediction sub-blocks based on the respective motion vector of the sub-block;

means for determining a current template for the current block in the current picture, wherein the current template corresponds to a second plurality of sub-blocks that are neighboring blocks of the first plurality of sub-blocks;

means for determining an initial reference template for the initial prediction block in the reference picture, wherein the initial reference template corresponds to a third plurality of sub-blocks that are neighboring blocks of the corresponding prediction sub-blocks; and

means for performing a motion vector refinement process to determine a modified prediction block based on a comparison of the current template to the initial reference template.

30. The apparatus of claim 29 , wherein the comparison of the current template to the initial reference template comprises a template matching cost, the apparatus further comprising:

a means for determining the template matching cost based on a weighted per-sample comparison of samples in the current template to samples in the initial reference template.