Low-latency and high-throughput motion vector refinement with template matching

1. A video decoder, comprising:

memory; and

a processor coupled to the memory and configured to:

convert a bitstream into inter-prediction parameters;

reconstruct motion data based on the inter-prediction parameters;

determine if a size of a current prediction unit meets a threshold;

if the size of the current prediction unit meets the threshold:

refine the motion data based on finding a match between a current template of a current picture and a reference template of a reference picture;

perform a motion compensation operation with the refined motion data and a reference block to generate an inter-prediction block; and

add the inter-prediction block to an inter-residual block to produce a reconstructed block,

wherein the motion data is reconstructed without refined motion data associated with a previous prediction unit;

if the size of the current prediction unit does not meet the threshold:

disable refining the motion data; and

perform the motion compensation operation with the reconstructed motion data and the reference block to generate the inter-prediction block.

2. The video decoder of claim 1 , wherein the processor is further configured to:

fetch a first set of reference blocks from the memory based on the reconstructed motion data;

determine a second set of reference blocks to be accessed in the motion compensation operation with the refined motion data includes at least one reference block not in the first set of reference blocks; and

use padding for the at least one reference block in the motion compensation operation.

3. The video decoder of claim 1 , wherein the current template of the current picture comprises reconstructed samples from neighboring prediction units, and

wherein the processor is further configured to:

exclude, from the current template reconstructed samples, neighboring prediction units that are intra-coded.

4. The video decoder of claim 3 , wherein the processor is further configured to:

determine all of the samples in the current template are from neighboring prediction units that are intra-coded;

disable refining the motion data; and

perform the motion compensation operation with the reconstructed motion data and the reference block to generate the inter-prediction block.

5. The video decoder of claim 1 , wherein the processor is further configured to:

pass inter-residual blocks and intra-residual blocks to an inter-prediction stage; and

pass the intra-residual blocks from the inter-prediction stage to an intra-prediction stage.

6. The video decoder of claim 5 , wherein the inter-prediction block is added to the inter-residual block to produce the reconstructed block in the inter-prediction stage.

7. The video decoder of claim 1 , wherein the current template of the current picture comprises inter-prediction samples from neighboring prediction units, and

wherein the processor is further configured to exclude from the current template samples from neighboring prediction units that are intra-coded.

8. The video decoder of claim 7 , wherein the inter-prediction samples from neighboring prediction units are generated using the reconstructed motion data.

9. The video decoder of claim 7 , wherein the processor is further configured to:

determine all of the samples in the current template are from neighboring prediction units that are intra-coded;

disable refining the motion data; and

perform the motion compensation operation with the reconstructed motion data and the reference block to generate the inter-prediction block.

10. A method, comprising:

converting a bitstream into inter-prediction parameters;

reconstructing motion data based on the inter-prediction parameters;

determining if a size of a current prediction unit meets a threshold;

if the size of the current prediction unit meets the threshold:

refining the motion data based on finding a match between a current template of a current picture and a reference template of a reference picture;

performing a motion compensation operation with the refined motion data and a reference block to generate an inter-prediction block; and

adding the inter-prediction block to an inter-residual block to produce a reconstructed block,

wherein the motion data is reconstructed without refined motion data associated with a previous prediction unit;

if the size of the current prediction unit does not meet the threshold:

disabling refining the motion data; and

performing the motion compensation operation with the reconstructed motion data and the reference block to generate the inter-prediction block.

11. The method of claim 10 , further comprising:

fetching a first set of reference blocks from the memory based on the reconstructed motion data;

determining a second set of reference blocks to be accessed in the motion compensation operation with the refined motion data includes at least one reference block not in the first set of reference blocks; and

using padding for the at least one reference block in the motion compensation operation.

12. The method of claim 10 , wherein the current template of the current picture comprises reconstructed samples from neighboring prediction units, and

wherein the method further comprises:

excluding from the current template reconstructed samples, neighboring prediction units that are intra-coded.

13. The method of claim 12 , further comprising:

determining all of the samples in the current template are from neighboring prediction units that are intra-coded;

disabling refining the motion data; and

performing the motion compensation operation with the reconstructed motion data and the reference block to generate the inter-prediction block.

14. The method of claim 10 , further comprising:

passing inter-residual blocks and intra-residual blocks to an inter-prediction stage; and

passing the intra-residual blocks from the inter-prediction stage to an intra-prediction stage.

15. The method of claim 14 , wherein the inter-prediction block is added to the inter-residual block to produce the reconstructed block in the inter-prediction stage.

16. The method of claim 10 , wherein the current template of the current picture comprises inter-prediction samples from neighboring prediction units, and

wherein the method further comprises excluding from the current template samples from neighboring prediction units that are intra-coded.

17. The method of claim 16 , wherein the inter-prediction samples from neighboring prediction units are generated using the reconstructed motion data.

18. The method of claim 16 , further comprising:

determining all of the samples in the current template are from neighboring prediction units that are intra-coded;

disabling refining the motion data; and

performing the motion compensation operation with the reconstructed motion data and the reference block to generate the inter-prediction block.