Constructing motion vector candidates list for video coding
Constructing a motion vector candidate list for inter-prediction in video coding is provided. A video encoder performs inter prediction for a frame of the video to generate prediction residuals. The inter prediction is performed using motion vectors (MVs) for blocks of the frame. To determine the motion vector of a block, the encoder determines an integer-valued MV for the block based on a reference frame and selects, based on the integer-valued MV, a subset of candidate MVs from a list of MVs including MVs of neighboring blocks. The encoder identifies a MV from the subset of candidate MVs for the block and searches in a neighborhood of the identified MV for a refined MV. The encoder calculates the prediction residuals for the block based on a reference block in the reference frame pointed by the refined MV and encodes the prediction residuals into a bitstream representing the video.
1. A method for encoding a video, the method comprising:
accessing a plurality of frames of the video;
performing inter prediction for a frame in the plurality of frames to generate prediction residuals for the frame, wherein performing the inter prediction comprises:
determining an integer-valued motion vector (MV) for a block of the frame based on a reference frame of the frame,
selecting, based on the integer-valued MV, a subset of candidate MVs from a list of MVs comprising MVs of a plurality of neighboring blocks of the block,
identifying a MV from the subset of candidate MVs for the block,
searching in a neighborhood of the identified MV for a refined MV for the block, and
calculating the prediction residuals for the block based on a reference block in the reference frame pointed by the refined MV; and
encoding the prediction residuals for the frame into a bitstream representing the video.
2. The method of claim 1 , wherein selecting the subset of candidate MVs from the list of MVs comprises:
calculating a distance of each MV in the list of MVs from the integer-valued MV; and
selecting the subset of MVs from the list of MVs, each MV in the subset of MVs has a distance smaller than a distance associated with an unselected MVs.
3. The method of claim 1 , wherein determining the integer-valued MV for a block comprises:
searching in the reference frame for a reference block at an integer pixel location that has a smallest distortion from the block; and
identifying an offset between the block and the reference block as the integer-valued MV.
4. The method of claim 1 , wherein performing the inter prediction further comprises:
prior to identifying a MV from the subset of candidate MVs for the block,
adding one or more additional MVs to the subset of candidate MVs, the one or more additional MVs comprising a MV from a previous coded block of the block; and
updating the subset of candidate MVs by removing duplicate MVs.
5. The method of claim 1 , wherein identifying the MV from the subset of candidate MVs for the block comprises:
determining a reference block in the reference frame for each MV in the subset of candidate MVs;
calculating distortions between the block and the respective reference blocks; and
selecting a MV from the subset of candidate MVs that corresponds to a smallest distortion among the distortions as the identified MV for the block.
6. The method of claim 1 , wherein searching in the neighborhood of the identified MV for a refined MV for the block comprises:
generating a group of refined MVs with sub-pixel coordinates in a neighborhood of the identified MV;
determining a reference block in the reference frame for each refined MV in the group of refined MVs;
calculating distortions between the block and the respective reference blocks; and
selecting a refined MV from the group of refined MVs that corresponds to a smallest distortion among the distortions as the refined MV for the block.
7. The method of claim 1 , wherein performing the inter prediction further comprises determining availabilities of neighboring blocks of the block, and wherein the plurality of neighboring blocks of the block are available neighboring blocks of the block.
8. The method of claim 7 , wherein determining the availabilities of neighboring blocks of the block comprises:
determining a category of a super block containing the block;
retrieving a set of neighboring availability masks based on the determined category;
retrieving neighboring availability flags associated the block from the set of neighboring availability masks; and
determining the availabilities of the neighboring blocks based on the retrieved neighboring availability flags.
9. The method of claim 8 , wherein the category is one of a first category where the super block is at a left border of the frame or a tile of the frame; a second category where the super block is at a top border of the frame or a tile of the frame; a third category where the super block is at a right border of the frame or a tile of the frame; a fourth category where the super block is at a bottom border of the frame or a tile of the frame; a fifth category where the super block is at a top-left corner of the frame or a tile of the frame; a sixth category where the super block is at a top-right corner of the frame or a tile of the frame; a seventh category where the super block is at a bottom-right corner of the frame or a tile of the frame; an eighth category where the super block is at a bottom-left corner of the frame or a tile of the frame; and a ninth category where the super block is at a center of the frame or a tile of the frame.
10. The method of claim 8 , wherein the neighboring blocks comprise one or more of a top-left neighboring block, a top neighboring block, a top-right neighboring block, a left neighboring block, or a bottom-left neighboring block.
11. The method of claim 8 , wherein retrieving neighboring availability flags associated the block from the set of neighboring availability masks is performed based on a partitioning scheme of a super block containing the block and a location of the block in the partitioning scheme.
12. A system comprising:
a processor; and
at least one memory device including instructions that are executable by the processor to cause the processor to:
access a plurality of frames of a video;
perform inter prediction for a frame in the plurality of frames to generate prediction residuals for the frame, wherein performing the inter prediction comprises:
determining an integer-valued motion vector (MV) for a block of the frame based on a reference frame of the frame,
selecting, based on the integer-valued MV, a subset of candidate MVs from a list of MVs comprising MVs of a plurality of neighboring blocks of the block,
identifying a MV from the subset of candidate MVs for the block,
searching in a neighborhood of the identified MV for a refined MV for the block, and
calculating the prediction residuals for the block based on a reference block in the reference frame pointed by the refined MV; and
encode the prediction residuals for the frame into a bitstream representing the video.
13. The system of claim 12 , wherein selecting the subset of candidate MVs from the list of MVs comprises:
calculating a distance of each MV in the list of MVs from the integer-valued MV; and
selecting the subset of MVs from the list of MVs, each MV in the subset of MVs has a distance smaller than a distance associated with an unselected MVs.
14. The system of claim 12 , wherein performing the inter prediction further comprises:
prior to identifying a MV from the subset of candidate MVs for the block,
adding one or more additional MVs to the subset of candidate MVs, the one or more additional MVs comprising a MV from a previous coded block of the block; and
updating the subset of candidate MVs by removing duplicate MVs.
15. The system of claim 12 , wherein performing the inter prediction further comprises determining availabilities of neighboring blocks of the block, and wherein the plurality of neighboring blocks of the block are available neighboring blocks of the block.
16. The system of claim 15 , wherein determining the availabilities of neighboring blocks of the block comprises:
determining a category of a super block containing the block;
retrieving a set of neighboring availability masks based on the determined category;
retrieving neighboring availability flags associated the block from the set of neighboring availability masks; and
determining the availabilities of the neighboring blocks based on the retrieved neighboring availability flags.
17. A non-transitory computer-readable medium comprising program code that is executable by one or more processors to cause the one or more processors to:
access a plurality of frames of a video;
perform inter prediction for a frame in the plurality of frames to generate prediction residuals for the frame, wherein performing the inter prediction comprises:
determining an integer-valued motion vector (MV) for a block of the frame based on a reference frame of the frame,
selecting, based on the integer-valued MV, a subset of candidate MVs from a list of MVs comprising MVs of a plurality of neighboring blocks of the block,
identifying a MV from the subset of candidate MVs for the block,
searching in a neighborhood of the identified MV for a refined MV for the block, and
calculating the prediction residuals for the block based on a reference block in the reference frame pointed by the refined MV; and
encode the prediction residuals for the frame into a bitstream representing the video.
18. The non-transitory computer-readable medium of claim 17 , wherein performing the inter prediction further comprises:
prior to identifying a MV from the subset of candidate MVs for the block,
adding one or more additional MVs to the subset of candidate MVs, the one or more additional MVs comprising a MV from a previous coded block of the block; and
updating the subset of candidate MVs by removing duplicate MVs.
19. The non-transitory computer-readable medium of claim 17 , wherein performing the inter prediction further comprises determining availabilities of neighboring blocks of the block, and wherein the plurality of neighboring blocks of the block are available neighboring blocks of the block and wherein determining the availabilities of neighboring blocks of the block comprises:
determining a category of a super block containing the block;
retrieving a set of neighboring availability masks based on the determined category;
retrieving neighboring availability flags associated the block from the set of neighboring availability masks; and
determining the availabilities of the neighboring blocks based on the retrieved neighboring availability flags.
20. The non-transitory computer-readable medium of claim 19 , wherein the category is one of a first category where the super block is at a left border of the frame or a tile of the frame; a second category where the super block is at a top border of the frame or a tile of the frame; a third category where the super block is at a right border of the frame or a tile of the frame; a fourth category where the super block is at a bottom border of the frame or a tile of the frame; a fifth category where the super block is at a top-left corner of the frame or a tile of the frame; a sixth category where the super block is at a top-right corner of the frame or a tile of the frame; a seventh category where the super block is at a bottom-right corner of the frame or a tile of the frame; an eighth category where the super block is at a bottom-left corner of the frame or a tile of the frame; and a ninth category where the super block is at a center of the frame or a tile of the frame.