Inter-prediction mode based image processing method, and apparatus therefor
In the present invention, an inter-prediction mode based image processing method and an apparatus therefor are disclosed. Specifically, the inter-prediction based image processing method may comprise the steps of: deriving movement information of a control point for specifying a reference block of a current block; dividing the current block into sub-blocks; deriving movement information of the sub-blocks using the movement information of the control point; and generating a prediction block for each sub-block using the movement information of the sub-blocks.
1. A method of decoding an image based on inter prediction, the method comprising:
deriving, by a processor, at least two motion vectors of at least two control points of a current block;
splitting, by the processor, the current block into sub-blocks;
deriving, by the processor, a motion vector of each of at least a plurality of the sub-blocks based on the at least two motion vectors of the at least two control points of the current block;
generating, by the processor, a prediction block for each of the plurality of sub-blocks based on the motion vector of each of the plurality of sub-blocks; and
updating, by the processor, the prediction block for each of the plurality of sub-blocks pixel-by-pixel based on an optical flow,
wherein the at least two motion vectors of the at least two control points of the current block are derived based on a motion vector of a control point of a neighboring block, and
wherein a motion vector of one among the at least two control points of the current block is derived based on modifying of the motion vector of the control point of the neighboring block based on a difference between a position of an upper left sample of the neighboring block and a position of an upper left sample of the current block, and a difference between two motion vectors of two control points of the neighboring block with respect to at least one of a horizontal direction or a vertical direction.
2. The method of claim 1 , wherein the motion vector of the one among the at least two control points of the current block is derived based on modifying of the motion vector of the control point of the neighboring block or the reference block based on a difference between a size of the current block and, a size of the neighboring block or the reference block.
3. The method of claim 1 , wherein a size of the sub-block and/or a split type of the sub-block are determined based on the at least two motion vectors of the at least two control points of the current block.
4. The method of claim 3 , wherein:
a width of the sub-block is determined based on a difference between a motion vector of an upper left control point of the current block and a motion vector of an upper right control point of the current block, and
a height of the sub-block is determined based on a difference between a motion vector of an upper left control point of the current block and a motion vector of a lower left control point of the current block.
5. The method of claim 1 , wherein a motion vector of a specific reference point in the sub-block is used as the motion vector of the sub-block.
6. The method of claim 5 , wherein in case of a sub-block comprising the control point or adjacent to the control point, a motion vector of the control point is used as the motion vector of the sub-block.
7. The method of claim 5 , wherein a motion vector of a plurality of reference points in the sub-block is used as the motion vector of the sub-block.
8. The method of claim 1 , wherein:
the prediction block is generated based on a motion vector of a single reference point, and
the generated prediction block is updated pixel-by-pixel based on the optical flow to a plurality of prediction blocks derived based on a motion vector of a plurality of reference points.
9. The method of claim 1 , wherein the optical flow is applied only for sub-blocks that do not include the control point or are not adjacent to the control point.
10. The method of claim 1 , wherein the neighboring block comprises a block adjacent to a bottom-left corner of the current block.
11. A method of encoding an image based on inter prediction, comprising:
deriving, by a processor, at least two motion vectors of at least two control points of a current block;
splitting, by the processor, the current block into sub-blocks;
deriving, by the processor, a motion vector of each of the plurality of the sub-blocks based on the at least two motion vectors of the at least two control points of the current block;
generating, by the processor, a prediction block for each of the plurality of sub-blocks based on the motion vector of each of the plurality of sub-blocks; and
updating, by the processor, the prediction block for each of the plurality of sub-blocks pixel-by-pixel based on an optical flow,
wherein the at least two motion vectors of the at least two control points of the current block are derived based on a motion vector of a control point of a neighboring block, and
wherein a motion vector of one among the at least two control points of the current block is derived based on modifying of the motion vector of the control point of the neighboring block based on a difference between a position of an upper left sample of the neighboring block and a position of an upper left sample of the current block, and a difference between two motion vectors of two control points of the neighboring block with respect to at least one of a horizontal direction or a vertical direction.
12. A non-transitory computer-readable recording medium storing instructions that, when executed, cause one or more processors to perform operations comprising:
deriving at least two motion vectors of at least two control points of a current block;
splitting the current block into sub-blocks;
deriving a motion vector of each of the plurality of sub-blocks based on the at least two motion vectors of the at least two control points of the current block;
generating a prediction block for each of the plurality of sub-blocks based on the motion vector of each of the plurality of sub-blocks; and
updating the prediction block for each of the plurality of sub-blocks pixel-by-pixel based on an optical flow,
wherein the at least two motion vectors of the at least two control points of the current block are derived based on a motion vector of a control point of a neighboring block, and
wherein a motion vector of one among the at least two control points of the current block is derived based on modifying of the motion vector of the control point of the neighboring block based on a difference between a position of an upper left sample of the neighboring block and a position of an upper left sample of the current block, and a difference between two motion vectors of two control points of the neighboring block with respect to at least one of a horizontal direction or a vertical direction.
13. The non-transitory computer-readable recording medium of claim 12 , wherein the motion vector of the one among the at least two control points of the current block is derived based on modifying of the motion vector of the control point of the neighboring block or the reference block based on a difference between a size of the current block and, a size of the neighboring block or the reference block.
14. The non-transitory computer-readable recording medium of claim 12 , wherein a size of the sub-block and/or a split type of the sub-block are determined based on the at least two motion vectors of the at least two control points of the current block.
15. The non-transitory computer-readable recording medium of claim 14 , wherein:
a width of the sub-block is determined based on a difference between a motion vector of an upper left control point of the current block and a motion vector of an upper right control point of the current block, and
a height of the sub-block is determined based on a difference between a motion vector of an upper left control point of the current block and a motion vector of a lower left control point of the current block.
16. The non-transitory computer-readable recording medium of claim 12 , wherein a motion vector of a specific reference point in the sub-block is used as the motion vector of the sub-block.
17. The non-transitory computer-readable recording medium of claim 16 , wherein in case of a sub-block comprising the control point or adjacent to the control point, a motion vector of the control point is used as the motion vector of the sub-block.
18. The non-transitory computer-readable recording medium of claim 16 , wherein a motion vector of a plurality of reference points in the sub-block is used as the motion vector of the sub-block.
19. The non-transitory computer-readable recording medium of claim 12 , wherein:
the prediction block is generated based on a motion vector of a single reference point, and
the generated prediction block is updated pixel-by-pixel based on the optical flow to a plurality of prediction blocks derived based on a motion vector of a plurality of reference points.
20. The non-transitory computer-readable recording medium of claim 12 , wherein the optical flow is applied only for sub-blocks that do not include the control point or are not adjacent to the control point.