Methods and systems to estimate motion based on reconstructed reference frames at a video decoder
View Patent ↗Methods and systems to apply motion estimation (ME) based on reconstructed reference pictures in a B frame or in a P frame at a video decoder. For a P frame, projective ME may be performed to obtain a motion vector (MV) for a current input block. In a B frame, both projective ME and mirror ME may be performed to obtain an MV for the current input block. The ME process can be performed on sub-partitions of the input block, which may reduce the prediction error without increasing the amount of MV information in the bitstream. Decoder-side ME can be applied for the prediction of existing inter frame coding modes, and traditional ME or the decoder-side ME can be adaptively selected to predict a coding mode based on a rate distribution optimization (RDO) criterion.
1. A method, comprising:
specifying, at a video decoder, a search window in a first forward reference frame;
specifying a search path in the search window of the first forward reference frame;
for each motion vector MV 0 in the search path, where each MV 0 points from a current block to a reference block in the search window, determining a corresponding second motion vector MV 1 that points to a reference block in a second reference frame, where the corresponding second motion vector MV 1 is a function of MV 0 ;
calculating a metric for each pair of MV 0 and MV 1 that is found in the search path wherein the metric consists of a sum of absolute differences (SAD) between the reference block in the search window and the reference block in the second reference frame;
selecting the MV 0 whose corresponding value for the SAD is closest to zero, where the selected MV 0 is used as a motion vector for the current block;
determining a predicted motion for the current block, where the predicted motion is one of
an average, and
a weighted average,
of predicted motion for MV 0 and MV 1 .
2. The method of claim 1 , wherein the second reference frame comprises a backward reference frame, and each MV 1 is a mirror motion vector.
3. The method of claim 2 , wherein the current block is located in a bi-predictive frame.
4. The method of claim 1 , wherein the second reference frame comprises a second forward reference frame, and each MV 1 is a projective motion vector.
5. The method of claim 4 , wherein the current block is located in a predictive frame.
6. The method of claim 1 , wherein for each MV 0 , the corresponding MV 1 is determined by a function
MV
1
=
d
1
d
0
MV
0
where d 0 is a temporal distance between the current block and the first forward reference frame, and d 1 is a temporal distance between the current block and the second reference frame.
7. The method of claim 1 , wherein the current block is a sub-block of a larger partitioned block.
8. A non-transitory computer program product including a computer readable medium having computer program logic stored therein, the computer program logic including:
motion estimation logic to
cause a processor to determine a motion vector MV 1 for each motion vector MV 0 that is found in a search path in a search window of a first forward reference frame, where each MV 0 points from a current block to a reference block in the search window, and where the MV 1 is a function of the MV 0 ;
further cause the processor to calculate a metric for each pair of MV 0 and MV 1 wherein the metric consists of a sum of absolute differences (SAD) between the reference block in the search window and the reference block in the second reference frame; and
further cause the processor to select the MV 0 whose corresponding value for the metric is closest to zero, where the selected MV 0 is used as a motion vector for the current block; and
further cause the processor to determine a predicted motion for the current block, where the predicted motion is one of
an average, and
a weighted average,
of predicted motion for MV 0 and MV 1 .
9. The computer program product of claim 8 , wherein said motion estimation logic comprises mirror motion estimation logic, and
wherein the second reference frame comprises a backward reference frame, each MV 1 is a mirror motion vector, and the current block is located in a bi-predictive frame.
10. The computer program product of claim 8 , wherein said motion estimation logic comprises projective motion estimation logic, and
wherein the second reference frame comprises a second forward reference frame, each MV 1 is a projective motion vector, and the current block is located in a predictive frame.
11. The computer program product of claim 8 , wherein for each MV 0 , the corresponding MV 1 is determined by a function
MV
1
=
d
1
d
0
MV
0
where d 0 is a temporal distance between the current block and the first forward reference frame, and d 1 is a temporal distance between the current block and the second reference frame.
12. A system, comprising:
a processor; and
a memory in communication with said processor, said memory for storing a plurality of processing instructions for directing said processor to
specify, at a video decoder, a search window in a first forward reference frame;
specify a search path in a search window of the first forward reference frame;
for each motion vector MV 0 in the search path, where each MV 0 points from a current block to a reference block in the search window, determine a corresponding second motion vector MV 1 that points to a reference block in a second reference frame, where the corresponding second motion vector MV 1 is a function of MV 0 ;
calculate a metric for each pair of MV 0 and MV 1 that is found in the search path wherein said metric consists of a sum of absolute differences (SAD) between the reference block in the search window and the reference block in the second reference frame;
select the MV 0 whose corresponding value for the SAD is closest to zero, where the selected MV 0 is used as a motion vector for the current block;
determine a predicted motion for the current block, where the predicted motion is one of
an average, and
a weighted average,
of predicted motion for MV 0 and MV 1 .
13. The system of claim 12 , wherein the second reference frame comprises a backward reference frame, each MV 1 is a minor motion vector, and the current block is located in a bi-predictive frame.
14. The system of claim 12 , wherein the second reference frame comprises a second forward reference frame, each MV 1 is a projective motion vector, and the current block is located in a predictive frame.