Motion compensation using combined inter and intra prediction
View Patent ↗The present disclosure relates to a video coding method, a computing device, and a storage medium. The method includes: obtaining one or more reference blocks associated with a current block of a video picture and one or more previously coded pictures for predicting the current block; generating an inter prediction based on one or more motion vectors from the current block to the one or more reference blocks; generating a first intra prediction based on a plurality of neighboring reconstructed reference samples associated with a plurality of neighboring blocks of the current block in the video picture; determining a first weight based on the plurality of neighboring blocks; generating a third prediction by combining the inter prediction and the first intra prediction based on the first weight; and generating a final prediction based on the plurality of neighboring reconstructed samples and the third prediction.
1. A method of video coding, the method comprising:
obtaining one or more reference blocks associated with a current block of a video picture, wherein the one or more reference blocks are associated with one or more previously coded pictures for predicting the current block;
determining a first weight based on a plurality of neighboring blocks of the current block in a same video picture, wherein the first weight is a CHIP weight;
generating an inter prediction based on the first weight, a left neighboring weight, a top neighboring weight, and one or more motion vectors from the current block to the one or more reference blocks;
generating a position dependent intra prediction combination (PDPC)-based intra prediction based on a left neighboring reconstructed reference sample of a plurality of neighboring reconstructed reference samples, a top neighboring reconstructed reference sample of the plurality of neighboring reconstructed reference samples, the first weight, the left neighboring weight and the top neighboring weight, wherein the plurality of neighboring reconstructed reference samples are associated with the plurality of neighboring blocks of the current block in the same video picture; and
generating a final prediction based on the PDPC-based intra prediction, the inter prediction and the first weight.
2. The method of claim 1 , further comprising:
generating a first intra prediction based on the plurality of neighboring reconstructed reference samples associated with the plurality of neighboring blocks of the current block in the same video picture;
generating a third prediction by combining the inter prediction and the first intra prediction based on the first weight.
3. The method of claim 1 , further comprising:
receiving a flag indicating whether a weighted intra and inter prediction (WIIP) mode or a combined intra and inter prediction mode (CIIP) is enabled;
in response to determining that the flag indicates that the WIIP mode is enabled, setting the first weight to be zero; and
in response to determining that the flag indicates that the CIIP mode is enabled, setting the first weight to be non-zero.
4. The method of claim 1 , wherein
generating the PDPC-based intra prediction based on the left neighboring reconstructed reference sample of the plurality of neighboring reconstructed reference samples, the top neighboring reconstructed reference sample of the plurality of neighboring reconstructed reference samples, the first weight, the left neighboring weight and the top neighboring weight comprises generating the PDPC-based intra prediction using following equation:
predPdpc
(
x
,
y
)
=
(
WL
×
R
-
1
,
y
+
W
T
×
R
x
,
-
1
)
/
W
ciip
;
wherein (x, y) indicates a position of a sample in the current block, predPdpc(x, y) indicates the PDPC-based intra prediction associated with the sample at the position, W ciip indicates the first weight, WL indicates the left neighboring weight, WT indicates the top neighboring weight, R −1,y indicates the left neighboring reconstructed reference sample of the sample at the position, R x,−1 indicates the top neighboring reconstructed reference sample of the sample at the position;
generating the inter prediction based on the first weight, the left neighboring weight, the top neighboring weight, and the motion vector from the current block to the reference block comprises generating the inter prediction using following equation:
predInter
(
x
,
y
)
=
(
6
4
-
W
L
-
W
T
)
/
(
4
-
W
ciip
)
×
P
inter
(
x
,
y
)
;
wherein P inter (x, y) indicates the inter prediction associated with the sample at the position; and
generating the final prediction based on the PDPC-based intra prediction, the inter prediction and the first weight comprises generating the final prediction using following equation:
FinalPred
(
x
,
y
)
=
(
(
4
-
W
ciip
)
×
predInter
(
x
,
y
)
+
W
ciip
×
predPdpc
(
x
,
y
)
+
32
)
⪢
6
;
wherein FinalPred(x, y) indicates the final prediction associated with the sample at the position, predInter(x, y) indicates the inter prediction associated with the sample at the position, and >> indicates a bitwise right shift operator.
5. A computing device, comprising:
a storage medium; and
one or more processors coupled to the storage medium, wherein the one or more processors are configured to perform acts comprising:
obtaining one or more reference blocks associated with a current block of a video picture, wherein the one or more reference blocks are associated with one or more previously coded pictures for predicting the current block;
determining a first weight based on a plurality of neighboring blocks of the current block in a same video picture, wherein the first weight is a CHIP weight;
generating an inter prediction based on the first weight, a left neighboring weight, a top neighboring weight, and one or more motion vectors from the current block to the one or more reference blocks;
generating a position dependent intra prediction combination (PDPC)-based intra prediction based on a left neighboring reconstructed reference sample of a plurality of neighboring reconstructed reference samples, a top neighboring reconstructed reference sample of the plurality of neighboring reconstructed reference samples, the first weight, the left neighboring weight and the top neighboring weight, wherein the plurality of neighboring reconstructed reference samples are associated with the plurality of neighboring blocks of the current block in the same video picture; and
generating a final prediction based on the PDPC-based intra prediction, the inter prediction and the first weight.
6. The computing device of claim 5 , wherein the one or more processors are further configured to perform acts comprising:
generating a first intra prediction based on the plurality of neighboring reconstructed reference samples associated with the plurality of neighboring blocks of the current block in the same video picture;
generating a third prediction by combining the inter prediction and the first intra prediction based on the first weight.
7. A non-transitory computer readable storage medium storing a plurality of programs for execution by a computing device having one or more processors, wherein the plurality of programs, when executed by the one or more processors, cause the one or more processors to perform acts comprising:
obtaining one or more reference blocks associated with a current block of a video picture, wherein the one or more reference blocks are associated with one or more previously coded pictures for predicting the current block;
determining a first weight based on a plurality of neighboring blocks of the current block in a same video picture, wherein the first weight is a CIIP weight;
generating an inter prediction based on the first weight, a left neighboring weight, a top neighboring weight, and one or more motion vectors from the current block to the one or more reference blocks;
generating a position dependent intra prediction combination (PDPC)-based intra prediction based on a left neighboring reconstructed reference sample of a plurality of neighboring reconstructed reference samples, a top neighboring reconstructed reference sample of the plurality of neighboring reconstructed reference samples, the first weight, the left neighboring weight and the top neighboring weight, wherein the plurality of neighboring reconstructed reference samples are associated with the plurality of neighboring blocks of the current block in the same video picture; and
generating a final prediction based on the PDPC-based intra prediction, the inter prediction and the first weight.
8. The computing device of claim 5 , wherein
generating the PDPC-based intra prediction based on the left neighboring reconstructed reference sample of the plurality of neighboring reconstructed reference samples, the top neighboring reconstructed reference sample of the plurality of neighboring reconstructed reference samples, the first weight, the left neighboring weight and the top neighboring weight comprises generating the PDPC-based intra prediction using following equation:
predPdpc
(
x
,
y
)
=
(
WL
×
R
-
1
,
y
+
W
T
×
R
x
,
-
1
)
/
W
ciip
;
wherein (x, y) indicates a position of a sample in the current block, predPdpc(x, y) indicates the PDPC-based intra prediction associated with the sample at the position, W ciip indicates the first weight, WL indicates the left neighboring weight, WT indicates the top neighboring weight, R −1,y indicates the left neighboring reconstructed reference sample of the sample at the position, R x,−1 indicates the top neighboring reconstructed reference sample of the sample at the position;
generating the inter prediction based on the first weight, the left neighboring weight, the top neighboring weight and the motion vector from the current block to the reference block comprises generating the inter prediction using following equation:
predInter
(
x
,
y
)
=
(
6
4
-
W
L
-
W
T
)
/
(
4
-
W
ciip
)
×
P
inter
(
x
,
y
)
;
wherein P inter (x, y) indicates the inter prediction associated with the sample at the position; and
generating the final prediction based on the PDPC-based intra prediction, the inter prediction and the first weight comprises generating the final prediction using following equation:
FinalPred
(
x
,
y
)
=
(
(
4
-
W
ciip
)
×
predInter
(
x
,
y
)
+
W
ciip
×
predPdpc
(
x
,
y
)
+
32
)
⪢
6
;
wherein FinalPred(x, y) indicates the final prediction associated with the sample at the position, predInter(x, y) indicates the inter prediction associated with the sample at the position, and >> indicates a bitwise right shift operator.
9. The non-transitory computer readable storage medium of claim 7 , wherein the plurality of programs, when executed by the one or more processors, further cause the one or more processors to perform acts comprising:
generating a first intra prediction based on the plurality of neighboring reconstructed reference samples associated with the plurality of neighboring blocks of the current block in the same video picture;
generating a third prediction by combining the inter prediction and the first intra prediction based on the first weight.
10. The non-transitory computer readable storage medium of claim 7 , wherein generating the PDPC-based intra prediction based on the left neighboring reconstructed reference sample of the plurality of neighboring reconstructed reference samples, the top neighboring reconstructed reference sample of the plurality of neighboring reconstructed reference samples, the first weight, the left neighboring weight and the top neighboring weight comprises generating the PDPC-based intra prediction using following equation:
predPdpc
(
x
,
y
)
=
(
WL
×
R
-
1
,
y
+
W
T
×
R
x
,
-
1
)
/
W
ciip
;
wherein (x, y) indicates a position of a sample in the current block, predPdpc(x, y) indicates the PDPC-based intra prediction associated with the sample at the position, W ciip indicates the first weight, WL indicates the left neighboring weight, WT indicates the top neighboring weight, R −1,y indicates the left neighboring reconstructed reference sample of the sample at the position, R x,−1 indicates the top neighboring reconstructed reference sample of the sample at the position;
generating the inter prediction based on the first weight, the left neighboring weight, the top neighboring weight and the motion vector from the current block to the reference block comprises generating the inter prediction using following equation:
predInter
(
x
,
y
)
=
(
6
4
-
W
L
-
W
T
)
/
(
4
-
W
ciip
)
×
P
inter
(
x
,
y
)
;
wherein P inter (x, y) indicates the inter prediction associated with the sample at the position; and
generating the final prediction based on the PDPC-based intra prediction, the inter prediction and the first weight comprises generating the final prediction using following equation:
FinalPred
(
x
,
y
)
=
(
(
4
-
W
ciip
)
×
predInter
(
x
,
y
)
+
W
ciip
×
predPdpc
(
x
,
y
)
+
32
)
⪢
6
;
wherein FinalPred(x, y) indicates the final prediction associated with the sample at the position, predInter(x, y) indicates the inter prediction associated with the sample at the position, and >> indicates a bitwise right shift operator.