Interpolation filter clipping for sub-picture motion vectors
A video coding mechanism is disclosed. The mechanism includes receiving a bitstream comprising a current picture including a sub-picture coded according to inter-prediction. A motion vector for a block of the sub-picture is determined. A clipping function is applied to sample locations in a reference block to support application of an interpolation filter when the motion vector points outside of the sub-picture and when a flag is set to indicate the sub-picture is treated as a picture. The interpolation filter is applied to results of the clipping function to obtain a predicted sample value. The block is decoded based on the predicted sample value. The block is forwarded for display as part of a decoded video sequence.
1. A method implemented by a decoder, the method comprising:
receiving a current picture including a sub-picture with a block coded according to inter-prediction;
determining a motion vector for the block;
applying a clipping function to sample locations in a reference block during application of an interpolation process when a flag is set to indicate the sub-picture is treated as a picture;
applying the interpolation process to results of the clipping function to obtain a predicted sample value; and
decoding the block based on the predicted sample value,
wherein the interpolation process includes a luma sample bilinear interpolation process,
wherein the block includes a block of luma samples,
wherein the predicted sample value includes a predicted luma sample value,
wherein the luma sample bilinear interpolation process receives inputs including a luma location in full sample units (xInt L , yInt L ), wherein the luma sample bilinear interpolation process outputs the predicted luma sample value (predSampleLX L ), and wherein the clipping function is applied to the sample locations according to:
when subpic_treated_as_pic_flag[SubPicIdx] is equal to one, the following applies:
xInti =Clip3(SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xInt L +i ), and
yInti =Clip3(SubPicTopBoundaryPos, SubPicBotBoundaryPos, yInt L +i ),
where subpic_treated_as_pic_flag is the flag set to indicate the sub-picture is treated as a picture, SubPicIdx is an index of the sub-picture, xInti and yInti are a sample location at index i, SubPicRightBoundaryPos is a position of a right boundary of the sub-picture, SubPicLeftBoundaryPos is a position of a left boundary of the sub-picture, SubPicTopBoundaryPos is a position of a top boundary of the sub-picture,
SubPicBotBoundaryPos is a position of a bottom boundary of the sub-picture, and Clip3 is the clipping function according to:
Clip
3
(
x
,
y
,
z
)
=
{
x
;
z
<
y
y
;
z
>
y
z
;
otherwise
where x, y, and z are numerical input values.
2. The method of claim 1 , wherein the interpolation process includes a luma sample interpolation filtering process.
3. The method of claim 2 , wherein the luma sample interpolation filtering process receives inputs including a luma location in full sample units (xIntL, yIntL), wherein the luma sample interpolation filtering process outputs the predicted luma sample value (predSampleLX L ), and wherein the clipping function is applied to the sample locations according to:
when subpic_treated_as_pic_flag[SubPicIdx] is equal to one, the following applies:
xInti =Clip3(SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xInt L +i −3), and
yInti =Clip3(SubPicTopBoundaryPos, SubPicBotBoundaryPos, yInt L +i −3),
where subpic_treated_as_pic_flag is the flag set to indicate the sub-picture is treated as a picture, SubPicIdx is an index of the sub-picture, xInti and yInti are a sample location at index i, SubPicRightBoundaryPos is a position of a right boundary of the sub-picture, SubPicLeftBoundaryPos is a position of a left boundary of the sub-picture, SubPicTopBoundaryPos is a position of a top boundary of the sub-picture,
SubPicBotBoundaryPos is a position of a bottom boundary of the sub-picture, and Clip3 is the clipping function according to:
Clip
3
(
x
,
y
,
z
)
=
{
x
;
z
<
y
y
;
z
>
y
z
;
otherwise
where x, y, and z are numerical input values.
4. The method of claim 1 , wherein the interpolation process includes a chroma sample interpolation process, wherein the block includes a block of chroma samples, and wherein the predicted sample value includes a predicted chroma sample value.
5. The method of claim 4 , wherein the chroma sample interpolation process receives inputs including a chroma location in full sample units (xInt C , yInt C ), wherein the chroma sample interpolation process outputs the predicted chroma sample value (predSampleLX C ), and wherein the clipping function is applied to the sample locations according to:
when subpic_treated_as_pic_flag[SubPicIdx] is equal to one, the following applies:
xInti =Clip3(SubPicLeftBoundaryPos/SubWidthC, SubPicRightBoundaryPos/SubWidthC, xInt C +i ), and
yInti =Clip3(SubPicTopBoundaryPos/SubHeightC, SubPicBotBoundaryPos/SubHeightC, yInt C +i ),
where subpic_treated_as_pic_flag is the flag set to indicate the sub-picture is treated as a picture, SubPicIdx is an index of the sub-picture, xInti and yInti are a sample location at index i, SubPicRightBoundaryPos is a position of a right boundary of the sub-picture, SubPicLeftBoundaryPos is a position of a left boundary of the sub-picture, SubPicTopBoundaryPos is a position of a top boundary of the sub-picture,
SubPicBotBoundaryPos is a position of a bottom boundary of the sub-picture, SubWidthC and SubHeightC indicate a horizontal and vertical sampling rate ratio between luma and chroma samples, and Clip3 is the clipping function according to:
Clip
3
(
x
,
y
,
z
)
=
{
x
;
z
<
y
y
;
z
>
y
z
;
otherwise
where x, y, and z are numerical input values.
6. A method implemented in an encoder, the method comprising:
partitioning a current picture into a sub-picture and the sub-picture into a block;
determining to encode the block according to inter-prediction;
selecting a motion vector to encode the block;
applying a clipping function to sample locations in a reference block to support application of an interpolation process when the motion vector points outside of the sub-picture and when a flag is set to indicate the sub-picture is treated as a picture;
applying the interpolation process to results of the clipping function to obtain a predicted sample value;
encoding the block into a bitstream based on the predicted sample value and the motion vector; and
storing the bitstream for communication toward a decoder,
wherein the interpolation process includes a luma sample bilinear interpolation process,
wherein the block includes a block of luma samples,
wherein the predicted sample value includes a predicted luma sample value, wherein the luma sample bilinear interpolation process receives inputs including a luma location in full sample units (xInt L , yInt L ), wherein the luma sample bilinear interpolation process outputs the predicted luma sample value (predSampleLX L ), and wherein the clipping function is applied to the sample locations according to:
when subpic_treated_ as_pic_flag[SubPicIdx] is equal to one, the following applies:
xInti =Clip3(SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xInt L +i ), and
yInti =Clip3(SubPicTopBoundaryPos, SubPicBotBoundaryPos, yInt L +i ),
where subpic_treated_as_pic_flag is the flag set to indicate the sub-picture is treated as a picture, SubPicIdx is an index of the sub-picture, xInti and yInti are a sample location at index i, SubPicRightBoundaryPos is a position of a right boundary of the sub-picture, SubPicLeftBoundaryPos is a position of a left boundary of the sub-picture, SubPicTopBoundaryPos is a position of a top boundary of the sub-picture,
SubPicBotBoundaryPos is a position of a bottom boundary of the sub-picture, and Clip3 is the clipping function according to:
Clip
3
(
x
,
y
,
z
)
=
{
x
;
z
<
y
y
;
z
>
y
z
;
otherwise
where x, y, and z are numerical input values.
7. The method of claim 6 , wherein the interpolation process includes a luma sample interpolation filtering process, wherein the predicted sample value includes a predicted luma sample value.
8. The method of claim 7 , wherein the luma sample interpolation filtering process receives inputs including a luma location in full sample units (xInt L , yInt L ), wherein the luma sample interpolation filtering process outputs the predicted luma sample value (predSampleLX L ), and wherein the clipping function is applied to the sample locations according to:
when subpic_treated_as_pic_flag[SubPicIdx] is equal to one, the following applies:
xInti =Clip3(SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xInt L +i −3), and
yInti =Clip3(SubPicTopBoundaryPos, SubPicBotBoundaryPos, yInt L +i −3),
where subpic_treated_as_pic_flag is the flag set to indicate the sub-picture is treated as a picture, SubPicIdx is an index of the sub-picture, xInti and yInti are a sample location at index i, SubPicRightBoundaryPos is a position of a right boundary of the sub-picture, SubPicLeftBoundaryPos is a position of a left boundary of the sub-picture, SubPicTopBoundaryPos is a position of a top boundary of the sub-picture,
SubPicBotBoundaryPos is a position of a bottom boundary of the sub-picture, and Clip3 is the clipping function according to:
Clip
3
(
x
,
y
,
z
)
=
{
x
;
z
<
y
y
;
z
>
y
z
;
otherwise
where x, y, and z are numerical input values.
9. The method of claim 6 , wherein the interpolation process includes a chroma sample interpolation process, wherein the block includes a block of chroma samples, and wherein the predicted sample value includes a predicted chroma sample value.
10. The method of claim 9 , wherein the chroma sample interpolation process receives inputs including a chroma location in full sample units (xInt C , yInt C ), wherein the chroma sample interpolation process outputs the predicted chroma sample value (predSampleLX C ), and wherein the clipping function is applied to the sample locations according to:
when subpic_treated_as_pic_flag[SubPicIdx] is equal to one, the following applies:
xInti =Clip3(SubPicLeftBoundaryPos/SubWidthC, SubPicRightBoundaryPos/SubWidthC, xInt C +i ), and
yInti =Clip3(SubPicTopBoundaryPos/SubHeightC, SubPicBotBoundaryPos/SubHeightC, yInt C +i ),
where subpic_treated_as_pic_flag is the flag set to indicate the sub-picture is treated as a picture, SubPicIdx is an index of the sub-picture, xInti and yInti are a sample location at index i, SubPicRightBoundaryPos is a position of a right boundary of the sub-picture, SubPicLeftBoundaryPos is a position of a left boundary of the sub-picture, SubPicTopBoundaryPos is a position of a top boundary of the sub-picture,
SubPicBotBoundaryPos is a position of a bottom boundary of the sub-picture, SubWidthC and SubHeightC indicate a horizontal and vertical sampling rate ratio between luma and chroma samples, and Clip3 is the clipping function according to:
Clip
3
(
x
,
y
,
z
)
=
{
x
;
z
<
y
y
;
z
>
y
z
;
otherwise
where x, y, and z are numerical input values.
11. A decoder comprising:
a receiver configured to receive a current picture including a sub-picture with a block coded according to inter-prediction; and
a processor coupled to the receiver and configured to:
determine a motion vector for the block;
apply a clipping function to sample locations in a reference block during application of an interpolation process when a flag is set to indicate the sub-picture is treated as a picture;
apply the interpolation process to results of the clipping function to obtain a predicted sample value; and
decode the block based on the predicted sample value,
wherein the interpolation process includes a luma sample bilinear interpolation process,
wherein the block includes a block of luma samples,
wherein the predicted sample value includes a predicted luma sample value,
wherein the luma sample bilinear interpolation process receives inputs including a luma location in full sample units (xInt L , yInt L ), wherein the luma sample bilinear interpolation process outputs the predicted luma sample value (predSampleLX L ), and wherein the clipping function is applied to the sample locations according to:
when subpic_treated_as_pic_flag[SubPicIdx] is equal to one, the following applies:
xInti =Clip3(SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xInt L +i ), and
yInti =Clip3(SubPicTopBoundaryPos, SubPicBotBoundaryPos, yInt L +i ),
where subpic_treated_as_pic_flag is the flag set to indicate the sub-picture is treated as a picture, SubPicIdx is an index of the sub-picture, xInti and yInti are a sample location at index i, SubPicRightBoundaryPos is a position of a right boundary of the sub-picture, SubPicLeftBoundaryPos is a position of a left boundary of the sub-picture, SubPicTopBoundaryPos is a position of a top boundary of the sub-picture,
SubPicBotBoundaryPos is a position of a bottom boundary of the sub-picture, and Clip3 is the clipping function according to:
Clip
3
(
x
,
y
,
z
)
=
{
x
;
z
<
y
y
;
z
>
y
z
;
otherwise
where x, y, and z are numerical input values.
12. The decoder of claim 11 , wherein the interpolation process includes a luma sample interpolation filtering process, and wherein the predicted sample value includes a predicted luma sample value.
13. The decoder of claim 12 , wherein the luma sample interpolation filtering process receives inputs including a luma location in full sample units (xInt L , yInt L ), wherein the luma sample interpolation filtering process outputs the predicted luma sample value (predSampleLX L ), and wherein the clipping function is applied to the sample locations according to:
when subpic_treated_as_pic_flag[SubPicIdx] is equal to one, the following applies:
xInti =Clip3(SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xInt L +i −3), and
yInti =Clip3(SubPicTopBoundaryPos, SubPicBotBoundaryPos, yInt L +i −3),
where subpic_treated_as_pic_flag is the flag set to indicate the sub-picture is treated as a picture, SubPicIdx is an index of the sub-picture, xInti and yInti are a sample location at index i, SubPicRightBoundaryPos is a position of a right boundary of the sub-picture, SubPicLeftBoundaryPos is a position of a left boundary of the sub-picture, SubPicTopBoundaryPos is a position of a top boundary of the sub-picture,
SubPicBotBoundaryPos is a position of a bottom boundary of the sub-picture, and Clip3 is the clipping function according to:
Clip
3
(
x
,
y
,
z
)
=
{
x
;
z
<
y
y
;
z
>
y
z
;
otherwise
where x, y, and z are numerical input values.
14. The decoder of claim 11 , wherein the interpolation process includes a chroma sample interpolation process, wherein the block includes a block of chroma samples, wherein the predicted sample value includes a predicted chroma sample value, wherein the chroma sample interpolation process receives inputs including a chroma location in full sample units (xInt C , yInt C ), wherein the chroma sample interpolation process outputs the predicted chroma sample value (predSampleLX C ), and wherein the clipping function is applied to the sample locations according to:
when subpic_treated_as_pic_flag[SubPicIdx] is equal to one, the following applies:
xInti =Clip3(SubPicLeftBoundaryPos/SubWidthC, SubPicRightBoundaryPos/SubWidthC, xInt C +i ), and
yInti =Clip3(SubPicTopBoundaryPos/SubHeightC, SubPicBotBoundaryPos/SubHeightC, yInt C +i ),
where subpic_treated_as_pic_flag is the flag set to indicate the sub-picture is treated as a picture, SubPicIdx is an index of the sub-picture, xInti and yInti are a sample location at index i, SubPicRightBoundaryPos is a position of a right boundary of the sub-picture, SubPicLeftBoundaryPos is a position of a left boundary of the sub-picture, SubPicTopBoundaryPos is a position of a top boundary of the sub-picture,
SubPicBotBoundaryPos is a position of a bottom boundary of the sub-picture, SubWidthC and SubHeightC indicate a horizontal and vertical sampling rate ratio between luma and chroma samples, and Clip3 is the clipping function according to:
Clip
3
(
x
,
y
,
z
)
=
{
x
;
z
<
x
y
;
z
>
y
z
;
otherwise
where x, y, and z are numerical input values.