IP Library Granted Patent US 11,632,540
Granted Patent B2
US 11,632,540 · App. 17/119,758 · Granted Apr 18, 2023

Reference picture scaling ratios for reference picture resampling in video coding

Inventors: Yao-Jen Chang (San Diego, CA); Vadim Seregin (San Diego, CA); Muhammed Zeyd Coban (Carlsbad, CA)
Assignee: QUALCOMM INCORPORATED
H04N19/105H04N19/132H04N19/176H04N19/30H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,632,540
App. No.
17/119,758
Granted
Apr 18, 2023
Kind
B2
Abstract

A video encoder and video decoder may be configured to disable subpicture processing in situations where reference picture scaling is enabled. In another example, when reference picture scaling is enabled, a video encoder may determine a reference picture scaling ratio for the reference picture resampling process based on a width of a scaling window (PicOutputWidthL) of the current picture and a height of the scaling window (PicOutputHeightL) of the current picture. The values of PicOutputWidthL and PicOutputHeightL are constrained, respectively, relative to a width of a scaling window (refPicOutputWidthL) of the reference picture and a height of the scaling window (refPicOutputHeightL) of the reference picture.

Claims (98)

1. A method of decoding video data, the method comprising:

receiving a current picture of video data;

determining, based on a first syntax element, that reference picture resampling is enabled for the current picture;

determining that a number of subpictures is one based on the reference picture resampling being enabled;

determining a reference picture for a first block of the current picture;

performing a reference picture scaling process on the reference picture based on the reference picture having a different size than the current picture to produce a scaled reference picture; and

decoding the first block using the scaled reference picture.

2. The method of claim 1 , wherein determining that the number of subpictures is one based on the reference picture resampling being enabled comprises:

determining that a value of a second syntax element indicating the number of subpictures minus 1 is equal to 0 based on the reference picture resampling being enabled.

3. The method of claim 1 , further comprising:

determining a reference picture scaling ratio for the reference picture resampling based on a width of a scaling window (PicOutputWidthL) of the current picture and a height of the scaling window (PicOutputHeightL) of the current picture, wherein values of PicOutputWidthL and PicOutputHeightL are constrained, respectively, relative to a width of a scaling window (refPicOutputWidthL) of the reference picture and a height of the scaling window (refPicOutputHeightL) of the reference picture; and

performing the reference picture resampling using the determined reference picture scaling ratio.

4. The method of claim 3 , wherein the values of PicOutputWidthL and PicOutputHeightL are constrained such that PicOutputWidthL*A is greater than or equal to refPicOutputWidthL, PicOutputHeightL*A is greater than or equal to refPicOutputHeightL, PicOutputWidthL is less than or equal to refPicOutputWidthL*B, and PicOutputHeightL is less than or equal to refPicOutputHeightL*B, where parameters A and B are any non-zero positive number, and B is greater than A.

5. The method of claim 4 , wherein A equals 2 and B equals 8.

6. The method of claim 1 , further comprising:

determining one or more reference picture scaling values for the reference picture scaling process;

clipping the one or more reference picture scaling values to be within a predefined range; and

performing the reference picture resampling in accordance with the clipped reference picture scaling values.

7. The method of claim 1 , further comprising:

displaying the current picture.

8. An apparatus configured to decode video data, the apparatus comprising:

a memory configured to store video data; and

one or more processors implemented in circuitry and in communication with the memory, the one or more processors configured to:

receive a current picture of the video data;

determine, based on a first syntax element, that reference picture resampling is enabled for the current picture;

determine that a number of subpictures is one based on the reference picture resampling being enabled;

determine a reference picture for a first block of the current picture;

perform a reference picture scaling process on the reference picture based on the reference picture having a different size than the current picture to produce a scaled reference picture; and

decode the first block using the scaled reference picture.

9. The apparatus of claim 8 , wherein to determine that the number of subpictures is one based on the reference picture resampling being enabled, the one or more processors are further configured to:

determine that a value of a second syntax element indicating the number of subpictures minus 1 is equal to 0 based on the reference picture resampling being enabled.

10. The apparatus of claim 8 , wherein the one or more processors are further configured to:

determine a reference picture scaling ratio for the reference picture resampling based on a width of a scaling window (PicOutputWidthL) of the current picture and a height of the scaling window (PicOutputHeightL) of the current picture, wherein values of PicOutputWidthL and PicOutputHeightL are constrained, respectively, relative to a width of a scaling window (refPicOutputWidthL) of the reference picture and a height of the scaling window (refPicOutputHeightL) of the reference picture; and

perform the reference picture resampling using the determined reference picture scaling ratio.

11. The apparatus of claim 10 , wherein the values of PicOutputWidthL and PicOutputHeightL are constrained such that PicOutputWidthL*A is greater than or equal to refPicOutputWidthL, PicOutputHeightL*A is greater than or equal to refPicOutputHeightL, PicOutputWidthL is less than or equal to refPicOutputWidthL*B, and PicOutputHeightL is less than or equal to refPicOutputHeightL*B, where parameters A and B are any non-zero positive number, and B is greater than A.

12. The apparatus of claim 11 , wherein A equals 2 and B equals 8.

13. The apparatus of claim 8 , wherein the one or more processors are further configured to:

determine one or more reference picture scaling values for the reference picture scaling process;

clip the one or more reference picture scaling values to be within a predefined range; and

perform the reference picture resampling in accordance with the clipped reference picture scaling values.

14. The apparatus of claim 8 , further comprising:

a display configured to display the current picture.

15. An apparatus configured to decode video data, the apparatus comprising:

means for receiving a current picture of video data;

means for determining, based on a first syntax element, that reference picture resampling is enabled for the current picture;

means for determining that a number of subpictures is one based on the reference picture resampling being enabled;

means for determining a reference picture for a first block of the current picture;

means for performing a reference picture scaling process on the reference picture based on the reference picture having a different size than the current picture to produce a scaled reference picture; and

means for decoding the first block using the scaled reference picture.

16. A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors configured to decode video data to:

receive a current picture of the video data;

determine, based on a first syntax element, that reference picture resampling is enabled for the current picture;

determine that a number of subpictures is one based on the reference picture resampling being enabled;

determine a reference picture for a first block of the current picture;

perform a reference picture scaling process on the reference picture based on the reference picture having a different size than the current picture to produce a scaled reference picture; and

decode the first block using the scaled reference picture.

17. A method of encoding video data, the method comprising:

receiving a current picture of video data;

determining that reference picture resampling is enabled for the current picture;

determining that a number of subpictures is one based on the reference picture resampling being enabled;

determining a reference picture for a first block of the current picture;

performing a reference picture scaling process on the reference picture based on the reference picture having a different size than the current picture to produce a scaled reference picture; and

encoding the first block using the scaled reference picture.

18. The method of claim 17 , wherein determining that the number of subpictures is one based on the reference picture resampling being enabled comprises:

determining that a value of a second syntax element indicating the number of subpictures minus 1 is equal to 0 based on the reference picture resampling being enabled.

19. The method of claim 17 , further comprising:

determining a reference picture scaling ratio for the reference picture resampling based on a width of a scaling window (PicOutputWidthL) of the current picture and a height of the scaling window (PicOutputHeightL) of the current picture, wherein values of PicOutputWidthL and PicOutputHeightL are constrained, respectively, relative to a width of a scaling window (refPicOutputWidthL) of the reference picture and a height of the scaling window (refPicOutputHeightL) of the reference picture; and

performing the reference picture resampling using the determined reference picture scaling ratio.

20. The method of claim 19 , wherein the values of PicOutputWidthL and PicOutputHeightL are constrained such that PicOutputWidthL * A is greater than or equal to refPicOutputWidthL, PicOutputHeightL * A is greater than or equal to refPicOutputHeightL, PicOutputWidthL is less than or equal to refPicOutputWidthL * B, and PicOutputHeightL is less than or equal to refPicOutputHeightL * B, where parameters A and B are any non-zero positive number, and B is greater than A.

21. The method of claim 20 , wherein A equals 2 and B equals 8.

22. The method of claim 17 , further comprising:

determining one or more reference picture scaling values for the reference picture scaling process;

clipping the one or more reference picture scaling values to be within a predefined range; and

performing the reference picture resampling in accordance with the clipped reference picture scaling values.

23. The method of claim 17 , further comprising:

capturing the current picture.

24. An apparatus configured to encode video data, the apparatus comprising:

a memory configured to store video data; and

one or more processors implemented in circuitry and in communication with the memory, the one or more processors configured to:

receive a current picture of the video data;

determine that reference picture resampling is enabled for the current picture;

determine that a number of subpictures is one based on the reference picture resampling being enabled;

determine a reference picture for a first block of the current picture;

perform a reference picture scaling process on the reference picture based on the reference picture having a different size than the current picture to produce a scaled reference picture; and

encode the first block using the scaled reference picture.

25. The apparatus of claim 24 , wherein to determine that the number of subpictures is one based on the reference picture resampling being enabled, the one or more processors are further configured to:

determine that a value of a second syntax element indicating the number of subpictures minus 1 is equal to 0 based on the reference picture resampling being enabled.

26. The apparatus of claim 24 , wherein the one or more processors are further configured to:

determine a reference picture scaling ratio for the reference picture resampling based on a width of a scaling window (PicOutputWidthL) of the current picture and a height of the scaling window (PicOutputHeightL) of the current picture, wherein values of PicOutputWidthL and PicOutputHeightL are constrained, respectively, relative to a width of a scaling window (refPicOutputWidthL) of the reference picture and a height of the scaling window (refPicOutputHeightL) of the reference picture; and

perform the reference picture resampling using the determined reference picture scaling ratio.

27. The apparatus of claim 26 , wherein the values of PicOutputWidthL and PicOutputHeightL are constrained such that PicOutputWidthL * A is greater than or equal to refPicOutputWidthL, PicOutputHeightL * A is greater than or equal to refPicOutputHeightL, PicOutputWidthL is less than or equal to refPicOutputWidthL * B, and PicOutputHeightL is less than or equal to refPicOutputHeightL * B, where parameters A and B are any non-zero positive number, and B is greater than A.

28. The apparatus of claim 27 , wherein A equals 2 and B equals 8.

29. The apparatus of claim 24 , wherein the one or more processors are further configured to:

determine one or more reference picture scaling values for the reference picture scaling process;

clip the one or more reference picture scaling values to be within a predefined range; and

perform the reference picture resampling in accordance with the clipped reference picture scaling values.

30. The apparatus of claim 24 , further comprising:

a camera configured to capture the current picture.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 5, 2021
From: CHANG, YAO-JEN; SEREGIN, VADIM; COBAN, MUHAMMED ZEYD
To: QUALCOMM INCORPORATED
Reel/Frame 054818/0820 →
Continuity (2)
Provisional Application 62951961 · Dec 20, 2019
Related Publication 20210195172A1 · Jun 24, 2021