IP Library › Granted Patent US 12,316,880
Granted Patent B2
US 12,316,880 · App. 18/523,025 · Granted May 27, 2025

Reference picture resampling

Inventors: Zhipin Deng (Beijing, CN); Li Zhang (San Diego, CA); Ye-kui Wang (San Diego, CA); Kai Zhang (San Diego, CA)
Assignees: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.; BYTEDANCE INC.
H04N19/70H04N19/105H04N19/174
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,316,880
App. No.
18/523,025
Granted
May 27, 2025
Kind
B2
Abstract

Examples of video encoding methods and apparatus and video decoding methods and apparatus are described. An example method of video processing includes performing a conversion between a current picture of a video and a bitstream of the video according to a rule. The rule specifies that a number of entries in a reference picture list of the current picture is greater than 0 in response to (1) one or more slices in the current picture being allowed to have a slice type other than an Intra (I) slice type, and (2) reference picture list (RPL) information being present in a picture header.

Claims (52)

1. A method of processing video data, comprising:

performing a conversion between a current picture of a video and a bitstream of the video according to a first rule,

wherein the first rule includes a bitstream constraint which specifies that a number of entries in a reference picture list of the current picture is greater than 0 in response to (1) one or more slices in the current picture being allowed to have a slice type other than an Intra (I) slice type, and (2) reference picture list (RPL) information being present in a picture header,

wherein the current picture of the video comprises a current video block,

wherein a range of a value of a fourth syntax element specifying a number of points in a quantization parameter (QP) table associated with the conversion of the current video block is dependent on one or more other syntax elements comprising a fifth syntax element according to a second rule, and

wherein the second rule specifies that the range of the value of the fourth syntax element is based on a value of the fifth syntax element specifying a starting luma and chroma QP used to describe the QP table.

2. The method of claim 1 , wherein the reference picture list is a reference picture list 0.

3. The method of claim 1 , wherein a first syntax flag in the picture header indicates whether the one or more slices in the current picture are allowed to have the slice type other than the Intra (I) slice type.

4. The method of claim 1 , wherein a second syntax flag in a picture parameter set indicates whether the RPL information is present in the picture header instead of slice headers.

5. The method of claim 1 , wherein the number of entries in the reference picture list of the current picture is indicated by a third syntax element included in the bitstream.

6. The method of claim 1 , wherein the QP table is an i-th chroma QP mapping table.

7. The method of claim 6 , wherein a maximum allowed value of the fourth syntax element is equal to (K—the value of the fifth syntax element), wherein K is a positive integer.

8. The method of claim 7 , wherein the fifth syntax element is represented as sps_qp_table_start_minus26 [i] whose value plus 26 specifies the starting luma and chroma QP used to describe the i-th chroma QP mapping table.

9. The method of claim 8 , wherein the maximum allowed value of the fourth syntax element is equal to (36-sps_qp_table_start_minus26 [i]).

10. The method of claim 1 , wherein the second rule further specifies that the current picture is a recovery point picture in response to the current picture being a gradual decoding refresh (GDR) picture with a recovery picture order count (POC) count of 0, and wherein the recovery POC count specifies a recovery point of decoded pictures in an output order.

11. The method of claim 1 , wherein the conversion comprises encoding the video into the bitstream.

12. The method of claim 1 , wherein the conversion comprises decoding the video from the bitstream.

13. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

perform a conversion between a current picture of a video and a bitstream of the video according to a first rule,

wherein the first rule includes a bitstream constraint which specifies that a number of entries in a reference picture list of the current picture is greater than 0 in response to (1) one or more slices in the current picture being allowed to have a slice type other than an Intra (I) slice type, and (2) reference picture list (RPL) information being present in a picture header,

wherein the current picture of the video comprises a current video block,

wherein a range of a value of a fourth syntax element specifying a number of points in a quantization parameter (QP) table associated with the conversion of the current video block is dependent on one or more other syntax elements comprising a fifth syntax element according to a second rule, and

wherein the second rule specifies that the range of the value of the fourth syntax element is based on a value of the fifth syntax element specifying a starting luma and chroma QP used to describe the QP table.

14. The apparatus of claim 13 , wherein the reference picture list is a reference picture list 0,

wherein a first syntax flag in the picture header indicates whether the one or more slices in the current picture are allowed to have the slice type other than the Intra (I) slice type,

wherein a second syntax flag in a picture parameter set indicates whether the RPL information is present in the picture header instead of slice headers, and

wherein the number of entries in the reference picture list of the current picture is indicated by a third syntax element included in the bitstream.

15. The apparatus of claim 13 , wherein the QP table is an i-th chroma QP mapping table,

wherein a maximum allowed value of the fourth syntax element is equal to (K—the value of the fifth syntax element), wherein K is a positive integer,

wherein the fifth syntax element is represented as sps_qp_table_start_minus26 [i] whose value plus 26 specifies the starting luma and chroma QP used to describe the i-th chroma QP mapping table, and

wherein the maximum allowed value of the fourth syntax element is equal to (36-sps_qp_table_start_minus26 [i]).

16. The apparatus of claim 13 , wherein the second rule further specifies that the current picture is a recovery point picture in response to the current picture being a gradual decoding refresh (GDR) picture with a recovery picture order count (POC) count of 0, and wherein the recovery POC count specifies a recovery point of decoded pictures in an output order.

17. A non-transitory computer-readable storage medium storing instructions that cause a processor to:

perform a conversion between a current picture of a video and a bitstream of the video according to a first rule,

wherein the first rule includes a bitstream constraint which specifies that a number of entries in a reference picture list of the current picture is greater than 0 in response to (1) one or more slices in the current picture being allowed to have a slice type other than an Intra (I) slice type, and (2) reference picture list (RPL) information being present in a picture header,

wherein the current picture of the video comprises a current video block,

wherein a range of a value of a fourth syntax element specifying a number of points in a quantization parameter (QP) table associated with the conversion of the current video block is dependent on one or more other syntax elements comprising a fifth syntax element according to a second rule, and

wherein the second rule specifies that the range of the value of the fourth syntax element is based on a value of the fifth syntax element specifying a starting luma and chroma QP used to describe the QP table.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the reference picture list is a reference picture list 0,

wherein a first syntax flag in the picture header indicates whether the one or more slices in the current picture are allowed to have the slice type other than the Intra (I) slice type,

wherein a second syntax flag in a picture parameter set indicates whether the RPL information is present in the picture header instead of slice headers, and

wherein the number of entries in the reference picture list of the current picture is indicated by a third syntax element included in the bitstream.

19. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:

generating the bitstream of the video from a current picture in the video according to a first rule,

wherein the first rule includes a bitstream constraint which specifies that a number of entries in a reference picture list of the current picture is greater than 0 in response to (1) one or more slices in the current picture being allowed to have a slice type other than an Intra (I) slice type, and (2) reference picture list (RPL) information being present in a picture header,

wherein the current picture of the video comprises a current video block,

wherein a range of a value of a fourth syntax element specifying a number of points in a quantization parameter (QP) table associated with a conversion of the current video block is dependent on one or more other syntax elements comprising a fifth syntax element according to a second rule, and

wherein the second rule specifies that the range of the value of the fourth syntax element is based on a value of the fifth syntax element specifying a starting luma and chroma QP used to describe the QP table.

20. The non-transitory computer-readable recording medium of claim 19 , wherein the reference picture list is a reference picture list 0,

wherein a first syntax flag in the picture header indicates whether the one or more slices in the current picture are allowed to have the slice type other than the Intra (I) slice type,

wherein a second syntax flag in a picture parameter set indicates whether the RPL information is present in the picture header instead of slice headers, and

wherein the number of entries in the reference picture list of the current picture is indicated by a third syntax element included in the bitstream.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 29, 2024
From: ZHANG, LI; WANG, YE-KUI; ZHANG, KAI
To: BYTEDANCE INC.
Reel/Frame 066276/0544 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 29, 2024
From: DENG, ZHIPIN
To: BEIJING ZITIAO NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 066276/0764 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 29, 2024
From: BEIJING ZITIAO NETWORK TECHNOLOGY CO., LTD.
To: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 066276/0956 →
Priority Claims (3)
WO PCT/CN2020/094397 · Jun 4, 2020 · international
WO PCT/CN2020/095688 · Jun 11, 2020 · international
WO PCT/CN2020/097390 · Jun 22, 2020 · international
Continuity (3)
Continuation 18074887 · Dec 5, 2022
Continuation PCTCN2021097844 · Jun 2, 2021
Related Publication 20240107071A1 · Mar 28, 2024
References Cited (43)
US 10136152B2 · Hendry · 2018 [cited by applicant]
US 10390020B2 · Chang · 2019 [cited by applicant]
US 11611778B2 · Samuelsson · 2023 [cited by applicant]
US 11924475B2 · Deng · 2024 [cited by applicant]
US 20160360198A1 · Chang · 2016 [cited by applicant]
US 20200177923A1 · Chen · 2020 [cited by applicant]
US 20220060752A1 · Pettersson · 2022 [cited by applicant]
US 20220217347A1 · Chernyak · 2022 [cited by applicant]
US 20220286667A1 · Lu · 2022 [cited by applicant]
CN 101646084A · 2010 [cited by applicant]
CN 104365099A · 2015 [cited by applicant]
CN 106165427 · 2016 [cited by applicant]
CN 106464911A · 2017 [cited by applicant]
CN 106165427B · 2019 [cited by applicant]
JP 2022549911A · 2022 [cited by applicant]
WO 2019191218A1 · 2019 [cited by applicant]
WO 2020031061A2 · 2020 [cited by applicant]
Document: JVET-S0092, Samuelsson, J., et al., “On constraint info signalling,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 19th Meeting: by teleconference, Jun. 22-Jul. 1, 2020, 3 p… [cited by applicant]
Document: JVET-S0152-v3, Wang, Y., “AHG2: Editorial input of a text integration for the May 2020 HLS AHG meeting outcome,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 19th Meeting: … [cited by applicant]
“Information technology—High efficiency coding and media delivery in heterogeneous environments—Part 2: High efficiency video coding”, ISO/IEC JTC 1/SC 29/WG 11 N17661, Rec. ITU-T H.265 | ISO/IEC 23008-2:201x (4th Ed.) … [cited by applicant]
Document: JVET-R2001-vA, Bross, B., et al., “Versatile Video Coding (Draft 9),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 18th Meeting: by teleconference, Apr. 15-24, 2020, 524 pa… [cited by applicant]
Document: JVET-Q2002-v3, Chen, J., et al., “Algorithm description for Versatile Video Coding and Test Model 8 (VTM 8),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Bru… [cited by applicant]
Bossen, F., VTM software, Retrieved from the internet: https://vcgit.hhi.fraunhofer.dc/ivct/VVCSoftwargVTM.git, Feb. 27, 2023, 3 pages. [cited by applicant]
Document: JVET-R0253, Yu, R., et al., “AHG9: Three restrictions when RPL is present in PH,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 18th Meeting: by teleconference, Apr. 15-24, … [cited by applicant]
Document: JVET-Q0043, Nishi, T., et al., “AHG9: Constraint about usage of reference picture resampling and subpictures,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Br… [cited by applicant]
Document: JVET-Q0625-v4, Boyce, J., “Report of BoG on high level tool control, feature combinations, and parameter sets,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: B… [cited by applicant]
Document: JVET-O0417-v1, Bordes, P., et al., “AHG17: On general constraint information,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, 3… [cited by applicant]
Document: JVET-R0192, Hendry, “AHG9: On signalling recovery point picture,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 18th Meeting: by teleconference, Apr. 15-24, 2020, 4 pages. [cited by applicant]
Document: JVET-S2001-v5, Bross, B., et al., “Versatile Video Coding (Draft 10),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 19th Meeting: by teleconference, Jun. 22-Jul. 1, 2020, 5… [cited by applicant]
Document: JVET-S0130-v3, Li, L., et al., “On chroma QP mapping,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 19th Meeting: by teleconference, Jun. 22-Jul. 1, 2020, 6 pages. [cited by applicant]
Foreign Communication From a Related Counterpart Application, PCT Application No. PCT/CN2021/097844, International Search Report dated Aug. 30, 2021, 10 pages. [cited by applicant]
Foreign Communication From a Related Counterpart Application, Indian Application No. 202227070068, Indian Office Action dated Feb. 17, 2023, 8 pages. [cited by applicant]
Foreign Communication From a Related Counterpart Application, PCT Application No. PCT/CN2021/101389, International Search Report dated Sep. 26, 2021, 13 pages. [cited by applicant]
Foreign Communication From a Related Counterpart Application, PCT Application No. PCT/CN2021/101390, International Search Report dated Sep. 27, 2021, 13 pages. [cited by applicant]
Extended European Search Report from European Application No. 21827968.5 dated Oct. 20, 2023, 11 pages. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 18/085,988 dated Jul. 5, 2023, 19 pages. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 18/074,887 dated Apr. 7, 2023, 14 pages. [cited by applicant]
Final Office Action from U.S. Appl. No. 18/074,887 dated Jul. 17, 2023, 9 pages. [cited by applicant]
Document: JVET-S0241, Hendry, et al., “AHG9: On signalling of POC MSB information B43in picture header,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 19th Meeting: by teleconference,… [cited by applicant]
Extended European Search Report from European Application No. 23220634.2 dated Mar. 18, 2024, 14 pages. [cited by applicant]
Japanese Notification of Reasons for Refusal from Japanese Application No. 2022-577200 dated Mar. 12, 2024, 6 pages. [cited by applicant]
Document: JVET-G1001-v1, Chen, J., et al., “Algorithm Description of Joint Exploration Test Model 7 (JEM 7),” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 7th Meeting: Torino, IT… [cited by applicant]
Foreign Communication From A Related Counterpart Application, Office Action for Indian Application No. 202227074501, mailed Mar. 8, 2023, 6 Pages. [cited by applicant]