IP Library › Granted Patent US 12,489,924
Granted Patent B2
US 12,489,924 · App. 18/514,689 · Granted Dec 2, 2025

Scaling window in video coding

Inventors: Ye-kui Wang (San Diego, CA); Zhipin Deng (Beijing, CN); Kai Zhang (San Diego, CA); Li Zhang (San Diego, CA)
Assignees: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.; BYTEDANCE INC.
H04N19/70H04N19/105H04N19/119H04N19/132H04N19/174H04N19/184H04N19/186H04N19/30
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,489,924
App. No.
18/514,689
Granted
Dec 2, 2025
Kind
B2
Abstract

Video processing, including video coding, video decoding and video transcoding, is described. One example method includes performing a conversion between a video comprising a video picture and a bitstream of a video according to a rule, where the rule specifies that syntax elements indicate a first width and a first height of a scaling window for the video picture, and where the rule specifies that a range of permissible values for the syntax elements include values greater than or equal to two times a second width of the video picture and two times a second height of the video picture.

Claims (94)

1 . A method of video processing, comprising:

performing a conversion between a video comprising a video picture and a bitstream of the video according to a rule,

wherein the rule specifies that a first syntax element is included in a sequence parameter set that the video picture refers to,

wherein the first syntax element indicates whether a reference picture resampling is enabled and whether one or more slices of the video picture are allowed to refer to a reference picture in an active entry in a reference picture list, and

wherein the reference picture has one or more of multiple parameters that are different than that of the video picture, the multiple parameters include the following six parameters: 1) a video picture width, 2) a video picture height, 3) a scaling window left offset, 4) a scaling window right offset, 5) a scaling window top offset, and 6) a scaling window bottom offset, which are specified by second syntax elements.

2 . The method of claim 1 , wherein the first syntax element having a value of 1 indicates that the reference picture resampling is enabled and that the video picture is allowed to have one or more slices that refer to the reference picture in the active entry in the reference picture list.

3 . The method of claim 1 , wherein the first syntax element having a value of 0 indicates that the reference picture resampling is disabled and that no slice of the video picture is allowed to refer to the reference picture in the active entry of the reference picture list.

4 . The method of claim 1 , wherein the rule specifies that second syntax elements indicate a first width and a first height of a scaling window for the video picture, and

wherein the rule specifies that a range of permissible values for the first width and the first height that are indicated by the second syntax elements are allowed to be greater than or equal to two times a second width of the video picture and two times a second height of the video picture.

5 . The method of claim 4 , wherein values of the second syntax elements specify offsets that are applied to a size of the video picture for scaling ratio calculation.

6 . The method of claim 4 , wherein the second syntax elements include pps_scaling_win_left_offset, pps_scaling_win_right_offset, pps_scaling_win_top_offset, and pps_scaling_win_bottom offset.

7 . The method of claim 6 , wherein the rule specifies that a value of SubWidthC*(pps_scaling_win_left_offset+pps_scaling_win_right_offset) is greater than or equal to −pps_pic_width_in_luma_samples*M, and less than pps_pic_width_in_luma_samples,

wherein CurrPicScalWinWidthL is the first width of the scaling window,

wherein pps_pic_width_in_luma_samples is the second width of the video picture,

wherein SubWidthC is a chroma width sampling variable of the video picture and obtained from a table according to a chroma format of the video picture,

wherein CurrPicScalWinWidthL=pps_pic_width_in_luma_samples−SubWidthC*(pps_scaling_win_right_offset+pps_scaling_win_left_offset), and

wherein M is a positive integer value that is greater than or equal to 2.

8 . The method of claim 7 , wherein the rule specifies that a value of SubHeightC*(pps_scaling_win_top_offset+pps_scaling_win_bottom_offset) is greater than or equal to −pps_pic_height_in_luma_samples*N and less than pps_pic_height_in_luma_samples,

wherein CurrPicScalWinHeightL is the first height of the scaling window,

wherein pps_pic_height_in_luma_samples is the second height of the video picture,

wherein SubHeightC is a chroma height sampling variable of the video picture and obtained from the table according to the chroma format of the video picture,

wherein CurrPicScalWinHeightL=pps_pic_height_in_luma_samples−SubHeightC*(pps_scaling_win_bottom_offset+pps_scaling_win_top_offset), and

wherein N is a positive integer value that is greater than or equal to 2.

9 . The method of claim 8 , wherein N is equal to M.

10 . The method of claim 1 , wherein the performing the conversion comprises encoding the video into the bitstream.

11 . The method of claim 1 , wherein the performing the conversion comprises decoding the video from the bitstream.

12 . An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

perform a conversion between a video comprising a video picture and a bitstream of the video according to a rule,

wherein the rule specifies that a first syntax element is included in a sequence parameter set that the video picture refers to,

wherein the first syntax element indicates whether a reference picture resampling is enabled and whether one or more slices of the video picture are allowed to refer to a reference picture in an active entry in a reference picture list, and

wherein the reference picture has one or more of multiple parameters that are different than that of the video picture, the multiple parameters include the following six parameters: 1) a video picture width, 2) a video picture height, 3) a scaling window left offset, 4) a scaling window right offset, 5) a scaling window top offset, and 6) a scaling window bottom offset, which are specified by second syntax elements.

13 . The apparatus of claim 12 , wherein the first syntax element having a value of 1 indicates that the reference picture resampling is enabled and that the video picture is allowed to have one or more slices that refer to the reference picture in the active entry in the reference picture list, and

wherein the first syntax element having a value of 0 indicates that the reference picture resampling is disabled and that no slice of the video picture is allowed to refer to the reference picture in the active entry of the reference picture list.

14 . The apparatus of claim 12 , wherein the rule specifies that second syntax elements indicate a first width and a first height of a scaling window for the video picture, and

wherein the rule specifies that a range of permissible values for the first width and the first height that are indicated by the second syntax elements are allowed to be greater than or equal to two times a second width of the video picture and two times a second height of the video picture.

15 . The apparatus of claim 14 , wherein values of the second syntax elements specify offsets that are applied to a size of the video picture for scaling ratio calculation.

16 . The apparatus of claim 14 , wherein the second syntax elements include pps_scaling_win_left_offset, pps_scaling_win_right_offset, pps_scaling_win_top_offset, and pps_scaling_win_bottom_offset,

wherein the rule specifies that a value of SubWidthC*(pps_scaling_win_left_offset+pps_scaling_win_right_offset) is greater than or equal to pps_pic_width_in_luma_samples*M and less than pps_pic_width_in_luma_samples,

wherein CurrPicScalWinWidthL is the first width of the scaling window,

wherein pps_pic_width_in_luma_samples is the second width of the video picture,

wherein SubWidthC is a chroma width sampling variable of the video picture and obtained from a table according to a chroma format of the video picture,

wherein CurrPicScalWinWidthL=pps_pic_width_in_luma_samples−SubWidthC*(pps_scaling_win_right_offset+pps_scaling_win_left_offset),

wherein M is a positive integer value that is greater than or equal to 2,

wherein the rule specifies that a value of SubHeightC*(pps_scaling_win_top_offset+pps_scaling_win_bottom_offset) is greater than or equal to −pps_pic_height_in_luma_samples*N and less than pps_pic_height_in_luma_samples,

wherein CurrPicScalWinHeightL is the first height of the scaling window,

wherein pps_pic_height_in_luma_samples is the second height of the video picture,

wherein SubHeightC is a chroma height sampling variable of the video picture and obtained from the table according to the chroma format of the video picture,

wherein CurrPicScalWinHeightL=pps_pic_height_in_luma_samples−SubHeightC*(pps_scaling_win_bottom_offset+pps_scaling_win_top_offset),

wherein N is a positive integer value that is greater than or equal to 2, and

wherein N is equal to M.

17 . A non-transitory computer-readable storage medium storing instructions that cause a processor to:

perform a conversion between a video comprising a video picture and a bitstream of the video according to a rule,

wherein the rule specifies that a first syntax element is included in a sequence parameter set that the video picture refers to,

wherein the first syntax element indicates whether a reference picture resampling is enabled and whether one or more slices of the video picture are allowed to refer to a reference picture in an active entry in a reference picture list, and

wherein the reference picture has one or more of multiple parameters that are different than that of the video picture, the multiple parameters include the following six parameters: 1) a video picture width, 2) a video picture height, 3) a scaling window left offset, 4) a scaling window right offset, 5) a scaling window top offset, and 6) a scaling window bottom offset, which are specified by second syntax elements.

18 . The non-transitory computer-readable storage medium of claim 17 , wherein the first syntax element having a value of 1 indicates that the reference picture resampling is enabled and that the video picture is allowed to have one or more slices that refer to the reference picture in the active entry in the reference picture list,

wherein the first syntax element having a value of 0 indicates that the reference picture resampling is disabled and that no slice of the video picture is allowed to refer to the reference picture in the active entry of the reference picture list,

wherein the rule specifies that second syntax elements indicate a first width and a first height of a scaling window for the video picture,

wherein the rule specifies that a range of permissible values for the first width and the first height that are indicated by the second syntax elements are allowed to be greater than or equal to two times a second width of the video picture and two times a second height of the video picture,

wherein values of the second syntax elements specify offsets that are applied to a size of the video picture for scaling ratio calculation, and the second syntax elements include pps_scaling_win_left_offset, pps_scaling_win_right_offset, pps_scaling_win_top_offset, and pps_scaling_win_bottom_offset,

wherein the rule specifies that a value of SubWidthC*(pps_scaling_win_left_offset+pps_scaling_win_right_offset)) is greater than or equal to pps_pic_width_in_luma_samples*M and less than pps_pic_width_in_luma_samples,

wherein CurrPicScalWinWidthL is the first width of the scaling window,

wherein pps_pic_width_in_luma_samples is the second width of the video picture,

wherein SubWidthC is a chroma width sampling variable of the video picture and obtained from a table according to a chroma format of the video picture,

wherein CurrPicScalWinWidthL=pps_pic_width_in_luma_samples−SubWidthC*(pps_scaling_win_right_offset+pps_scaling_win_left_offset),

wherein M is a positive integer value that is greater than or equal to 2,

wherein the rule specifies that a value of SubHeightC*(pps_scaling_win_top_offset+pps_scaling_win_bottom_offset) is greater than or equal to −pps_pic_height_in_luma_samples*N and less than pps_pic_height_in_luma_samples,

wherein CurrPicScalWinHeightL is the first height of the scaling window,

wherein pps_pic_height_in_luma_samples is the second height of the video picture,

wherein SubHeightC is a chroma height sampling variable of the video picture and obtained from the table according to the chroma format of the video picture,

wherein CurrPicScalWinHeightL=pps_pic_height_in_luma_samples−SubHeightC*(pps_scaling_win_bottom_offset+pps_scaling_win_top_offset), and

wherein N is a positive integer value that is greater than or equal to 2, and N is equal to M.

19 . A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:

generating the bitstream of the video comprising a video picture according to a rule,

wherein the rule specifies that a first syntax element is included in a sequence parameter set that the video picture refers to,

wherein the first syntax element indicates whether a reference picture resampling is enabled and whether one or more slices of the video picture are allowed to refer to a reference picture in an active entry in a reference picture list, and

wherein the reference picture has one or more of multiple parameters that are different than that of the video picture, the multiple parameters include the following six parameters: 1) a video picture width, 2) a video picture height, 3) a scaling window left offset, 4) a scaling window right offset, 5) a scaling window top offset, and 6) a scaling window bottom offset, which are specified by second syntax elements.

20 . The non-transitory computer-readable recording medium of claim 19 , wherein the first syntax element having a value of 1 indicates that the reference picture resampling is enabled and that the video picture is allowed to have one or more slices that refer to the reference picture in the active entry in the reference picture list,

wherein the first syntax element having a value of 0 indicates that the reference picture resampling is disabled and that no slice of the video picture is allowed to refer to the reference picture in the active entry of the reference picture list,

wherein the rule specifies that second syntax elements indicate a first width and a first height of a scaling window for the video picture,

wherein the rule specifies that a range of permissible values for the first width and the first height that are indicated by the second syntax elements are allowed to be greater than or equal to two times a second width of the video picture and two times a second height of the video picture,

wherein values of the second syntax elements specify offsets that are applied to a size of the video picture for scaling ratio calculation, and the second syntax elements include pps_scaling_win_left_offset, pps_scaling_win_right_offset, pps_scaling_win_top_offset, and pps_scaling_win_bottom_offset,

wherein the rule specifies that a value of SubWidthC*(pps_scaling_win_left_offset+pps_scaling_win_right_offset)) is greater than or equal to pps_pic_width_in_luma_samples*M and less than pps_pic_width_in_luma_samples,

wherein CurrPicScalWinWidthL is the first width of the scaling window,

wherein pps_pic_width_in_luma_samples is the second width of the video picture,

wherein SubWidthC is a chroma width sampling variable of the video picture and obtained from a table according to a chroma format of the video picture,

wherein CurrPicScalWinWidthL=pps_pic_width_in_luma_samples-SubWidthC*(pps_scaling_win_right_offset+pps_scaling_win_left_offset),

wherein M is a positive integer value that is greater than or equal to 2,

wherein the rule specifies that a value of SubHeightC*(pps_scaling_win_top_offset+pps_scaling_win_bottom_offset) is greater than or equal to −pps_pic_height_in_luma_samples*N and less than pps_pic_height_in_luma_samples,

wherein CurrPicScalWinHeightL is the first height of the scaling window,

wherein pps_pic_height_in_luma_samples is the second height of the video picture,

wherein SubHeightC is a chroma height sampling variable of the video picture and obtained from the table according to the chroma format of the video picture,

wherein CurrPicScalWinHeightL=pps_pic_height_in_luma_samples-SubHeightC*(pps_scaling_win_bottom_offset+pps_scaling_win_top_offset), and

wherein N is a positive integer value that is greater than or equal to 2, and N is equal to M.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 24, 2024
From: WANG, YE-KUI; ZHANG, KAI; ZHANG, LI
To: BYTEDANCE INC.
Reel/Frame 066226/0285 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 24, 2024
From: DENG, ZHIPIN
To: BEIJING ZITIAO NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 066226/0372 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 24, 2024
From: BEIJING ZITIAO NETWORK TECHNOLOGY CO., LTD.
To: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 066226/0514 →
Priority Claims (1)
WO PCT/CN2020/091533 · May 21, 2020 · international
Continuity (3)
Continuation 17990267 · Nov 18, 2022
Continuation PCTCN2021095012 · May 21, 2021
Related Publication 20240089508A1 · Mar 14, 2024
References Cited (36)
US 10499068B2 · Hannuksela · 2019 [cited by applicant]
US 20180242008A1 · Minoo · 2018 [cited by applicant]
US 20210185332A1 · Chuang · 2021 [cited by applicant]
US 20210195172A1 · Chang · 2021 [cited by applicant]
US 20220286667A1 · Lu · 2022 [cited by examiner]
US 20230082583A1 · Wang · 2023 [cited by examiner]
CN 101646084A · 2010 [cited by applicant]
CN 108293136A · 2018 [cited by applicant]
JP 7506184B2 · 2024 [cited by applicant]
WO 2018127625A1 · 2018 [cited by applicant]
Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services—Coding of moving video High efficiency video coding ITU-T and ISO/IEC, “High efficiency video coding”, Rec. ITU-T H.265 | ISO/IEC 23008… [cited by applicant]
JVET-G1001-v1, Chen, J., et al., “Algorithm Description of Joint Exploration Test Model 7 (JEM 7),” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 7th Meeting: Torino, IT, Jul. 13-… [cited by applicant]
Document: JVET-R2001-vA, Bross, B., et al., “Versatile Video Coding (Draft 9),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 18th Meeting: by teleconference, Apr. 15-24, 2020, 524 pa… [cited by applicant]
JVET-Q2002-v3, Chen, J., et al., “Algorithm description for Versatile Video Coding and Test Model 8 (VTM 8),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Brussels, BE,… [cited by applicant]
VTM software, Retrieved from the internet: https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM.git, Feb. 8, 2023, 3 pages. [cited by applicant]
JCTVC-AC1005-v2, Boyce, J., et al., “HEVC Additional Supplemental Enhancement Information (Draft 4),” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 29th Meeting: Mac… [cited by applicant]
JVET-S0048-v2, Wang, Y-K., et al., “AHG9/AHG8: On reference picture resampling,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 19th Meeting: by teleconference, Jun. 22-Jul. 1, 2020, 4… [cited by applicant]
Jvet-Q0334, Chang, Y-J., et al., “AhG8/AhG12: On the reference picture resampling for the subpictures,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Brussels, BE, Jan. … [cited by applicant]
JVET-Q0333, Chang, Y-J., et al., “AhG12: On the subpicture-based scaling process,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Brussels, BE, Jan. 7-17, 2020, 9 pages. [cited by applicant]
JVET-R0278, Seregin, V., et al., “AHG8: On SPS sharing and slice type constraint,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 18th Meeting: by teleconference, Apr. 15-24, 2020, 5 p… [cited by applicant]
JVET-S0147-v2, Wang, Y-K., et al., “AHG12: A summary of proposals on scalability and RPR,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 19th Meeting: by teleconference, Jun. 22-Jul. … [cited by applicant]
JVET-Q0262, Samuelsson, J., et al., “AHG9: On reference picture resampling enabled flag,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Brussels, BE, Jan. 7-17, 2020, 2 … [cited by applicant]
Document: JVET-R0114, Samuelsson, J., et al., “AHG 9: On scaling window offsets,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 18th Meeting: by teleconference, Apr. 15-24, 2020, 4 pa… [cited by applicant]
Document: JVET-O0204, Samuelsson, J., et al., “AHG8: Adaptive Resolution Change (ARC) High-Level Syntax (HLS),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 15th Meeting: Gothenburg,… [cited by applicant]
Document: JVET-Q0332, Choi, B., et al., “AHG8: Signaling and Filtering for Reference Picture Resampling,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 15th Meeting: Gothenburg, SE, J… [cited by applicant]
Foreign Communication From A Related Counterpart Application, European Application No. 21807814.5, Extended European Search Report dated May 19, 2023, 12 pages. [cited by applicant]
Foreign Communication From A Related Counterpart Application, PCT Application No. PCT/CN2021/095012, International Search Report dated Aug. 25, 2021, 11 pages. [cited by applicant]
Foreign Communication From A Related Counterpart Application, PCT Application No. PCT/CN2021/095013, International Search Report dated Jul. 22, 2021, 12 pages. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 17/990,267 dated Apr. 6, 2023. [cited by applicant]
Document: JVET-R0114-v3, Samuelsson, J., et al., “AHG 9: On scaling window offsets,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP3 and ISO/IEC JTC 1/SC 29/WG 11 18th Meeting: by teleconference, Apr. 15-24, 2020, 4 … [cited by applicant]
Japanese Notice of Allowance from Japanese Patent Application No. 2022-570205 dated May 14, 2024, 6 pages. [cited by applicant]
Document: JVET-S0048-v2, Wang, Y.K., et al., “AHG9/AHG8: On reference picture resampling,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 19th Meeting: by teleconference, Jun. 22-Jul. … [cited by applicant]
Document: JVET-Q2001-vE, Bross, B., et al., “Versatile Video Coding (Draft 8),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC 1/SC 29/WG 11 17th Meeting: Brussels, BE, Jan. 7-17, 2020, 512 pages. [cited by applicant]
Document: JVET-Q0334-v2, Chang, Y-J., et al., “AhG8/AhG12: On the reference picture resampling for the subpictures,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Brusse… [cited by applicant]
Singapore Office Action from Singapore Patent Application No. 11202260430V dated Sep. 9, 2025, 9 pages. [cited by applicant]
Brazilian Notice of Allowance from Brazilian Patent Application No. 112022023391-4 dated Jul. 8, 2025, 10 pages. [cited by applicant]