IP Library Granted Patent US 12,713,071
Granted Patent B2
US 12,713,071 · App. 19/011,018 · Granted Aug 18, 2026

Picture timing information signaling in coded video

Inventor: Ye-Kui Wang (San Diego, CA)
Assignee: Bytedance Inc.
H04N19/70H04N19/136H04N19/184H04N19/188H04N19/30H04N19/433H04N19/46
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,713,071
App. No.
19/011,018
Filed
Jan 6, 2025
Granted
Aug 18, 2026
Kind
B2
Art Unit
2482
USPC
375/240.26
Abstract

Examples of video encoding methods and apparatus and video decoding methods and apparatus are described. An example method of video processing includes performing a conversion between a video and a bitstream of the video comprising one or more output layer sets according to a rule, wherein the rule specifies that a supplemental enhancement information (SEI) network abstraction layer (NAL) unit that includes a scalable-nested SEI message carrying picture timing information is not included due to use of a same picture timing in all output layer sets in the bitstream.

Claims (33)

1 . A method of processing video data, comprising:

performing a conversion between a video and a bitstream of the video comprising one or more output layer sets according to a rule,

wherein the rule specifies that a supplemental enhancement information (SEI) network abstraction layer (NAL) unit that includes a scalable-nested SEI message with a payload type of picture timing (PT) is not included when a flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1, and

wherein the rule further specifies that a particular payload type value corresponding to subpicture level information is disallowed from VclAssociatedSeiList that is a list including one or more SEI payload type values.

2 . The method of claim 1 , wherein the scalable-nested SEI message with the payload type of PT carries picture timing information.

3 . The method of claim 1 , wherein the scalable-nested SEI message with the payload type of PT has a payload type equal to 1.

4 . The method of claim 1 , wherein the rule further specifies that when the flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1, same picture timing in all output layer sets is used.

5 . The method of claim 4 , wherein the rule further specifies that when the flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1, non-scalable-nested SEI messages with the payload type of PT apply to all output layer sets.

6 . The method of claim 1 , wherein the one or more SEI payload type values include 3, 19, 45, 129, 137, 144, 145, 147 to 150, 153 to 156, 168, and 204.

7 . The method of claim 1 , wherein the particular payload type value is 203.

8 . The method of claim 1 , wherein the conversion includes encoding the video into the bitstream.

9 . The method of claim 1 , wherein the conversion includes decoding the video from the bitstream.

10 . An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

perform a conversion between a video and a bitstream of the video comprising one or more output layer sets according to a rule,

wherein the rule specifies that a supplemental enhancement information (SEI) network abstraction layer (NAL) unit that includes a scalable-nested SEI message with a payload type of picture timing (PT) is not included when a flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1, and

wherein the rule further specifies that a particular payload type value corresponding to subpicture level information is disallowed from VclAssociatedSeiList that is a list including one or more SEI payload type values.

11 . The apparatus of claim 10 , wherein the scalable-nested SEI message with the payload type of PT carries picture timing information.

12 . The apparatus of claim 10 , wherein the scalable-nested SEI message with the payload type of PT has a payload type equal to 1.

13 . The apparatus of claim 10 , wherein the rule further specifies that when the flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1, same picture timing in all output layer sets is used.

14 . The apparatus of claim 13 , wherein the rule further specifies that when the flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1, non-scalable-nested SEI messages with the payload type of PT apply to all output layer sets.

15 . A non-transitory computer-readable storage medium storing instructions that cause a processor to:

perform a conversion between a video and a bitstream of the video comprising one or more output layer sets according to a rule,

wherein the rule specifies that a supplemental enhancement information (SEI) network abstraction layer (NAL) unit that includes a scalable-nested SEI message with a payload type of picture timing (PT) is not included when a flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1, and

wherein the rule further specifies that a particular payload type value corresponding to subpicture level information is disallowed from VclAssociatedSeiList that is a list including one or more SEI payload type values.

16 . The non-transitory computer-readable storage medium of claim 15 , wherein the scalable-nested SEI message with the payload type of PT carries picture timing information, and wherein the scalable-nested SEI message with the payload type of PT has a payload type equal to 1.

17 . The non-transitory computer-readable storage medium of claim 15 , wherein the rule further specifies that when the flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1, same picture timing in all output layer sets is used, and non-scalable-nested SEI messages with the payload type of PT apply to all output layer sets.

18 . A method for storing a bitstream of a video, wherein the method comprises:

generating the bitstream of the video comprising one or more output layer sets according to a rule; and

storing the bitstream in a non-transitory computer-readable recording medium,

wherein the rule specifies that a supplemental enhancement information (SEI) network abstraction layer (NAL) unit that includes a scalable-nested SEI message with a payload type of picture timing (PT) is not included when a flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1, and

wherein the rule further specifies that a particular payload type value corresponding to subpicture level information is disallowed from VclAssociatedSeiList that is a list including one or more SEI payload type values.

19 . The method of claim 18 , wherein the scalable-nested SEI message with the payload type of PT carries picture timing information, and wherein the scalable-nested SEI message with the payload type of PT has a payload type equal to 1.

20 . The method of claim 18 , wherein the rule further specifies that when the flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1, same picture timing in all output layer sets is used, and non-scalable-nested SEI messages with the payload type of PT apply to all output layer sets.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 7, 2025
From: WANG, YE-KUI
To: BYTEDANCE INC.
Reel/Frame 070757/0227 →
Continuity (4)
Continuation 18078662 · Dec 9, 2022
Continuation PCTUS2021036487 · Jun 8, 2021
Provisional Application 63036808 · Jun 9, 2020
Related Publication 20250142127A1 · May 1, 2025
References Cited (133)
US 6324415B1 · Spehr · 2001 [cited by applicant]
US 9395142B2 · Rodich · 2016 [cited by applicant]
US 10645404B2 · Wang · 2020 [cited by applicant]
US 11871012B2 · Wang · 2024 [cited by applicant]
US 12096009B2 · Wang · 2024 [cited by applicant]
US 12126820B2 · Wang · 2024 [cited by applicant]
US 12192529B2 · Wang · 2025 [cited by applicant]
US 12316879B2 · Wang · 2025 [cited by applicant]
US 20100091837A1 · Zhu · 2010 [cited by applicant]
US 20130135434A1 · Chen · 2013 [cited by applicant]
US 20130290492A1 · ElArabawy · 2013 [cited by applicant]
US 20140016707A1 · Wang · 2014 [cited by applicant]
US 20140086341A1 · Wang · 2014 [cited by applicant]
US 20140098895A1 · Wang · 2014 [cited by applicant]
US 20140192149A1 · Wang · 2014 [cited by applicant]
US 20140301476A1 · Deshpande · 2014 [cited by applicant]
US 20140355692A1 · Ramasubramonian · 2014 [cited by applicant]
US 20150016547A1 · Tabatabai · 2015 [cited by applicant]
US 20150103927A1 · Hannuksela · 2015 [cited by applicant]
US 20150271498A1 · Wang · 2015 [cited by applicant]
US 20150271513A1 · Hendry · 2015 [cited by applicant]
US 20150271528A1 · Wang · 2015 [cited by applicant]
US 20150271529A1 · Wang · 2015 [cited by applicant]
US 20150358640A1 · Hendry · 2015 [cited by applicant]
US 20150373346A1 · Wang · 2015 [cited by applicant]
US 20150373374A1 · Ramasubramonian · 2015 [cited by applicant]
US 20160381385A1 · Ugur · 2016 [cited by applicant]
US 20160381394A1 · Sychev · 2016 [cited by applicant]
US 20170214923A1 · He · 2017 [cited by applicant]
US 20180014033A1 · Wu · 2018 [cited by applicant]
US 20190174144A1 · Hannuksela · 2019 [cited by applicant]
US 20190364279A1 · Yasugi · 2019 [cited by applicant]
US 20200077107A1 · Deshpande · 2020 [cited by applicant]
US 20200154144A1 · Deshpande · 2020 [cited by applicant]
US 20200296351A1 · Wang · 2020 [cited by applicant]
US 20220217375A1 · Wang · 2022 [cited by examiner]
US 20220217395A1 · Wang · 2022 [cited by examiner]
US 20220217411A1 · Wang · 2022 [cited by examiner]
US 20220217417A1 · Wang · 2022 [cited by applicant]
US 20220232259A1 · Wang · 2022 [cited by examiner]
US 20220377384A1 · Wang · 2022 [cited by applicant]
US 20230037902A1 · Sanchez De La Fuente · 2023 [cited by applicant]
US 20230043130A1 · Sanchez De La Fuente · 2023 [cited by applicant]
US 20230095686A1 · Sanchez De La Fuente · 2023 [cited by applicant]
CA 2951522C · 2019 [cited by applicant]
CN 1905541A · 2007 [cited by applicant]
CN 104662916A · 2015 [cited by applicant]
CN 104685891A · 2015 [cited by applicant]
CN 104704840A · 2015 [cited by applicant]
CN 104704842A · 2015 [cited by applicant]
CN 104919802A · 2015 [cited by applicant]
CN 105981387A · 2016 [cited by applicant]
CN 106105225A · 2016 [cited by applicant]
CN 106464935A · 2017 [cited by applicant]
CN 107431819A · 2017 [cited by applicant]
CN 107771395A · 2018 [cited by applicant]
JP 2011172252A · 2011 [cited by applicant]
JP 6162236B2 · 2017 [cited by applicant]
JP 6549288B2 · 2019 [cited by applicant]
JP 2020501430A · 2020 [cited by applicant]
JP 7503663B2 · 2024 [cited by applicant]
JP 7508598B2 · 2024 [cited by applicant]
JP 7508599B2 · 2024 [cited by applicant]
KR 102845874B1 · 2025 [cited by applicant]
RU 2619194C2 · 2017 [cited by applicant]
RU 2646378C2 · 2018 [cited by applicant]
RU 2833592C1 · 2025 [cited by applicant]
WO 2014107396A1 · 2014 [cited by applicant]
WO 2015007946A1 · 2015 [cited by applicant]
WO 2021123159A1 · 2021 [cited by applicant]
WO 2021123166A1 · 2021 [cited by applicant]
WO 2021201548A1 · 2021 [cited by applicant]
WO 2021252545A1 · 2021 [cited by applicant]
Document: JVET-Q2001-vE, Bross, B., et al., “Versatile Video Coding (Draft 8),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Brussels, BE, Jan. 7-17, 2020, 512 pages. [cited by applicant]
Document: JVET-Q0397, Skupin, R., et al., “AHG12: On subpicture extraction,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Brussels, BE, Jan. 7-17, 2020, 6 pages. [cited by applicant]
Implementation of H.264 Decoder Based on the Ffmpeg, Wang Tong, “Chinese Excellent Master's Thesis Electronic Journal,” Dec. 1, 2011, 56 pages. [cited by applicant]
Chinese Notice of Allowance from Chinese Patent Application No. 202180042416.8 dated Aug. 21, 2025, 8 pages. [cited by applicant]
Chinese Notice of Allowance from Chinese Patent Application No. 202180041866.5 dated Oct. 1, 2025, 8 pages. [cited by applicant]
JVET-P0125-v2 , Wang, Y.K., et al., “AHG8/AHG17: Miscellaneous HLS topics,” Joint Video Experts Team (JVET) of ITU-TSG 16 WP3 and ISO /IEC JTC 1/SC29 /WG 11 16th Meeting: Geneva, CH, Oct. 1-11, 2019, 3 pages. [cited by applicant]
Yuan Qichao, Design and Optimization of Storage and Transmission of Cross Random Access Point Reference Encoding Video, Information Technology, Issue 2, Feb. 15, 2021, 81 pages. [cited by applicant]
Chinese Office Action from Chinese Patent Application No. 202180041808.2 dated Mar. 21, 2025, 9 pages. [cited by applicant]
Chinese Office Action from Chinese Patent Application No. 202180041770.9 dated Mar. 31, 2025, 12 pages. [cited by applicant]
Chinese Office Action from Chinese Patent Application No. 202180041866.5 dated Apr. 9, 2025, 14 pages. [cited by applicant]
Chinese Office Action from Chinese Patent Application No. 202180042416.8 dated Apr. 16, 2025, 10 pages. [cited by applicant]
“Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services—Coding of moving video High efficiency video coding,” Rec. ITU-T H.265, ISO/IEC 23008-2 (in force edition), Feb. 2018, 692 pages. [cited by applicant]
Document: JVET-G1001-v1, Chen, J., et al., “Algorithm Description of Joint Exploration Test Model 7 (JEM 7),” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 7th Meeting: Torino, IT… [cited by applicant]
Document: JVET-R2001-vB/v11, Bross, B., et al., “Versatile Video Coding (Draft 9),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 18th Meeting: by teleconference, Apr. 15-24, 2020, 53… [cited by applicant]
Document: JVET-S0152-v5, Wang, Y., “AHG2: Editorial input of a text integration for the May 2020 HLS AHG meeting outcome,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 19th Meeting: … [cited by applicant]
Document: JVET-Q2002-v3, Chen, J., et al., “Algorithm description for Versatile Video Coding and Test Model 8 (VTM 8),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Bru… [cited by applicant]
“Li, X., VTM software: Retrieved from the internet: https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM.git, Mar. 3, 2023, 3 pages.”. [cited by applicant]
Document: JCTVC-AC1005-v2, Boyce, J., et al., “HEVC Additional Supplemental Enhancement Information (Draft 4).” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 29th Me… [cited by applicant]
Schierl et al.“System Layer Integration of High Efficiency Video Coding ”, IEEE Transactions On Circuits and Systems for Video Technology, vol. 22, No. 12, Dec. 2012, 14 pages. [cited by applicant]
Wang, Y., et al. “The High-Level Syntax of the Versatile Video Coding (WC) Standard” Retrieved from the Internet: URL: https://ieeexplore.ieee.org/stamp/stamp.jspamumber=9395142, IEEE Transactions on Circuits and System… [cited by applicant]
JVET-Q0488-v1, Wan, W., et al., “AHG9: Bounding redundant SEI messages,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Brussels, BE, Jan. 7-17, 2020, 4 pages. [cited by applicant]
JVET-P2001-vE, Bross, B., et al., “Versatile Video Coding (Draft 7),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 16th Meeting: Geneva, CH, Oct. 1-11, 2019, 491 pages. [cited by applicant]
JVET-P0125-v1, Wang, Y.K., “AHG8/AHG17: Miscellaneous HLS topics,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 16th Meeting: Geneva, CH, Oct. 1-11, 2019, 3 pages. [cited by applicant]
JVET-Q0394, Sanchez, Y., et al., “AHG9: On OLS extraction and scalable nesting SEI message,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Brussels, BE, Jan. 7-17, 2020,… [cited by applicant]
JVET-Q0397, Skupin, R., et al., “AHG12: On subpicture extraction,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Brussels, BE, Jan. 7-17, 2020, 6 pages. [cited by applicant]
JVET-Q2001-vB, Bross, B., et al., “Versatile Video Coding (Draft 8),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Brussels, BE, Jan. 7-17, 2020, 519 pages. [cited by applicant]
Document: JVET-R0101-v1, Deshpande, S., et al., “On Alternative Timing Information Signalling,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 18th Meeting: by teleconference, Apr. 15-… [cited by applicant]
JVET-Q0394-v2-SANCHEZ Y. et al., AHG9: On OLS extraction and scalable nesting SEI message, JVET-Q0394, 17 JVET Meeting, Jan. 7, 2020-Jan. 17, 2020, Brussels, The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 an… [cited by applicant]
Document: JVET-R2001-vA, Bross< B., et al., “Versatile Video Coding (Draft 9),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 18th Meeting: by teleconference, Apr. 15-24, 2020, 524 pa… [cited by applicant]
Document: JVET-S0157-v1, Wang, Y.K., et al., “AHG9: HRD and related cleanups,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 19th Meeting: by teleconference, Jun. 22-Jul. 1, 2020, 7 p… [cited by applicant]
Ye-Kui WangAHG8/AHG17: Miscellaneous HLS topicsJoint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11JVET-P0125-v216th Meeting: Geneva, CHpp. 1-3. [cited by applicant]
Document: JVET-S0178-v3, Wang, Y.K., et al., “AHG9: General SEI semantics and constraints,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 19th Meeting: by teleconference, Jun. 22-Jul.… [cited by applicant]
Document: JVET-Q2001-vE, Bross, B., et al., “Versatile Video Coding (Draft 8),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 17th Meeting: Brussels, BE, Jan. 7-17, 2020, 512 pages. … [cited by applicant]
Request for the Submission of an Opinion for Korean Application No. 10-2022-7043181, Nov. 12, 2024, 13 [g] pages. [cited by applicant]
Japanese Notice of Allowance from Japanese Patent Application No. 2022-575702 dated May 21, 2024, 6 pages. [cited by applicant]
Japanese Notice of Allowance from Japanese Patent Application No. 2022-575701 dated May 14, 2024, 5 pages. [cited by applicant]
Russian Office Action from Russian Application No. 2022132302 dated Jun. 17, 2024, 19 pages. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 18/077,810 dated Feb. 1, 2024, 8 pages. [cited by applicant]
Extended European Search Report from European Application No. 21822668.6 dated Feb. 9, 2024, 14 pages. [cited by applicant]
Extended European Search Report from European Application No. 21821514.3 dated Oct. 18, 2023, 9 pages. [cited by applicant]
Extended European Search Report from European Application No. 21821749.5 dated Oct. 9, 2023, 11 pages. [cited by applicant]
Partial Supplementary European Search Report from European Application No. 21822668.6 dated Oct. 20, 2023, 14 pages. [cited by applicant]
Foreign Communication From A Related Counterpart Application, Indian Application No. 202247071333, Indian Office Action dated May 9, 2023, 7 pages. [cited by applicant]
Non-Final Office Action dated May 17, 2023, 14 pages, U.S. Appl. No. 18/077,886, filed Dec. 8, 2022. [cited by applicant]
Non-Final Office Action dated Apr. 10, 2023, 16 pages, U.S. Appl. No. 18/077,866, filed Dec. 8, 2022. [cited by applicant]
Foreign Communication From A Counterpart Application, PCT Application No. PCT/US2021/036484, International Search Report dated Sep. 1, 2021, 66 pages. [cited by applicant]
Foreign Communication From A Counterpart Application, PCT Application No. PCT/US2021/036487, International Search Report dated Sep. 1, 2021, 29 pages. [cited by applicant]
Foreign Communication From A Counterpart Application, Indian Application No. 202247071748, Indian Office Action dated Feb. 13, 2023, 6 pages. [cited by applicant]
Foreign Communication From A Counterpart Application, PCT Application No. PCT/US2021/036489, International Search Report dated Sep. 1, 2021, 61 pages. [cited by applicant]
Foreign Communication From A Counterpart Application, Indian Application No. 202247071693, Indian Office Action dated Feb. 13, 2023, 7 pages. [cited by applicant]
Foreign Communication From A Counterpart Application, PCT Application No. PCT/US2021/036491, International Search Report dated Sep. 14, 2021, 16 pages. [cited by applicant]
Foreign Communication From A Counterpart Application, Indian Application No. 202247071247, Indian Office Action dated Feb. 22, 2023, 6 pages. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 18/078,662 dated Apr. 10, 2023, 15 pages. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 18/078,662 dated Dec. 20, 2023, 15 pages. [cited by applicant]
Final Office Action from U.S. Appl. No. 18/078,662 dated Mar. 26, 2024, 10 pages. [cited by applicant]
Final Office Action from U.S. Appl. No. 18/078,662 dated Jul. 17, 2023, 15 pages. [cited by applicant]
Notice of Allowance from U.S. Appl. No. 18/078,662 dated Aug. 22, 2024, 16 pages. [cited by applicant]
Japanese Office Action from JP Application No. 2022-575701 dated Jan. 23, 2024, 2 pages. [cited by applicant]
Japanese Office Action from JP Patent Application No. 2022-575702 dated Feb. 6, 2024, 5 pages. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 18/517,283 dated Aug. 28, 2024, 22 pages. [cited by applicant]