IP Library Granted Patent US 12,192,529
Granted Patent B2
US 12,192,529 · App. 18/078,662 · Granted Jan 7, 2025

Picture timing information signaling in coded video

Inventor: Ye-kui Wang (San Diego, CA)
Assignee: Bytedance Inc.
H04N19/70H04N19/136H04N19/184H04N19/188H04N19/30H04N19/433H04N19/46
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,192,529
App. No.
18/078,662
Granted
Jan 7, 2025
Kind
B2
Abstract

Examples of video encoding methods and apparatus and video decoding methods and apparatus are described. An example method of video processing includes performing a conversion between a video and a bitstream of the video comprising one or more output layer sets according to a rule, wherein the rule specifies that a supplemental enhancement information (SEI) network abstraction layer (NAL) unit that includes a scalable-nested SEI message carrying picture timing information is not included due to use of a same picture timing in all output layer sets in the bitstream.

Claims (36)

1. A method of processing video data, comprising:

performing a conversion between a video and a bitstream of the video comprising one or more output layer sets according to a rule,

wherein the rule specifies that a supplemental enhancement information (SEI) network abstraction layer (NAL) unit that includes a scalable-nested SEI message with a payload type of picture timing (PT) is not included when a flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1,

wherein the rule further specifies that a non-scalable-nested hypothetical reference decoder (HRD)-related supplemental enhancement information (SEI) message is omitted responsive to a condition that there does not exist an output layer set that includes video layers as the bitstream, and

wherein the rule further specifies that a particular payload type value corresponding to subpicture level information is disallowed from VclAssociatedSeiList that is a list including one or more supplemental enhancement information (SEI) payload type values.

2. The method of claim 1 , wherein the scalable-nested SEI message with the payload type of PT carries picture timing information.

3. The method of claim 1 , wherein the scalable-nested SEI message with the payload type of PT has a payload type equal to 1.

4. The method of claim 1 , wherein the rule further specifies that when the flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1, same picture timing in all output layer sets is used.

5. The method of claim 4 , wherein the rule further specifies that when the flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1, non-scalable-nested SEI messages with the payload type of PT apply to all output layer sets.

6. The method of claim 1 , wherein the conversion includes encoding the video into the bitstream.

7. The method of claim 1 , wherein the conversion includes decoding the video from the bitstream.

8. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

perform a conversion between a video and a bitstream of the video comprising one or more output layer sets according to a rule,

wherein the rule specifies that a supplemental enhancement information (SEI) network abstraction layer (NAL) unit that includes a scalable-nested SEI message with a payload type of picture timing (PT) is not included when a flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1,

wherein the rule further specifies that a non-scalable-nested hypothetical reference decoder (HRD)-related supplemental enhancement information (SEI) message is omitted responsive to a condition that there does not exist an output layer set that includes video layers as the bitstream, and

wherein the rule further specifies that a particular payload type value corresponding to subpicture level information is disallowed from VclAssociatedSeiList that is a list including one or more supplemental enhancement information (SEI) payload type values.

9. The apparatus of claim 8 , wherein the scalable-nested SEI message with the payload type of PT carries picture timing information.

10. The apparatus of claim 8 , wherein the scalable-nested SEI message with the payload type of PT has a payload type equal to 1.

11. The apparatus of claim 8 , wherein the rule further specifies that when the flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1, same picture timing in all output layer sets is used.

12. The apparatus of claim 11 , wherein the rule further specifies that when the flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1, non-scalable-nested SEI messages with the payload type of PT apply to all output layer sets.

13. A non-transitory computer-readable storage medium storing instructions that cause a processor to:

perform a conversion between a video and a bitstream of the video comprising one or more output layer sets according to a rule,

wherein the rule specifies that a supplemental enhancement information (SEI) network abstraction layer (NAL) unit that includes a scalable-nested SEI message with a payload type of picture timing (PT) is not included when a flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1,

wherein the rule further specifies that a non-scalable-nested hypothetical reference decoder (HRD)-related supplemental enhancement information (SEI) message is omitted responsive to a condition that there does not exist an output layer set that includes video layers as the bitstream, and wherein the rule further specifies that a particular payload type value corresponding to subpicture level information is disallowed from VclAssociatedSeiList that is a list including one or more supplemental enhancement information (SEI) payload type values.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the scalable-nested SEI message with the payload type of PT carries picture timing information, and wherein the scalable-nested SEI message with the payload type of PT has a payload type equal to 1.

15. The non-transitory computer-readable storage medium of claim 13 , wherein the rule further specifies that when the flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1, same picture timing in all output layer sets is used, and non-scalable-nested SEI messages with the payload type of PT apply to all output layer sets.

16. A method for storing a bitstream of a video, wherein the method comprises:

generating the bitstream of the video comprising one or more output layer sets according to a rule, and

storing the bitstream in a non-transitory computer-readable recording medium,

wherein the rule specifies that a supplemental enhancement information (SEI) network abstraction layer (NAL) unit that includes a scalable-nested SEI message with a payload type of picture timing (PT) is not included when a flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1,

wherein the rule further specifies that a non-scalable-nested hypothetical reference decoder (HRD)-related supplemental enhancement information (SEI) message is omitted responsive to a condition that there does not exist an output layer set that includes video layers as the bitstream, and

wherein the rule further specifies that a particular payload type value corresponding to subpicture level information is disallowed from VclAssociatedSeiList that is a list including one or more supplemental enhancement information (SEI) payload type values.

17. The non-transitory computer-readable recording medium of claim 16 , wherein the scalable-nested SEI message with the payload type of PT carries picture timing information, and wherein the scalable-nested SEI message with the payload type of PT has a payload type equal to 1.

18. The non-transitory computer-readable recording medium of claim 16 , wherein the rule further specifies that when the flag included in the bitstream named general_same_pic_timing_in_all_ols_flag has a value of 1, same picture timing in all output layer sets is used, and non-scalable-nested SEI messages with the payload type of PT apply to all output layer sets.

19. The method of claim 1 , wherein the one or more SEI payload type values include 3, 19, 45, 129, 137, 144, 145, 147 to 150, 153 to 156, 168, and 204.

20. The method of claim 1 , wherein the particular payload type value is 203.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 9, 2023
From: WANG, YE-KUI
To: BYTEDANCE INC.
Reel/Frame 062927/0805 →
Continuity (3)
Continuation PCTUS2021036487 · Jun 8, 2021
Provisional Application 63036808 · Jun 9, 2020
Related Publication 20230105511A1 · Apr 6, 2023
References Cited (85)
US 6324415B1 · Spehr · 2001 [cited by applicant]
US 9395142B2 · Rodich · 2016 [cited by applicant]
US 10645404B2 · Wang · 2020 [cited by applicant]
US 20100091837A1 · Zhu · 2010 [cited by applicant]
US 20130135434A1 · Chen · 2013 [cited by applicant]
US 20130290492A1 · ElArabawy · 2013 [cited by applicant]
US 20140016707A1 · Wang · 2014 [cited by applicant]
US 20140086341A1 · Wang · 2014 [cited by applicant]
US 20140098895A1 · Wang · 2014 [cited by applicant]
US 20140192149A1 · Wang · 2014 [cited by examiner]
US 20140301476A1 · Deshpande · 2014 [cited by applicant]
US 20140355692A1 · Ramasubramonian · 2014 [cited by applicant]
US 20150103927A1 · Hannuksela · 2015 [cited by applicant]
US 20150271498A1 · Wang · 2015 [cited by applicant]
US 20150271529A1 · Wang · 2015 [cited by applicant]
US 20150358640A1 · Hendry · 2015 [cited by applicant]
US 20160381385A1 · Ugur · 2016 [cited by applicant]
US 20160381394A1 · Sychev · 2016 [cited by applicant]
US 20170214923A1 · He et al. · 2017 [cited by applicant]
US 20180014033A1 · Wu · 2018 [cited by applicant]
US 20190174144A1 · Hannuksela · 2019 [cited by applicant]
US 20190364279A1 · Yasugi · 2019 [cited by applicant]
US 20200077107A1 · Deshpande · 2020 [cited by applicant]
US 20200154144A1 · Deshpande · 2020 [cited by applicant]
US 20200296351A1 · Wang · 2020 [cited by applicant]
US 20220377384A1 · Wang · 2022 [cited by applicant]
US 20230037902A1 · Sánchez De La Fuente · 2023 [cited by examiner]
US 20230043130A1 · Sánchez De La Fuente · 2023 [cited by examiner]
US 20230095686A1 · Sánchez De La Fuente · 2023 [cited by examiner]
CA 2951522C · 2019 [cited by applicant]
JP 6162236B2 · 2017 [cited by applicant]
JP 6549288B2 · 2019 [cited by applicant]
JP 2020501430A · 2020 [cited by applicant]
JP 7503663B2 · 2024 [cited by applicant]
JP 7508598B2 · 2024 [cited by applicant]
JP 7508599B2 · 2024 [cited by applicant]
RU 2619194C2 · 2017 [cited by applicant]
RU 2646378C2 · 2018 [cited by applicant]
WO 2014107396A1 · 2014 [cited by applicant]
WO 2021123159A1 · 2021 [cited by applicant]
WO 2021123166A1 · 2021 [cited by applicant]
WO WO2021201548A1 · 2021 [cited by examiner]
WO 2021252545A1 · 2021 [cited by applicant]
“Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services—Coding of moving video High efficiency video coding,” Rec. ITU-T H.265 | ISO/IEC 23008-2 (in force edition), Feb. 2018, 692 pages. [cited by applicant]
Document: JVET-G1001-v1, Chen, J., et al., “Algorithm Description of Joint Exploration Test Model 7 (JEM 7),” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 7th Meeting: Torino, IT… [cited by applicant]
Document: JVET-R2001-vB, Bross, B., et al., “Versatile Video Coding (Draft 9),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 18th Meeting: by teleconference, Apr. 15-24, 2020, 530 pa… [cited by applicant]
Document: JVET-S0152-v5, Wang, Y., “AHG2: Editorial input of a text integration for the May 2020 HLS AHG meeting outcome,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 19th Meeting: … [cited by applicant]
Document: JVET-Q2002-v3, Chen, J., et al., “Algorithm description for Versatile Video Coding and Test Model 8 (VTM 8),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Bru… [cited by applicant]
Li, X., VTM software: Retrieved from the internet: https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM.git, Mar. 3, 2023, 3 pages. [cited by applicant]
Document: JCTVC-AC1005-v2, Boyce, J., et al., “HEVC Additional Supplemental Enhancement Information (Draft 4).” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 29th Me… [cited by applicant]
Schierl et al.,“System Layer Integration of High Efficiency Video Coding ”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, No. 12, Dec. 2012, 14 pages. [cited by applicant]
Wang, Y., et al. “The High-Level Syntax of the Versatile Video Coding (VVC) Standard” Retrieved from the Internet: URL: https://ieeexplore.ieee.org/stamp/stamp.jsparnumber=9395142, IEEE Transactions on Circuits and Syst… [cited by applicant]
Foreign Communication From a Counterpart Application, PCT Application No. PCT/US2021/036484, International Search Report dated Sep. 1, 2021, 66 pages. [cited by applicant]
Foreign Communication From a Counterpart Application, PCT Application No. PCT/US2021/036487, International Search Report dated Sep. 1, 2021, 29 pages. [cited by applicant]
Foreign Communication From a Counterpart Application, Indian Application No. 202247071748, Indian Office Action dated Feb. 13, 2023, 6 pages. [cited by applicant]
Foreign Communication From a Counterpart Application, PCT Application No. PCT/US2021/036489, International Search Report dated Sep. 1, 2021, 61 pages. [cited by applicant]
Foreign Communication From a Counterpart Application, Indian Application No. 202247071693, Indian Office Action dated Feb. 13, 2023, 7 pages. [cited by applicant]
Foreign Communication From a Counterpart Application, PCT Application No. PCT/US2021/036491, International Search Report dated Sep. 14, 2021, 16 pages. [cited by applicant]
Foreign Communication From a Counterpart Application, Indian Application No. 202247071247, Indian Office Action dated Feb. 22, 2023, 6 pages. [cited by applicant]
Foreign Communication From a Related Counterpart Application, Indian Application No. 202247071333, Indian Office Action dated May 9, 2023, 7 pages. [cited by applicant]
Non-Final Office Action dated May 17, 2023, 14 pages, U.S. Appl. No. 18/077,886, filed Dec. 8, 2022. [cited by applicant]
Non-Final Office Action dated Apr. 10, 2023, 16 pages, U.S. Appl. No. 18/077,866, filed Dec. 8, 2022. [cited by applicant]
Document: JVET-Q0488-v1, Wan, W., et al., “AHG9: Bounding redundant SEI messages,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Brussels, BE, Jan. 7-17, 2020, 4 pages. [cited by applicant]
Document: JVET-P2001-vE, Bross, B., et al., “Versatile Video Coding (Draft 7),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 16th Meeting: Geneva, CH, Oct. 1-11, 2019, 491 pages. [cited by applicant]
Document: JVET-P0125-v1, Wang, Y.K., “AHG8/AHG17: Miscellaneous HLS topics,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 16th Meeting: Geneva, CH, Oct. 1-11, 2019, 3 pages. [cited by applicant]
Document: JVET-Q0394, Sanchez, Y., “AHG9: On OLS extraction and scalable nesting SEI message,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Brussels, BE, Jan. 7-17, 202… [cited by applicant]
Document: JVET-Q0397, Skupin, R., et al., “AHG12: On subpicture extraction,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Brussels, BE, Jan. 7-17, 2020, 6 pages. [cited by applicant]
Document: JVET-Q2001-vB, Bross, B., et al., “Versatile Video Coding (Draft 8),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Brussels, BE, Jan. 7-17, 2020, 519 pages. [cited by applicant]
Extended European Search Report from European Application No. 21821514.3 dated Oct. 18, 2023, 9 pages. [cited by applicant]
Extended European Search Report from European Application No. 21821749.5 dated Oct. 9, 2023, 11 pages. [cited by applicant]
Partial Supplementary European Search Report from European Application No. 21822668.6 dated Oct. 20, 2023, 14 pages. [cited by applicant]
Document: JVET-R0101-v1, Deshpande, S., et al., “On Alternative Timing Information Signalling,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 18th Meeting: by teleconference, Apr. 15-… [cited by applicant]
Document: JVET-R2001-vA, Bross, B., et al., “Versatile Video Coding (Draft 9),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 18th Meeting: by teleconference, Apr. 15-24, 2020, 524 pa… [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 18/077,810 dated Feb. 1, 2024, 8 pages. [cited by applicant]
Extended European Search Report from European Application No. 21822668.6 dated Feb. 9, 2024, 14 pages. [cited by applicant]
Document: JVET-S0157-v1, Wang, Y.K., et al., “AHG9: HRD and related cleanups,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 19th Meeting: by teleconference, Jun. 22-Jul. 1, 2020, 7 p… [cited by applicant]
Document: JVET-P0125-v2, Wang, Y.K., et al., “AHG8/AHG17: Miscellaneous HLS topics,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 16th Meeting: Geneva, CH, Oct. 1-11, 2019, 3 pages. [cited by applicant]
Document: JVET-S0178-v3, Wang, Y.K., et al., “AHG9: General SEI semantics and constraints,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 19th Meeting: by teleconference, Jun. 22-Jul.… [cited by applicant]
Japanese Notice of Allowance from Japanese Patent Application No. 2022-575702 dated May 21, 2024, 6 pages. [cited by applicant]
Japanese Notice of Allowance from Japanese Patent Application No. 2022-575701 dated May 14, 2024, 5 pages. [cited by applicant]
Document: JVET-Q0394, Sanchez, Y., et al., “AHG9: On OLS extraction and scalable nesting SEI message,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 17th Meeting: Brussels, BE, Jan. 7… [cited by applicant]
Document: JVET-R2001-vA, Bross< B., et al., “Versatile Video Coding (Draft 9),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WVG 11 18th Meeting: by teleconference, Apr. 15-24, 2020, 524 p… [cited by applicant]
Russian Office Action from Russian Application No. 2022132302 dated Jun. 17, 2024, 19 pages. [cited by applicant]
Document: JVET-Q2001-vE, Bross, B., et al., “Versatile Video Coding (Draft 8),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 17th Meeting: Brussels, BE, Jan. 7-17, 2020, 512 pages. … [cited by applicant]
Request for the Submission of an Opinion for Korean Application No. 10-2022-7043181, Nov. 12, 2024, 13 pages. [cited by applicant]
Cited By (1)
US 12,713,071