IP Library › Granted Patent US 12,284,377
Granted Patent B2
US 12,284,377 · App. 17/701,359 · Granted Apr 22, 2025

Layer based parameter set NAL unit constraints

Inventor: Ye-Kui Wang (San Diego, CA)
Assignee: Huawei Technologies Co., Ltd.
H04N19/46H04N19/102H04N19/105H04N19/132H04N19/172H04N19/174H04N19/188H04N19/30H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,284,377
App. No.
17/701,359
Granted
Apr 22, 2025
Kind
B2
Abstract

A video coding mechanism is disclosed. The mechanism includes receiving a bitstream comprising a picture parameter set (PPS) in a PPS network abstraction layer (NAL) unit and a coded picture comprising a set of video coding layer (VCL) NAL units. A particular VCL NAL unit does not refer to the PPS NAL unit unless a value of a NAL unit header layer identifier (nuh_layer_id) of the PPS NAL unit is less than or equal to a value of a nuh_layer_id of the particular VCL NAL unit. The coded picture is decoded from the set of VCL NAL units based on the PPS in the PPS NAL unit to produce a decoded picture. The decoded picture is forwarded for display as part of a decoded video sequence.

Claims (24)

1. A method implemented by a decoder, the method comprising:

receiving a bitstream comprising a picture parameter set (PPS) in a PPS network abstraction layer (NAL) unit and a coded picture comprising a set of video coding layer (VCL) NAL units, wherein a particular VCL NAL unit does not refer to the PPS NAL unit unless a value of a NAL unit header layer identifier (nuh_layer_id) of the PPS NAL unit is less than or equal to a value of a nuh_layer_id of the particular VCL NAL unit, wherein the particular VCL NAL unit comprises a slice of the coded picture and a slice header containing data elements pertaining to the slice, wherein the slice header specifies an adaptation parameter set (APS) identifier (ID) of an APS, wherein the slice is associated with a picture header that contains syntax elements that apply to the slice of the coded picture, and wherein the picture header specifies a PPS ID of the PPS in use for the coded picture; and

decoding the coded picture from the set of VCL NAL units based on the PPS in the PPS NAL unit to produce a decoded picture.

2. The method of claim 1 , wherein the bitstream further comprises the APS in an APS NAL unit, and wherein the particular VCL NAL unit does not refer to the APS NAL unit unless a value of a nuh_layer_id of the APS NAL unit is less than or equal to the value of the nuh_layer_id of the particular VCL NAL unit.

3. The method of claim 1 , wherein the bitstream is a result of a sub-bitstream extraction process that removes NAL units from the bitstream based on nuh_layer_id value.

4. The method of claim 1 , further comprising:

receiving a second bitstream comprising a second PPS in a second PPS NAL unit and a second coded picture comprising a second set of VCL NAL units, wherein a second VCL NAL unit refers to the second PPS NAL unit with a value of a nuh_layer_id of the second PPS NAL unit is not less than or equal to a value of a nuh_layer_id of the second VCL NAL unit; and

in response to the receiving, taking some other corrective measures to ensure that a conforming bitstream corresponding to the second bitstream is received prior to decoding the coded picture from the second set of VCL NAL units.

5. The method of claim 1 , wherein the set of VCL NAL units are part of a layer when the set of VCL NAL units all have a particular value of nuh_layer_id.

6. The method of claim 5 , wherein the layer includes non-VCL NAL units that are associated with the set of VCL NAL units that all have a particular value of nuh_layer_id.

7. A method implemented by an encoder, the method comprising:

encoding a coded picture comprising a set of video coding layer (VCL) network abstraction layer (NAL) units in a bitstream;

encoding into the bitstream a picture parameter set (PPS) in a PPS NAL unit based on a constraint that a particular VCL NAL unit does not refer to the PPS NAL unit unless a value of a NAL unit header layer identifier (nuh_layer_id) of the PPS NAL unit is less than or equal to a value of a nuh_layer_id of the particular VCL NAL unit, wherein the particular VCL NAL unit comprises a slice of the coded picture and a slice header containing data elements pertaining to the slice, wherein the slice header specifies an adaptation parameter set (APS) identifier (ID) of an APS, wherein the slice is associated with a picture header that contains syntax elements that apply to the slice of the coded picture, and wherein the picture header specifies a PPS ID of the PPS in use for the coded picture.

8. The method of claim 7 , further comprising encoding into the bitstream the APS in an APS NAL unit based on a constraint that a particular VCL NAL unit does not refer to the APS NAL unit unless a value of a nuh_layer_id of the PPS NAL unit is less than or equal to a value of a nuh_layer_id of the particular VCL NAL unit.

9. The method of claim 7 , further comprising performing a sub-bitstream extraction process to remove NAL units from the bitstream based on nuh_layer_id value.

10. The method of claim 7 , wherein the set of VCL NAL units are part of a layer when the set of VCL NAL units all have a particular value of nuh_layer_id.

11. The method of claim 10 , wherein the layer includes non-VCL NAL units that are associated with the set of VCL NAL units that all have a particular value of nuh_layer_id.

12. A video coding device comprising:

a receiver configured to receive a bitstream comprising a picture parameter set (PPS) in a PPS network abstraction layer (NAL) unit and a coded picture comprising a set of video coding layer (VCL) NAL units, wherein a particular VCL NAL unit does not refer to the PPS NAL unit unless a value of a NAL unit header layer identifier (nuh_layer_id) of the PPS NAL unit is less than or equal to a value of a nuh_layer_id of the particular VCL NAL unit, wherein the particular VCL NAL unit comprises a slice of the coded picture and a slice header containing data elements pertaining to the slice, wherein the slice header specifies an adaptation parameter set (APS) identifier (ID) of an APS, wherein the slice is associated with a picture header that contains syntax elements that apply to the slice of the coded picture, and wherein the picture header specifies a PPS ID of the PPS in use for the coded picture; and

a processor coupled to the receiver and configured to decode the coded picture from the set of VCL NAL units based on the PPS in the PPS NAL unit to produce a decoded picture.

13. The video coding device of claim 12 , wherein the bitstream further comprises the APS in an APS NAL unit, and wherein the particular VCL NAL unit does not refer to the APS NAL unit unless a value of a nuh_layer_id of the APS NAL unit is less than or equal to the value of the nuh_layer_id of the particular VCL NAL unit.

14. The video coding device of claim 12 , wherein the bitstream is a result of a sub-bitstream extraction process that removes NAL units from the bitstream based on nuh_layer_id value.

15. The video coding device of claim 12 , wherein the set of VCL NAL units are part of a layer when the set of VCL NAL units all have a particular value of nuh_layer_id.

16. The video coding device of claim 15 , wherein the layer includes non-VCL NAL units that are associated with the set of VCL NAL units that all have a particular value of nuh_layer_id.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 24, 2024
From: FUTUREWEI TECHNOLOGIES, INC.
To: HUAWEI TECHNOLOGIES CO., LTD.
Reel/Frame 069236/0547 →
Continuity (3)
Continuation PCTUS2020051311 · Sep 17, 2020
Provisional Application 62905236 · Sep 24, 2019
Related Publication 20220217410A1 · Jul 7, 2022
References Cited (141)
US 7535383B2 · Segall et al. · 2009 [cited by applicant]
US 8830262B2 · Haskell et al. · 2014 [cited by applicant]
US 9154785B2 · Wang · 2015 [cited by applicant]
US 9479774B2 · Wang · 2016 [cited by applicant]
US 9654802B2 · Wang · 2017 [cited by applicant]
US 20060114993A1 · Xiong et al. · 2006 [cited by applicant]
US 20070086521A1 · Wang et al. · 2007 [cited by applicant]
US 20070121722A1 · Martinian et al. · 2007 [cited by applicant]
US 20080007438A1 · Segall et al. · 2008 [cited by applicant]
US 20080089412A1 · Ugur et al. · 2008 [cited by applicant]
US 20100098154A1 · Lou et al. · 2010 [cited by applicant]
US 20100142613A1 · Zhu · 2010 [cited by applicant]
US 20110064142A1 · Haskell et al. · 2011 [cited by applicant]
US 20110293091A1 · Macchetti · 2011 [cited by applicant]
US 20120023249A1 · Chen et al. · 2012 [cited by applicant]
US 20130077681A1 · Chen et al. · 2013 [cited by applicant]
US 20130077685A1 · Chen et al. · 2013 [cited by applicant]
US 20130266075A1 · Wang et al. · 2013 [cited by applicant]
US 20130272372A1 · Hannuksela · 2013 [cited by examiner]
US 20130279564A1 · Wang · 2013 [cited by applicant]
US 20130343465A1 · Chen · 2013 [cited by examiner]
US 20140003487A1 · Haque et al. · 2014 [cited by applicant]
US 20140003535A1 · Haque et al. · 2014 [cited by applicant]
US 20140016697A1 · Wang · 2014 [cited by applicant]
US 20140016707A1 · Wang · 2014 [cited by applicant]
US 20140022343A1 · Chen · 2014 [cited by applicant]
US 20140064142A1 · Mylarappa et al. · 2014 [cited by applicant]
US 20140064384A1 · Wang · 2014 [cited by applicant]
US 20140086303A1 · Wang · 2014 [cited by applicant]
US 20140086317A1 · Wang · 2014 [cited by applicant]
US 20140086331A1 · Wang · 2014 [cited by applicant]
US 20140086332A1 · Wang · 2014 [cited by applicant]
US 20140098894A1 · Wang · 2014 [cited by applicant]
US 20140133547A1 · Tanaka · 2014 [cited by examiner]
US 20140192149A1 · Wang et al. · 2014 [cited by applicant]
US 20140269896A1 · Dao · 2014 [cited by applicant]
US 20140355692A1 · Ramasubramonian et al. · 2014 [cited by applicant]
US 20150016504A1 · Auyeung et al. · 2015 [cited by applicant]
US 20150016547A1 · Tabatabai et al. · 2015 [cited by applicant]
US 20150103886A1 · He · 2015 [cited by examiner]
US 20150103926A1 · Hannuksela · 2015 [cited by applicant]
US 20150195545A1 · Wang et al. · 2015 [cited by applicant]
US 20150237372A1 · Choi et al. · 2015 [cited by applicant]
US 20150264366A1 · Hendry · 2015 [cited by examiner]
US 20150264404A1 · Hannuksela · 2015 [cited by applicant]
US 20150271498A1 · Wang et al. · 2015 [cited by applicant]
US 20150271528A1 · Wang et al. · 2015 [cited by applicant]
US 20150271529A1 · Wang et al. · 2015 [cited by applicant]
US 20150319462A1 · Ramasubramonian et al. · 2015 [cited by applicant]
US 20150373343A1 · Hendry et al. · 2015 [cited by applicant]
US 20160112724A1 · Hendry et al. · 2016 [cited by applicant]
US 20160165247A1 · Deshpande et al. · 2016 [cited by applicant]
US 20160191926A1 · Deshpande et al. · 2016 [cited by applicant]
US 20160255354A1 · Yamamoto et al. · 2016 [cited by applicant]
US 20160261877A1 · Wang · 2016 [cited by applicant]
US 20160301957A1 · McCarthy et al. · 2016 [cited by applicant]
US 20170085878A1 · Rojals et al. · 2017 [cited by applicant]
US 20170324981A1 · Deshpande · 2017 [cited by applicant]
US 20180295382A1 · Liu et al. · 2018 [cited by applicant]
US 20190052910A1 · Deshpande · 2019 [cited by applicant]
US 20190132612A1 · Deshpande · 2019 [cited by applicant]
US 20190243881A1 · Zia et al. · 2019 [cited by applicant]
US 20200389655A1 · Seregin · 2020 [cited by examiner]
US 20210067834A1 · Li et al. · 2021 [cited by applicant]
US 20210092426A1 · Choi · 2021 [cited by examiner]
US 20210297705A1 · Sjoberg et al. · 2021 [cited by applicant]
US 20210392333A1 · Paluri · 2021 [cited by examiner]
US 20210392381A1 · Wang · 2021 [cited by examiner]
US 20220060752A1 · Pettersson et al. · 2022 [cited by applicant]
US 20220132115A1 · Paluri · 2022 [cited by examiner]
US 20220210478A1 · Zhang · 2022 [cited by examiner]
US 20220217340A1 · Zhang · 2022 [cited by examiner]
US 20220217355A1 · Ouedraogo · 2022 [cited by examiner]
US 20220217375A1 · Wang · 2022 [cited by applicant]
US 20220217387A1 · Wang · 2022 [cited by applicant]
US 20220264106A1 · Zhang · 2022 [cited by examiner]
US 20220303582A1 · Sánchez De La Fuente · 2022 [cited by examiner]
CN 101317459A · 2008 [cited by applicant]
CN 101653002A · 2010 [cited by applicant]
CN 101658038A · 2010 [cited by applicant]
CN 104221387A · 2014 [cited by applicant]
CN 104412600A · 2015 [cited by applicant]
CN 104471943A · 2015 [cited by applicant]
CN 104685891A · 2015 [cited by applicant]
CN 105308971A · 2016 [cited by applicant]
CN 105706451A · 2016 [cited by applicant]
CN 105744295A · 2016 [cited by applicant]
CN 105981387A · 2016 [cited by applicant]
CN 106134200A · 2016 [cited by applicant]
CN 106170982A · 2016 [cited by applicant]
CN 108235006A · 2018 [cited by applicant]
CN 108650514A · 2018 [cited by applicant]
CN 109963176A · 2019 [cited by applicant]
JP 2007166129A · 2007 [cited by applicant]
JP 2015526970A · 2015 [cited by applicant]
JP 2015527813A · 2015 [cited by applicant]
JP 2015529437A · 2015 [cited by applicant]
JP 2015529438A · 2015 [cited by applicant]
JP 2015532551A · 2015 [cited by applicant]
JP 2016506695A · 2016 [cited by applicant]
JP 2016531451A · 2016 [cited by applicant]
JP 2017525215A · 2017 [cited by applicant]
JP 2017535184A · 2017 [cited by applicant]
RU 2544760C2 · 2015 [cited by applicant]
RU 2630181C2 · 2017 [cited by applicant]
RU 2758901C1 · 2021 [cited by applicant]
WO 2014046813A2 · 2014 [cited by applicant]
WO 2014047183A1 · 2014 [cited by applicant]
WO 2014107396A1 · 2014 [cited by applicant]
WO 2015054634A2 · 2015 [cited by applicant]
WO 2015056182A2 · 2015 [cited by applicant]
Guillaume Barroux et al., Proposed specification changes over “High Efficiency Video Coding (HEVC) Screen Content Coding: Draft 4” JCTVC-V0098_r2, Nov. 11, 2015, 35 pages. [cited by applicant]
Miska M. Hannuksela et al., MV-HEVC/SHVC HLS: On Temporalld constraints, JCTVC-Q0109, JCT3V-H0037, Mar. 18, 2014, 7 pages. [cited by applicant]
“Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video, High efficiency video coding,” ITU-T H. 265 Telecommunication Standardization Sector of ITU, Dec. 2016, 26 pa… [cited by applicant]
Document: JCTVC-Q1008_v5, Chen, J., et al., “High efficiency video coding (HEVC) scalable extension Draft 6,” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 17th Mee… [cited by applicant]
Document JVET-O2001-vE, Bross, B., et al., “Versatile Video Coding (Draft 6)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, 25 pages. [cited by applicant]
Document: JCTVC-V0098_r2, Barroux, G., et al., Proposed specification changes over “High Efficiency Video Coding (HEVC) Screen Content Coding: Draft 4”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 W… [cited by applicant]
Document: JCTVC-V0098_r2, Barroux, G., et al., Proposed specification changes over “High Efficiency Video Coding (HEVC) Screen Content Coding: Draft 4”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 W… [cited by applicant]
Chen, Y., et al., “The Emerging MVC Standard for 3D Video Services,” Hindawi Publishing Corporation, EURASIP Journal on Advances in Signal Processing, vol. 2009, Article ID 786015, Mar. 25, 2008, 13 pages. [cited by applicant]
Document JVET-P0203, Drugeon, V., et al., “AHG17: Parsing HRD related SEI messages independently from the SPS,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 16th Meeting: Geneva, CH… [cited by applicant]
“Line Transmission of Non-Telephone Signals Video Codec for Audiovisual Services at p×64 kbits,” ITU-T Recommendation H.261, Mar. 1993, 29 pages. [cited by applicant]
“Transmission of Non-Telephone Signals; Information Technology—Generic Coding of Moving Pictures and Associated Audio Information: Video,” ITU-T Recommendation H.262, Jul. 1995, 211 pages. [cited by applicant]
“Series H: Audiovisual and Multimedia Systems; Infrastructure of audiovisual services—Coding of moving video; Video coding for low bit rate communication,” ITU-T Recommendation H.263, Jan. 2005, 226 pages. [cited by applicant]
“Series H: Audiovisual and Multimedia Systems; Infrastructure of audiovisual services—Coding of moving video; Advanced video coding for generic audiovisual services,” Recommendation ITU-T H.264, Jun. 2019, 836 pages. [cited by applicant]
“Series H: Audiovisual and Multimedia Systems; Infrastructure of audiovisual services—Coding of moving video; High efficiency video coding” Recommendation ITU-T H.265, Apr. 2013, 317 pages. [cited by applicant]
Bross, B., et al., “Versatile Video Coding (Draft 6),” Document: JVET-O2001-vE, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, 455 pages. [cited by applicant]
Boyce, J., et al, “Overview of SHVC: Scalable Extensions of the High Efficiency Video Coding Standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, No. 1, Jan. 2016, 16 pages. [cited by applicant]
Document: JVET-S2001-vH, Bross, B., et al., “Versatile Video Coding (Draft 10),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 19th Meeting: by teleconference, Jun. 22-Jul. 1, 2020, … [cited by applicant]
Document: JVET-P0125-v2, Wang, Y.K., “AHG8/AHG17: Miscellaneous HLS topics,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and 1S0/IEC JTC 1/SC 29/WG 11, 16th Meeting: Geneva, CH, Oct. 1-11, 2019, 3 pages, XP03021… [cited by applicant]
Document: JVET-R0066-v1, Wang, Y.K., “AHG8/AHG9: On DPB memory allocation and derivation of NoOutputOfPriorPicsFlag,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG11, 18th Meeting: by te… [cited by applicant]
Document: JVET-P0190-v1, Wang, Y.K., “AHG8/AHG17: Scalable nesting SEI message,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and 1S0/IEC JTC 1/SC 29/WG 11, 16th Meeting: Geneva, CH, Oct. 1-11, 2019, 4 pages, XP0… [cited by applicant]
Document: JVET-P0203, Drugeon, V., “AHG17: Parsing HRD related SEI messages independently from the SPS,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IECJTC 1/SC 29/WG 11, 16th Meeting: Geneva, CH, Oct. 1… [cited by applicant]
Chen, J., et al., “Scalable HEVC (SHVC) Working Draft 2,” Video and JCT-VC, ISO/IEC JTC1/SC29/WG11 N13570, Apr. 2013, Incheon, South Korea, 66 pages, XP030020318. [cited by applicant]
Document: JVET-P2001-v9, Bross, B., et al., “Versatile Video Coding (Draft 7),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and 1S0/IEC JTC 1/SC 29/WG 11, 16th Meeting: Geneva, CH, Oct. 1-11, 2019, 491 pages, XP… [cited by applicant]
Document: JVET-S2001-vH, Bross, B., et al., “Versatile Video Coding (Draft 10),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/ SC 29/WG11, 19th Meeting: by teleconference, Jun. 22-Jul. 1, 2020, … [cited by applicant]
Document: JCTVC-R1013_v6 , Boyce, J., et al., “Draft high efficiency video coding (HEVC) version 2, combined format range extensions (RExt), scalability (SHVC), and multi-view (MV-HEVC) extensions,” Joint Collaborative … [cited by applicant]
Document: JCTVC-Q1008-v3, Chen, J., et al., “Preliminary version of High efficiency video coding (HEVC) scalable extension Draft 6,” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC … [cited by applicant]
Document: JVET-02001-v8, Bross, B., et al., “Versatile Video Coding (Draft 6),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/ SC 29/WG11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, 421 pages… [cited by applicant]
Document: JVET-S0177-v1, Wang, Y.K., et al., “AHG9: On the scalable nesting SEI message,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 19th Meeting: by teleconference, Jun. 22-Jul. … [cited by applicant]
Document: JVET-O2001-vA, Bross, B., et al., “Versatile Video Coding (Draft 6),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/ SC 29/WG11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, 432 pages… [cited by applicant]
Schwarz et al.: “Overview of the scalable video coding extension of the H. 264/AVC standard”, In: IEEE Transactions on circuits and systems for video technology, Sep. 2007, 18 pages. [cited by applicant]