IP Library › Granted Patent US 12,407,840
Granted Patent B2
US 12,407,840 · App. 18/677,600 · Granted Sep 2, 2025

Mixed NAL unit picture constraints in video coding

Inventors: Ye-Kui Wang (San Diego, CA); F N U Hendry (San Diego, CA)
Assignee: Huawei Technologies Co., Ltd.
H04N19/188H04N19/105H04N19/107H04N19/172H04N19/174H04N19/70H04N19/117
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,407,840
App. No.
18/677,600
Granted
Sep 2, 2025
Kind
B2
Abstract

A video coding mechanism is disclosed. The mechanism includes receiving a bitstream comprising a current picture including a plurality of video coding layer (VCL) network abstraction layer (NAL) units that do not have a same NAL unit type. Active entries of reference picture lists are obtained for slices positioned in a sub-picture A (subpicA) in subsequent pictures following the current picture in decoding order. The active entries do not refer to any reference picture preceding the current picture in decoding order when the subpicA at the current picture is associated with an intra-random access point (IRAP) NAL unit type. The subsequent pictures are decoded based on the reference picture list active entries. The subsequent pictures are forwarded for display as part of a decoded video sequence.

Claims (29)

1. A method for storing a coded video bitstream, the method comprising:

obtaining the coded video bitstream comprising a current picture that has more than one video coding layer (VCL) network abstraction layer (NAL) units that do not have a same NAL unit type;

obtaining active entries of reference picture lists for slices positioned in a sub-picture A (subpicA) in subsequent pictures following the current picture in decoding order, wherein the active entries do not include a reference to any reference picture preceding the current picture in decoding order when the subpicA at the current picture is associated with an intra-random access point (IRAP) NAL unit type, wherein all slices of the current picture positioned in subpicA are associated with the same NAL unit type; and

storing the coded video bitstream in at least one storage medium based on the reference picture list active entries.

2. The method of claim 1 , wherein the IRAP NAL unit type is a clean random access (CRA) NAL unit type.

3. The method of claim 1 , wherein the IRAP NAL unit type is an instantaneous decoder refresh (IDR) NAL unit type.

4. The method of claim 1 , wherein a flag in the coded video bitstream indicates a first NAL unit type value for the VCL NAL units of the current picture is different than a second NAL unit type value for VCL NAL units of the current picture.

5. The method of claim 4 , wherein the coded video bitstream includes a picture parameter set (PPS), and wherein the flag is obtained from the PPS.

6. The method of claim 5 , wherein the flag is a mixed_nalu_types_in_pic_flag, and wherein the mixed_nalu_types_in_pic_flag is equal to one when specifying that each picture referring to the PPS has more than one VCL NAL unit and the VCL NAL units do not have the same value of NAL unit type (nal_unit_type).

7. A video coding device comprising:

a processor configured to:

determine that a current picture includes a plurality of video coding layer (VCL) network abstraction layer (NAL) units that do not have a same NAL unit type;

determine that a sub-picture A (subpicA) at the current picture is associated with an intra-random access point (IRAP) NAL unit type;

generate active entries of reference picture lists for slices positioned in the subpicA in subsequent pictures following the current picture in decoding order, wherein the active entries do not include a reference to any reference picture preceding the current picture in decoding order when the subpicA at the current picture is associated with the IRAP NAL unit type, wherein all slices of the current picture positioned in subpicA are associated with the same NAL unit type; and

encode the subsequent pictures into a bitstream based on the reference picture lists.

8. The video coding device of claim 7 , wherein the IRAP NAL unit type is a clean random access (CRA) NAL unit type.

9. The video coding device of claim 7 , wherein the IRAP NAL unit type is an instantaneous decoder refresh (IDR) NAL unit type.

10. The video coding device of claim 7 , wherein the processor is further configured to encode into the bitstream a flag indicating a first NAL unit type value for the VCL NAL units of the current picture is different than a second NAL unit type value for VCL NAL units of the current picture.

11. The video coding device of claim 10 , wherein the flag is encoded into a picture parameter set (PPS) in the bitstream.

12. The video coding device of claim 11 , wherein the flag is a mixed_nalu_types_in_pic_flag, and wherein the mixed_nalu_types_in_pic_flag is set equal to one when specifying that each picture referring to the PPS has more than one VCL NAL unit and the VCL NAL units do not have the same value of NAL unit type (nal_unit_type).

13. A non-transitory computer readable medium comprising a computer program product for use by a video coding device, the computer program product comprising computer executable instructions stored on the non-transitory computer readable medium such that when executed by a processor cause the video coding device to:

receive a bitstream comprising a current picture that has more than one video coding layer (VCL) network abstraction layer (NAL) units that do not have a same NAL unit type;

obtain active entries of reference picture lists for slices positioned in a sub-picture A (subpicA) in subsequent pictures following the current picture in decoding order, wherein the active entries do not include a reference to any reference picture preceding the current picture in decoding order when the subpicA at the current picture is associated with an intra-random access point (IRAP) NAL unit type, wherein all slices of the current picture positioned in subpicA are associated with the same NAL unit type; and

decode the subsequent pictures based on the reference picture list active entries.

14. The non-transitory computer readable medium of claim 13 , wherein the IRAP NAL unit type is a clean random access (CRA) NAL unit type.

15. The non-transitory computer readable medium of claim 13 , wherein the IRAP NAL unit type is an instantaneous decoder refresh (IDR) NAL unit type.

16. The non-transitory computer readable medium of claim 13 , wherein the executable instructions further cause the video coding device to determine a first NAL unit type value for the VCL NAL units of the current picture is different than a second NAL unit type value for VCL NAL units of the current picture based on a flag.

17. The non-transitory computer readable medium of claim 16 , wherein the bitstream includes a picture parameter set (PPS), and wherein the flag is obtained from the PPS.

18. The non-transitory computer readable medium of claim 17 , wherein the flag is a mixed_nalu_types_in_pic_flag, and wherein the mixed_nalu_types_in_pic_flag is equal to one when specifying that each picture referring to the PPS has more than one VCL NAL unit and the VCL NAL units do not have the same value of NAL unit type (nal_unit_type).

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 24, 2025
From: FUTUREWEI TECHNOLOGIES, INC.
To: HUAWEI TECHNOLOGIES CO., LTD.
Reel/Frame 071714/0201 →
Continuity (4)
Continuation 17568517 · Jan 4, 2022
Continuation PCTUS2020041035 · Jul 7, 2020
Provisional Application 62871524 · Jul 8, 2019
Related Publication 20240422331A1 · Dec 19, 2024
References Cited (39)
US 10645404B2 · Wang et al. · 2020 [cited by applicant]
US 20090323820A1 · Wu et al. · 2009 [cited by applicant]
US 20140301485A1 · Ramasubramonian et al. · 2014 [cited by applicant]
US 20140355692A1 · Ramasubramonian · 2014 [cited by applicant]
US 20150110192A1 · Wang et al. · 2015 [cited by applicant]
US 20150195572A1 · Chen · 2015 [cited by applicant]
US 20150264404A1 · Hannuksela · 2015 [cited by applicant]
US 20160219306A1 · Pettersson et al. · 2016 [cited by applicant]
US 20170085905A1 · Kadono · 2017 [cited by applicant]
US 20170094271A1 · Liu · 2017 [cited by applicant]
US 20170105004A1 · Chen et al. · 2017 [cited by applicant]
US 20180277164A1 · Wang · 2018 [cited by applicant]
US 20190014337A1 · Skupin · 2019 [cited by examiner]
CN 103703782A · 2014 [cited by applicant]
CN 105122815A · 2015 [cited by applicant]
CN 105379271A · 2016 [cited by applicant]
JP 2023512749A · 2023 [cited by applicant]
JP 7453329B2 · 2024 [cited by applicant]
RU 2676876C2 · 2019 [cited by applicant]
WO 2011129602A2 · 2011 [cited by applicant]
WO 2013157828A1 · 2013 [cited by applicant]
WO 2018071666A1 · 2018 [cited by applicant]
WO 2021116157A1 · 2021 [cited by applicant]
Bross et al., Versatile Video Coding (Draft 5), Mar. 19-27, 2019, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11,14th Meeting: Geneva, CH. (Year: 2019). [cited by examiner]
Document: JVET-N0108-v1, Wang, Y-K., et al, “AHG12: Allowing mixed IRAP and non-IRAP NAL unit types within a picture,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IECJTC 1/SC 29/WG11, 14th Meeting: Genev… [cited by applicant]
Zhong, J., “The characteristics of the new video coding standard H.264 are analyzed,” National Center for Science and Technology Books and Literature, 2005, with an English abstract, 4 pages. [cited by applicant]
Document: JVET-N0047-v1, Hannuksela, M., “AHG12/AHG17: Merging IRAP and non-IRAP VCL NAL units into the same coded picture,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 14th Meeting… [cited by applicant]
Document: JVET-M0128-v1, Wang, Y.K., et al., “On reference picture management for VVC,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, 5 p… [cited by applicant]
Document: JVET-M0259, Miska M. Hannuksela et al, “Use cases and proposed design choices for adaptive resolution changing (ARC),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 13th Me… [cited by applicant]
Document: JVET-N0047, Miska M. Hannuksela et al., “AHG12/AHG17: Merging IRAP and non-IRAP VCL NAL units into the same coded picture,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14… [cited by applicant]
High Efficiency Video Coding, Series H: Audiovisual and Multimedia Systems Infrastructure of Audiovisual Services—Coding of Moving Video, ITU-T H.265, Feb. 2018, 692 pages. [cited by applicant]
Line Transmission of Non-Telephone Signals, Video Codec for Audiovisual Services at p x 64 kbits, ITU-T Recommendation H.261, Mar. 1993, 29 pages. [cited by applicant]
Transmission of Non-Telephone Signals, Information Technology—Generic Coding of Moving Pictures and I\Associated Audio Information: Video, ITU-T H.262, Jul. 1995, 211 pages. [cited by applicant]
Video Coding for Low Bit Rate Communication, Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual Services—Coding of Moving Video, ITU-T H.263, Jan. 2005, 226 pages. [cited by applicant]
Bross, et al., “Versatile Video Coding (Draft 5),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, Mar. 19-27, 2019, JVET-N1001-v10, 407 pages. [cited by applicant]
Boyce, et al., “HEVC Additional Supplemental Enhancement Information (Draft 4),” Joint Collaborative Team on Video coding (JCT-VG) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG11, 29th Meeting: Macao, CN, Oct. 19-25, 2… [cited by applicant]
Nang, et al., “AHG12: On Mixed NAL Unit Types Within a Picture,” Joint Video Experts Team (JVET) of ITU-T-SG 16 MP3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, JVET-O0140-v1, 3 pages. [cited by applicant]
Wang, et al., “AHG12: On Mixed NAL Unit Types Within a Picture,” Joint Video Experts Team (JVET) of ITU-T-SG 16 MP3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, JVET-00140-v1, 3 pages. [cited by applicant]
ITU-T H.264, Advanced Video Coding for Generic Audiovisual Services, Series H: Audiovisual and Multimedia Systems infrastructure of Audiovisual Services—Coding of Moving Video, Jun. 2019, 836 pages. [cited by applicant]