IP Library Granted Patent US 12,341,968
Granted Patent B2
US 12,341,968 · App. 18/510,468 · Granted Jun 24, 2025

Method for mixed NAL unit type support in a coded picture

Inventors: Byeongdoo Choi (Palo Alto, CA); Stephan Wenger (Hillsborough, CA); Shan Liu (San Jose, CA)
Assignee: TENCENT AMERICA LLC
H04N19/136H04N19/174H04N19/46H04N19/70H04N19/91
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,341,968
App. No.
18/510,468
Granted
Jun 24, 2025
Kind
B2
Abstract

Methods and systems are provided for decoding at least one video stream. A method includes receiving a first network abstraction layer (NAL) unit of a first slice of a coded picture and a second VCL NAL unit of a second slice of the coded picture, the first VCL NAL unit having a first VCL NAL unit type and the second VCL NAL unit having a second VCL NAL unit type that is different from the first VCL NAL unit type, and decoding the coded picture, the decoding including determining a picture type of the coded picture based on the first VCL NAL unit type of the first VCL NAL unit and the second VCL NAL unit type of the second VCL NAL unit, or based on an indicator, received by the at least one processor, indicating that the coded picture includes mixed VCL NAL unit types.

Claims (47)

1. A method of video decoding performed by at least one processor, the method comprising:

receiving video data comprising a first video coding layer (VCL) network abstraction layer (NAL) unit of a first slice of a coded picture of the video data and a second VCL NAL unit of a second slice of the coded picture, the video data further comprising a mixed_nalu_types_in_pic_flag;

determining a value of the mixed_nalu_types_in_pic_flag and whether the coded picture is a step-wise temporal sub-layer access (STSA) picture based on a first VCL NAL unit type indicating the first VCL NAL unit contains an STSA picture coded slice, and a second VCL NAL unit type indicating the second VCL NAL unit does not contain an instantaneous decoding reference (IDR) picture coded slice; and

decoding the coded picture, the decoding comprising determining a nal_unit_type of the coded picture based on the value of the mixed_nalu_types_in_pic_flag,

wherein the value of the mixed_nalu_types_in_pic_flag being 0 indicates that a value of the nal_unit_type shall be the same for all coded slice NAL units of the coded picture.

2. The method of claim 1 ,

further comprising determining whether the coded picture is a trailing picture.

3. The method of claim 1 ,

further comprising determining whether the coded picture is a random access decodable leading (RADL) picture.

4. The method of claim 1 , wherein

the first VCL NAL unit indicates a gradual decoding refresh (GDR) picture coded slice.

5. The method of claim 1 , wherein

the second VCL NAL unit indicates an instantaneous decoding reference (IDR) picture coded slice.

6. The method of claim 1 , wherein

the second VCL NAL unit indicates a clean random access (CRA) picture coded slice.

7. The method of claim 1 , wherein,

the video data comprises the mixed_nalu_types_in_pic_flag in a picture header or a slice header,

the first VCL NAL unit indicates the STSA picture coded slice, and

the second VCL NAL unit indicates a clean random access (CRA) picture coded slice.

8. A method of video encoding performed by at least one processor, the method comprising:

obtaining video data comprising a picture to be encoded; and

determining a first video coding layer (VCL) network abstraction layer (NAL) unit of a first slice of the picture of the video data and a second VCL NAL unit of a second slice of the picture;

determining a value of mixed_nalu_types_in_pic_flag and whether the video data is a step-wise temporal sub-layer access (STSA) picture based on a first VCL NAL unit type indicating the first VCL NAL unit contains an STSA picture coded slice, and a second VCL NAL unit type indicating the second VCL NAL unit does not contain an instantaneous decoding reference (IDR) picture coded slice, wherein the value of the mixed_nalu_types_in_pic_flag being 0 indicates that a value of the nal_unit_type shall be the same for all coded slice NAL units of the coded picture; and

encoding the video data comprising the picture based at least on the determined value of mixed_nalu_types_in_pic_flag and whether the video data is the STSA picture.

9. The method of claim 8 , further comprising determining whether the picture is a trailing picture.

10. The method of claim 8 , further comprising determining whether the picture is a random access decodable leading (RADL) picture.

11. The method of claim 8 , wherein

the first VCL NAL unit indicates a gradual decoding refresh (GDR) picture coded slice.

12. The method of claim 8 , wherein

the second VCL NAL unit indicates an instantaneous decoding reference (IDR) picture coded slice.

13. The method of claim 8 , wherein

the second VCL NAL unit indicates a clean random access (CRA) picture coded slice.

14. The method of claim 8 , wherein,

the video data comprises the mixed_nalu_types_in_pic_flag in a picture header or a slice header,

the first VCL NAL unit indicates the STSA picture coded slice, and

the second VCL NAL unit indicates a clean random access (CRA) picture coded slice.

15. A method of processing visual media data, the method comprising:

performing a conversion between a visual media file and a bitstream of a visual media data according to a format rule, the bitstream of the visual media data comprising a first video coding layer (VCL) network abstraction layer (NAL) unit of a first slice of a coded picture of the visual media data and a second VCL NAL unit of a second slice of the coded picture, bitstream of the visual media data further comprising a mixed_nalu_types_in_pic_flag, and the format rule indicating that coding of the bitstream of the visual media data is based on whether the coded the picture is a step-wise temporal sub-layer access (STSA) picture based on (i) a first VCL NAL unit type indicating the first VCL NAL unit contains an STSA picture coded slice, and (ii) a second VCL NAL unit type indicating the second VCL NAL unit does not contain an instantaneous decoding reference (IDR) picture coded slice,

wherein the value of the mixed_nalu_types_in_pic_flag being 0 indicates that a value of the nal_unit_type shall be the same for all coded slice NAL units of the coded picture.

16. The method of claim 15 , wherein the format rule further indicates to determine whether the coded picture is a trailing picture.

17. The method of claim 15 , wherein the format rule further indicates to determine whether the coded picture is a random access decodable leading (RADL) picture.

18. The method of claim 15 , wherein the first VCL NAL unit indicates a gradual decoding refresh (GDR) picture coded slice.

19. The method of claim 15 , wherein the second VCL NAL unit indicates an instantaneous decoding reference (IDR) picture coded slice.

20. The method of claim 15 , wherein the second VCL NAL unit indicates a clean random access (CRA) picture coded slice.

21. The method of claim 15 , wherein the video data comprises the mixed_nalu_types_in_pic_flag in a picture header or a slice header,

the first VCL NAL unit indicates the STSA picture coded slice, and

the second VCL NAL unit indicates a clean random access (CRA) picture coded slice.

Continuity (4)
Continuation 17851467 · Jun 28, 2022
Continuation 17077035 · Oct 22, 2020
Provisional Application 62956254 · Jan 1, 2020
Related Publication 20240098269A1 · Mar 21, 2024
References Cited (39)
US 9584804B2 · Wang · 2017 [cited by examiner]
US 9648322B2 · Wang · 2017 [cited by examiner]
US 10999590B2 · Choi · 2021 [cited by examiner]
US 11032546B1 · Krishnan · 2021 [cited by examiner]
US 11057636B2 · Huang · 2021 [cited by examiner]
US 20140016697A1 · Wang · 2014 [cited by examiner]
US 20140016707A1 · Wang · 2014 [cited by examiner]
US 20150016546A1 · Wang et al. · 2015 [cited by applicant]
US 20150085917A1 · Hendry et al. · 2015 [cited by applicant]
US 20160234517A1 · Samuelsson et al. · 2016 [cited by applicant]
US 20200092577A1 · Huang · 2020 [cited by examiner]
US 20200092578A1 · Huang · 2020 [cited by examiner]
US 20200169748A1 · Chen · 2020 [cited by examiner]
US 20200186798A1 · Shlyakhov · 2020 [cited by examiner]
US 20200280736A1 · Wang · 2020 [cited by examiner]
US 20200381022A1 · Ilola · 2020 [cited by examiner]
US 20210092359A1 · Nassor · 2021 [cited by examiner]
US 20210092450A1 · Choi · 2021 [cited by examiner]
US 20210105492A1 · Aksu · 2021 [cited by examiner]
US 20210136396A1 · Deshpande · 2021 [cited by examiner]
US 20210194946A1 · Hannuksela · 2021 [cited by examiner]
US 20210250617A1 · Hannuksela · 2021 [cited by examiner]
US 20210281858A1 · Hannuksela · 2021 [cited by examiner]
US 20210297705A1 · Sjöberg · 2021 [cited by examiner]
US 20230045490A1 · Hendry · 2023 [cited by examiner]
WO 2020008115A1 · 2020 [cited by applicant]
WO 2020185922A1 · 2020 [cited by applicant]
Office Action issued May 21, 2024 in Australian Application No. 2023203749. [cited by applicant]
Benjamin Bross et al., “Versatile Video Coding (Draft 6)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting, JVET-O2001-vE, Jul. 3-12, 2019, 455 pages, Gothenburg, SE. [cited by applicant]
Benjamin Bross et al., “Versatile Video Coding (Draft 7)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 16th Meeting, JVET-P2001-vE, Oct. 1-11, 2019, 492 pages, Geneva, CH. [cited by applicant]
International Search Report dated Mar. 8, 2021, in International Application No. PCT/US2020/065318. [cited by applicant]
Miska M. Hannuksela, “AHG9: On Miixed NAL unit types in a coded picture”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Document: JVET-Q0239, 2020, pp. 1-4 (6 pages total). [cited by applicant]
Notification of reasons for refusal dated Oct. 11, 2022 from the Japanese Patent Office in Japanese Application No. 2021-559109. [cited by applicant]
Robert Skupin et al., “AHG9: On mixing NAL unit types in a coded picture”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Document: JVET:Q0396, 2020, pp. 1-6 (9 pages total). [cited by applicant]
Written Opinion dated Mar. 8, 2021, in International Application No. PCT/US2020/065318. [cited by applicant]
Ye-Kui Wang, et al., “AHG12: On mixed NAL unit types within a picture”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP3 and ISO/IEC JTC1/SC 29/WG 11,JVET-00140-v2, Gothensburg, SE, Jul. 3-12, 2019 (3 pages). [cited by applicant]
Miska M. Hannuksela, et al., “AHG12/AHG17: On signalling of picure-specific syntax elements in access unit delimiter”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-P0095, 16th … [cited by applicant]
Byeongdoo Choi, et al., “AHG9: On mixed NAL unit type support”, Joint Video Experts Team(JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-Q0284, 17th Meeting: Brussels, BE, Jan. 7-17, 2020 (7 pages). [cited by applicant]
Extended European Search Report dated Dec. 22, 2023 in Application No. 20909887.0. [cited by applicant]