IP Library › Granted Patent US 12,355,998

Granted Patent B2

US 12,355,998 · App. 18/209,169 · Granted Jul 8, 2025

Method for layerwise random access in a coded video stream

Inventors: Byeongdoo Choi (Palo Alto, CA); Shan Liu (San Jose, CA); Stephan Wenger (Hillsborough, CA)

Assignee: TENCENT AMERICA LLC

H04N19/46 H04N19/129 H04N19/159 H04N19/172 H04N19/188

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12,355,998
App. No.: 18/209,169
Granted: Jul 8, 2025
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

A method of generating an encoded video bitstream, including obtaining an IRAP AU from a video bitstream; determining whether the IRAP AU is a first AU in a decoding order, whether each picture is an IDR picture, or whether each picture is a first picture of a layer following an EOS NAL unit; if so, setting a first flag for the IRAP AU equal to one; if not, determining whether a second flag for the IRAP AU is externally set to an external value; if so, setting the first flag for the IRAP AU equal to the external value; and if not, setting the first flag for the IRAP AU and the second flag for the IRAP AU equal to zero; encoding the video bitstream based on the first flag and the second flag; and transmitting the encoded video bitstream.

Claims (109)

1. A method of decoding an encoded video bitstream using at least one processor, the method comprising:

receiving the encoded video bitstream;

obtaining, from the encoded video bitstream, an intra random access point (IRAP) access unit (AU);

determining whether the IRAP AU is a first AU in a decoding order, whether each picture included in the IRAP AU is an instantaneous decoder refresh (IDR) picture, and whether each picture included in the IRAP AU is a first picture of a layer that follows an end of sequence (EOS) network abstraction layer (NAL) unit in the decoding order;

when the IRAP AU is the first AU in the decoding order of the encoded video bitstream, the each picture included in the IRAP AU is the IDR picture, or the each picture included in the IRAP AU is the first picture of the layer that follows the EOS NAL unit in the decoding order, setting a first flag for the IRAP AU equal to one;

when the IRAP AU is not the first AU in the decoding order, the each picture included in the IRAP AU is not the IDR picture, and the each picture included in the IRAP AU is not the first picture of the layer that follows the EOS NAL unit in the decoding order, determining whether a second flag for the IRAP AU is externally set to an external value;

when the IRAP AU is not the first AU in the decoding order, the each picture included in the IRAP AU is not the IDR picture, the each picture included in the IRAP AU is not the first picture of the layer that follows the EOS NAL unit in the decoding order, and the second flag for the IRAP AU is externally set to the external value, setting a value of the first flag for the IRAP AU equal to the external value;

when the IRAP AU is not the first AU in the decoding order, the each picture included in the IRAP AU is not the IDR picture, the each picture included in the IRAP AU is not the first picture of the layer that follows the EOS NAL unit in the decoding order, and the second flag for the IRAP AU is not externally set to the external value, setting the value of the first flag for the IRAP AU equal to zero and setting the second flag for the IRAP AU equal to zero; and

decoding each picture in the IRAP AU based on the first flag for the IRAP AU and the second flag for the IRAP AU.

2. The method of claim 1 , wherein the first flag for the IRAP AU being set equal to one indicates that all random access skipped leading (RASL) pictures associated with a clean random access (CRA) picture included in the IRAP AU should be discarded without being decoded, and

wherein the second flag for the IRAP AU being set equal to one indicates that all leading pictures associated with the CRA picture included in the IRAP AU should be discarded without being decoded.

3. The method of claim 1 , wherein the first flag for the IRAP AU comprises a NoIncorrectPicOutputFlag flag, and

wherein the second flag for the IRAP AU comprises a HandleCraAsCvsStartFlag flag.

4. The method of claim 1 , further comprising:

obtaining a non-IRAP AU from the encoded video bitstream;

obtaining an IRAP picture unit (PU) from the non-IRAP AU;

determining whether a picture corresponding to the IRAP PU is the IDR picture, and whether the picture is the first picture of the layer that follows the EOS NAL unit in the decoding order;

when the picture is the IDR picture or the picture is the first picture of the layer that follows the EOS NAL unit in the decoding order, setting a first flag for the IRAP PU equal to one;

when the picture is not the IDR picture, and the picture is not the first picture of the layer that follows the EOS NAL unit in the decoding order, determining whether a second flag for the IRAP PU is externally set to the external value;

when the picture is not the IDR picture, the picture is not the first picture of the layer that follows the EOS NAL unit in the decoding order, and the second flag for the IRAP PU is externally set to the external value, setting the first flag for the IRAP PU to the external value; and

when the picture is not the IDR picture, the picture is not the first picture of the layer that follows the EOS NAL unit in the decoding order, and the second flag for the IRAP PU is not externally set to the external value, setting the first flag for the IRAP PU equal to zero and setting the second flag for the IRAP PU equal to zero,

wherein each picture in the IRAP PU is decoded based on the first flag for the IRAP PU and the second flag for the IRAP PU.

5. The method of claim 1 , further comprising:

obtaining a gradual decoding refresh (GDR) AU from the encoded video bitstream;

determining whether the GDR AU is the first AU in the decoding order, and whether each picture included in the GDR AU is the first picture of the layer that follows the EOS NAL unit in the decoding order;

when the GDR AU is the first AU in the decoding order of the video bitstream, or the each picture included in the GDR AU is the first picture of the layer that follows the EOS NAL unit in the decoding order, setting a first flag for the GDR AU equal to one;

when the GDR AU is not the first AU in the decoding order and the each picture included in the GDR AU is not the first picture of the layer that follows the EOS NAL unit in the decoding order, determining whether a second flag for the GDR AU is externally set to the external value;

when the GDR AU is not the first AU in the decoding order, the each picture included in the GDR AU is not the first picture of the layer that follows the EOS NAL unit in the decoding order, and the second flag for the GDR AU is externally set to the external value, setting the first flag for the GDR AU equal to the external value; and

when the GDR AU is not the first AU in the decoding order, the each picture included in the GDR AU is not the first picture of the layer that follows the EOS NAL unit in the decoding order, and the second flag is not externally set to the external value, setting the first flag for the GDR AU equal to zero and setting the second flag for the GDR AU equal to zero,

wherein each picture in the GDR AU is decoded based on the first flag for the GDR AU and the second flag for the GDR AU.

6. The method of claim 5 , wherein the first flag for the GDR AU being set equal to one indicates that all random access skipped leading (RASL) pictures associated with a GDR picture included in the GDR AU should be discarded without being decoded, and

wherein the second flag for the GDR AU being set equal to one indicates that all leading pictures associated with the GDR picture included in the GDR AU should be discarded without being decoded.

7. The method of claim 1 , further comprising:

setting the first flag for the IRAP AU equal to the external value based on determining that the second flag for the IRAP AU is externally set to the external value; and

setting the first flag for the IRAP AU equal to zero and setting the second flag for the IRAP AU equal to zero based on determining that the second flag is not externally set to the external value.

8. The method of claim 5 , further comprising:

obtaining a non-IRAP AU from the video bitstream;

obtaining a GDR picture unit (PU) from the non-IRAP AU;

determining whether a picture corresponding to the GDR PU is the first picture of the layer that follows the EOS NAL unit in the decoding order;

when the picture is the first picture of the layer that follows the EOS NAL unit in the decoding order, setting a first flag for the GDR PU equal to one;

when the picture is not the first picture of the layer that follows the EOS NAL unit in the decoding order, determining whether a second flag for the GDR PU is externally set to the external value;

when the picture is not the first picture of the layer that follows the EOS NAL unit in the decoding order, and the second flag for the GDR PU is externally set to the external value, setting the first flag for the GDR PU to the external value; and

when the picture is not the first picture of the layer that follows the EOS NAL unit in the decoding order, and the second flag for the GDR PU is not externally set to the external value, setting the first flag for the GDR PU equal to zero and setting the second flag for the GDR PU equal to zero,

wherein each picture in the GDR PU is decoded based on the first flag for the GDR PU and the second flag for the GDR PU.

9. A method of processing video data, the method comprising:

processing, by a decoder, the video data in a bitstream, wherein the bitstream includes an intra random access point (IRAP) access unit (AU); and

a coding of the video data in the bitstream causes the decoder to:

determine whether the IRAP AU is a first AU in a decoding order, whether each picture included in the IRAP AU is an instantaneous decoder refresh (IDR) picture, and whether each picture included in the IRAP AU is a first picture of a layer that follows an end of sequence (EOS) network abstraction layer (NAL) unit in the decoding order;

when the IRAP AU is the first AU in the decoding order of the bitstream, the each picture included in the IRAP AU is the IDR picture, or the each picture included in the IRAP AU is the first picture of the layer that follows the EOS NAL unit in the decoding order, set a first flag for the IRAP AU equal to one;

when the IRAP AU is not the first AU in the decoding order, the each picture included in the IRAP AU is not the IDR picture, and the each picture included in the IRAP AU is not the first picture of the layer that follows the EOS NAL unit in the decoding order, determine whether a second flag for the IRAP AU is externally set to an external value;

when the IRAP AU is not the first AU in the decoding order, the each picture included in the IRAP AU is not the IDR picture, the each picture included in the IRAP AU is not the first picture of the layer that follows the EOS NAL unit in the decoding order, and the second flag for the IRAP AU is externally set to the external value, set a value of the first flag for the IRAP AU equal to the external value;

when the IRAP AU is not the first AU in the decoding order, the each picture included in the IRAP AU is not the IDR picture, the each picture included in the IRAP AU is not the first picture of the layer that follows the EOS NAL unit in the decoding order, and the second flag for the IRAP AU is not externally set to the external value, set the value of the first flag for the IRAP AU equal to zero and the second flag for the IRAP AU equal to zero; and

process each picture in the IRAP AU based on the first flag for the IRAP AU and the second flag for the IRAP AU.

10. The method of claim 9 , wherein the first flag for the IRAP AU being set equal to one indicates that all random access skipped leading (RASL) pictures associated with a clean random access (CRA) picture included in the IRAP AU should be discarded without being decoded, and

wherein the second flag for the IRAP AU being set equal to one indicates that all leading pictures associated with the CRA picture included in the IRAP AU should be discarded without being decoded.

11. The method of claim 9 , wherein the first flag for the IRAP AU comprises a NoIncorrectPicOutputFlag flag, and

wherein the second flag for the IRAP AU comprises a HandleCraAsCvsStartFlag flag.

12. The method of claim 9 , wherein

the bitstream includes a non-IRAP AU; and

the coding specifies that:

an IRAP picture unit (PU) is obtained from the non-IRAP AU;

whether a picture corresponding to the IRAP PU is the IDR picture, and whether the picture is the first picture of the layer that follows the EOS NAL unit in the decoding order is determined;

when the picture is the IDR picture or the picture is the first picture of the layer that follows the EOS NAL unit in the decoding order, a first flag for the IRAP PU is set equal to one;

when the picture is not the IDR picture, and the picture is not the first picture of the layer that follows the EOS NAL unit in the decoding order, whether a second flag for the IRAP PU is externally set to the external value is determined;

when the picture is not the IDR picture, the picture is not the first picture of the layer that follows the EOS NAL unit in the decoding order, and the second flag for the IRAP PU is externally set to the external value, the first flag for the IRAP PU is set to the external value;

when the picture is not the IDR picture, the picture is not the first picture of the layer that follows the EOS NAL unit in the decoding order, and the second flag for the IRAP PU is not externally set to the external value, the first flag for the IRAP PU is set equal to zero and the second flag for the IRAP PU is set equal to zero; and

each picture in the IRAP PU is processed based on the first flag for the IRAP PU and the second flag for the IRAP PU.

13. A method of generating an encoded video bitstream using at least one processor, the method comprising:

obtaining an intra random access point (IRAP) access unit (AU) from a video bitstream;

determining whether the IRAP AU is a first AU in a decoding order, whether each picture included in the IRAP AU is an instantaneous decoder refresh (IDR) picture, and whether the each picture included in the IRAP AU is a first picture of a layer that follows an end of sequence (EOS) network abstraction layer (NAL) unit in the decoding order;

when the IRAP AU is the first AU in the decoding order of the video bitstream, the each picture included in the IRAP AU is the IDR picture, or the each picture included in the IRAP AU is the first picture of the layer that follows the EOS NAL unit in the decoding order, setting a first flag for the IRAP AU equal to one;

when the IRAP AU is not the first AU in the decoding order, the each picture included in the IRAP AU is not the IDR picture, the each picture included in the IRAP AU is not the first picture of the layer that follows the EOS NAL unit in the decoding order, and the second flag for the IRAP AU is externally set to the external value, setting the first flag for the IRAP AU equal to the external value;

when the IRAP AU is not the first AU in the decoding order, the each picture included in the IRAP AU is not the IDR picture, the each picture included in the IRAP AU is not the first picture of the layer that follows the EOS NAL unit in the decoding order, and the second flag is not externally set to the external value, setting the first flag for the IRAP AU equal to zero and setting the second flag for the IRAP AU equal to zero; and

encoding the video bitstream based on the first flag for the IRAP AU and the second flag for the IRAP AU.

14. The method of claim 13 , wherein the first flag for the IRAP AU being set equal to one indicates that all random access skipped leading (RASL) pictures associated with a clean random access (CRA) picture included in the IRAP AU should be discarded without being decoded, and

wherein the second flag for the IRAP AU being set equal to one indicates that all leading pictures associated with the CRA picture included in the IRAP AU should be discarded without being decoded.

15. The method of claim 13 , wherein the first flag for the IRAP AU comprises a NoIncorrectPicOutputFlag flag, and

wherein the second flag for the IRAP AU comprises a HandleCraAsCvsStartFlag flag.

16. The method of claim 13 , further comprising:

obtaining a non-IRAP AU from the video bitstream;

obtaining an IRAP picture unit (PU) from the non-IRAP AU;

determining whether a picture corresponding to the IRAP PU is the IDR picture, and whether the picture is the first picture of the layer that follows the EOS NAL unit in the decoding order;

when the picture is the IDR picture or the picture is the first picture of the layer that follows the EOS NAL unit in the decoding order, setting a first flag for the IRAP PU equal to one;

wherein the video bitstream is further encoded based on the first flag for the IRAP PU and the second flag for the IRAP PU.

17. The method of claim 13 , further comprising:

obtaining a gradual decoding refresh (GDR) AU from the video bitstream;

when the GDR AU is not the first AU in the decoding order, the each picture included in the GDR AU is not the first picture of the layer that follows the EOS NAL unit in the decoding order, and the second flag is not externally set to the external value, setting the first flag for the GDR AU equal to zero and setting the second flag for the GDR AU equal to zero,

wherein the video bitstream is further encoded based on the first flag for the GDR AU and the second flag for the GDR AU.

18. The method of claim 17 , wherein the first flag for the GDR AU being set equal to one indicates that all random access skipped leading (RASL) pictures associated with a GDR picture included in the GDR AU should be discarded without being decoded, and

wherein the second flag for the GDR AU being set equal to one indicates that all leading pictures associated with the GDR picture included in the GDR AU should be discarded without being decoded.

19. The method of claim 17 , wherein the first flag for the GDR AU comprises a NoIncorrectPicOutputFlag flag, and

wherein the second flag for the GDR AU comprises a HandleGdrAsCvsStartFlag flag.

20. The method of claim 17 , further comprising:

obtaining a non-IRAP AU from the video bitstream;

obtaining a GDR picture unit (PU) from the non-IRAP AU;

determining whether a picture corresponding to the GDR PU is the first picture of the layer that follows the EOS NAL unit in the decoding order;

when the picture is the first picture of the layer that follows the EOS NAL unit in the decoding order, setting a first flag for the GDR PU equal to one;

when the picture is not the first picture of the layer that follows the EOS NAL unit in the decoding order, determining whether a second flag for the GDR PU is externally set to the external value;

wherein the video bitstream is further encoded based on the first flag for the GDR PU and the second flag for the GDR PU.

Assignments (1)

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 18, 2024

From: CHOI, BYEONGDOO; LIU, SHAN; WENGER, STEPHAN

To: TENCENT AMERICA LLC

Reel/Frame 068018/0162 →

Continuity (4)

Continuation 17546780 · Dec 9, 2021

Continuation 17063082 · Oct 5, 2020

Provisional Application 62954880 · Dec 30, 2019

Related Publication 20230328265A1 · Oct 12, 2023

References Cited (20)

US 20140301437A1 · Wang · 2014 [cited by examiner]

US 20140301485A1 · Ramasubramonian · 2014 [cited by examiner]

US 20150271506A1 · Hendry · 2015 [cited by examiner]

US 20160088306A1 · Sjoberg et al. · 2016 [cited by applicant]

US 20170105004A1 · Chen et al. · 2017 [cited by applicant]

US 20220217369A1 · Wang · 2022 [cited by examiner]

US 20220217386A1 · Wang · 2022 [cited by examiner]

JP 2017225159A · 2017 [cited by applicant]

WO WO2016034058A1 · 2016 [cited by examiner]

WO WO2020058893A1 · 2020 [cited by examiner]

Bross et al., (hereinafter Bross): Versatile Video Coding—Draft 7-: JVET-P2001-vE; Geneva, CH, Oct. 2019 (Year: 2019). [cited by examiner]

Ye-Kui Wang, “AHG8: Scalability—general and output layer sets”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-P0115-v2, 16th Meeting, Geneva, CH, Oct. 1-11, 2019 (6 pages). [cited by applicant]

Benjamin Bross et al., “Versatile Video Coding (Draft 5)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 2019, JVET-N1001-v10, 14th Meeting: Geneva. CH, Mar. 19-27 (12 pages total). [cited by applicant]

Benjamin Bross et al., “Versatile Video Coding (Draft 6)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and IS O/IEC JTC 1/SC 29/WG 11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, 455 pages. [cited by applicant]

International Search Report dated Jan. 27, 2021 in International Application No. PCT/US2020/054972. [cited by applicant]

Lotter et al., “Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning”, arViv proprint, Mar. 1, 2017, Retrieved on Dec. 19, 2020, https://arxiv.org/pdf/1605.08104.pdf (18 pages total). [cited by applicant]

Office Action dated Nov. 22, 2022 issued by the Japanese Patent Office in Japanese Application No. 2021-557674. [cited by applicant]

Written Opinion of the International Searching Authority dated Jan. 27, 2021 in International Application No. PCT/US2020/054972. [cited by applicant]

Office Action dated Jun. 28, 2023 from the Korean Patent Office in Application No. 10-2021-7023757. [cited by applicant]

Benjamin Bross et al., “Versatile Video Coding (Draft 7)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Document: JVET-P2001-vE, 16th Meeting: Geneva, CH, Oct. 1-11, 2019, pp. 1-465… [cited by applicant]