IP Library Granted Patent US 12,684,129
Granted Patent B2
US 12,684,129 · App. 18/348,058 · Granted Jul 14, 2026

Cross random access point signaling enhancements

Inventors: Ye-kui Wang (San Diego, CA); Yang Wang (Beijing, CN); Li Zhang (San Diego, CA)
Assignees: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.; BYTEDANCE INC.
H04N19/136H04N19/172H04N19/46H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,684,129
App. No.
18/348,058
Filed
Jul 6, 2023
Granted
Jul 14, 2026
Kind
B2
Art Unit
2482
USPC
375/240.02
Abstract

A mechanism for processing video data is disclosed. One or more random access picture (RAP) picture identifiers are signaled for one or more Cross RAP Referencing (CRR) pictures. A conversion is performed between a visual media data and a bitstream based on the one or more RAP picture identifiers.

Claims (38)

1 . A method for processing video data comprising:

determining one or more random access point (RAP) picture identifiers for one or more Cross RAP Referencing (CRR) pictures; and

performing a conversion between a visual media data and a bitstream based on the one or more RAP picture identifiers;

wherein each of the CRR pictures is associated with an intra random access point (IRAP) picture, and wherein the IRAP picture is associated with a RAP picture identifier of zero; and

wherein the one or more RAP picture identifiers are different for each of the CRR pictures that are associated with a same IRAP picture.

2 . The method of claim 1 , wherein the one or more RAP picture identifiers are each coded in a coded layer video sequence minus one (t2drap_rap_id_in_clvs_minus1) field.

3 . The method of claim 1 , wherein the one or more RAP picture identifiers are each included in a type 2 dependent random access point (DRAP) supplemental enhancement information (SEI) message.

4 . The method of claim 2 , wherein each of the one or more RAP picture identifiers is specified by a value of the t2drap_rap_id_in_clvs_minus1 field plus one.

5 . The method of claim 1 , wherein the one or more RAP picture identifiers for each of the CRR pictures are set to a value greater than zero.

6 . The method of claim 1 , wherein a RAP picture identifier of the IRAP picture is inferred to be zero and is not signaled.

7 . The method of claim 1 , wherein the one or more RAP picture identifiers are denoted as RapPicIds.

8 . The method of claim 1 , wherein other syntax elements in a type 2 dependent random access point (DRAP) supplemental enhancement information (SEI) message are only signaled when a RAP picture identifier in the type 2 DRAP SEI message is greater than zero.

9 . The method of claim 1 , wherein the CRR pictures are denoted as type 2 dependent random access point (DRAP) pictures.

10 . The method of claim 1 , wherein the CRR pictures are denoted as enhanced dependent random access point (EDRAP) pictures.

11 . The method of claim 1 , wherein the conversion comprises generating the bitstream according to the visual media data.

12 . The method of claim 1 , wherein the conversion comprises parsing the bitstream to obtain the visual media data.

13 . An apparatus for processing video data comprising:

a processor; and

a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

determine one or more random access point (RAP) picture identifiers for one or more Cross RAP Referencing (CRR) pictures; and

perform a conversion between a visual media data and a bitstream based on the one or more RAP picture identifiers;

wherein each of the CRR pictures is associated with an intra random access point (IRAP) picture, and wherein the IRAP picture is associated with a RAP picture identifier of zero; and

wherein the one or more RAP picture identifiers are different for each of the CRR pictures that are associated with a same IRAP picture.

14 . The apparatus of claim 13 , wherein the one or more RAP picture identifiers are each coded in a coded layer video sequence minus one (t2drap_rap_id_in_clvs_minus1) field,

wherein the one or more RAP picture identifiers are each included in a type 2 dependent random access point (DRAP) supplemental enhancement information (SEI) message, and

wherein each of the one or more RAP picture identifiers is specified by a value of the t2drap_rap_id_in_clvs_minus1 field plus one.

15 . The apparatus of claim 13 , wherein the one or more RAP picture identifiers for each of the CRR pictures are set to a value greater than zero, and

wherein a RAP picture identifier of the IRAP picture is inferred to be zero and is not signaled.

16 . A non-transitory computer-readable storage medium storing instructions that cause a processor to:

determine one or more random access point (RAP) picture identifiers for one or more Cross RAP Referencing (CRR) pictures; and

perform a conversion between a visual media data and a bitstream based on the one or more RAP picture identifiers;

wherein each of the CRR pictures is associated with an intra random access point (IRAP) picture, and wherein the IRAP picture is associated with a RAP picture identifier of zero; and

wherein the one or more RAP picture identifiers are different for each of the CRR pictures that are associated with a same IRAP picture.

17 . The non-transitory computer-readable storage medium of claim 16 , wherein the one or more RAP picture identifiers are each coded in a coded layer video sequence minus one (t2drap_rap_id_in_clvs_minus1) field,

wherein the one or more RAP picture identifiers are each included in a type 2 dependent random access point (DRAP) supplemental enhancement information (SEI) message, and

wherein each of the one or more RAP picture identifiers is specified by a value of the t2drap_rap_id_in_clvs_minus1 field plus one.

18 . The non-transitory computer-readable storage medium of claim 16 , wherein the one or more RAP picture identifiers for each of the CRR pictures are set to a value greater than zero, and

wherein a RAP picture identifier of the IRAP picture is inferred to be zero and is not signaled.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 25, 2023
From: WANG, YE-KUI; ZHANG, LI
To: BYTEDANCE INC.
Reel/Frame 064374/0037 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 25, 2023
From: WANG, YANG
To: BEIJING OCEAN ENGINE NETWORK TECHNOLOGY CO., LTD.,
Reel/Frame 064374/0066 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 25, 2023
From: BEIJING OCEAN ENGINE NETWORK TECHNOLOGY CO., LTD.
To: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 064374/0113 →
Priority Claims (1)
WO PCT/CN2021/070400 · Jan 6, 2021 · international
Continuity (2)
Continuation PCTCN2021141942 · Dec 28, 2021
Related Publication 20230353748A1 · Nov 2, 2023
References Cited (106)
US 5790839A · Luk et al. · 1998 [cited by applicant]
US 5901304A · Hwang et al. · 1999 [cited by applicant]
US 6072834A · Kim et al. · 2000 [cited by applicant]
US 6097756A · Han · 2000 [cited by applicant]
US 6580754B1 · Wan et al. · 2003 [cited by applicant]
US 6728317B1 · Demos · 2004 [cited by applicant]
US 6765962B1 · Lee et al. · 2004 [cited by applicant]
US 6771703B1 · Oguz et al. · 2004 [cited by applicant]
US 6826232B2 · Chen et al. · 2004 [cited by applicant]
US 7016412B1 · van Zon · 2006 [cited by applicant]
US 7095782B1 · Cohen et al. · 2006 [cited by applicant]
US 7245662B2 · Piché et al. · 2007 [cited by applicant]
US 7263124B2 · Peng et al. · 2007 [cited by applicant]
US 7369610B2 · Xu et al. · 2008 [cited by applicant]
US 7391807B2 · Lin et al. · 2008 [cited by applicant]
US 7477688B1 · Zhang et al. · 2009 [cited by applicant]
US 7627034B2 · Park et al. · 2009 [cited by applicant]
US 7697608B2 · Lee · 2010 [cited by applicant]
US 7729421B2 · Campisano et al. · 2010 [cited by applicant]
US 8040952B2 · Park et al. · 2011 [cited by applicant]
US 8189659B2 · Han et al. · 2012 [cited by applicant]
US 8494042B2 · Park et al. · 2013 [cited by applicant]
US 11095916B2 · Seregin et al. · 2021 [cited by applicant]
US 12069308B2 · Hannuksela · 2024 [cited by examiner]
US 20060034374A1 · Park et al. · 2006 [cited by applicant]
US 20070064791A1 · Okada et al. · 2007 [cited by applicant]
US 20090028245A1 · Vieron et al. · 2009 [cited by applicant]
US 20110243231A1 · Li et al. · 2011 [cited by applicant]
US 20110268175A1 · Tan et al. · 2011 [cited by applicant]
US 20130028324A1 · Chang et al. · 2013 [cited by applicant]
US 20140010281A1 · Rodriguez · 2014 [cited by examiner]
US 20140016707A1 · Wang · 2014 [cited by applicant]
US 20140092953A1 · Deshpande · 2014 [cited by examiner]
US 20140092970A1 · Misra et al. · 2014 [cited by applicant]
US 20160112704A1 · Grange et al. · 2016 [cited by applicant]
US 20160219301A1 · Pettersson · 2016 [cited by examiner]
US 20160219306A1 · Pettersson · 2016 [cited by applicant]
US 20160373777A1 · Pettersson · 2016 [cited by applicant]
US 20170085917A1 · Hannuksela · 2017 [cited by applicant]
US 20170214937A1 · Lin et al. · 2017 [cited by applicant]
US 20180376126A1 · Hannuksela · 2018 [cited by applicant]
US 20200059666A1 · Pettersson et al. · 2020 [cited by applicant]
US 20200177923A1 · Chen et al. · 2020 [cited by applicant]
US 20200213617A1 · Choi et al. · 2020 [cited by applicant]
US 20200260070A1 · Yoo et al. · 2020 [cited by applicant]
US 20220086431A1 · Wang · 2022 [cited by applicant]
US 20220086432A1 · Wang et al. · 2022 [cited by applicant]
US 20220103781A1 · Wang · 2022 [cited by applicant]
US 20220103866A1 · Bossen · 2022 [cited by applicant]
US 20220103867A1 · Wang · 2022 [cited by applicant]
US 20220132104A1 · Zhang et al. · 2022 [cited by applicant]
US 20220217406A1 · Hannuksela · 2022 [cited by examiner]
US 20230007291A1 · Samuelsson et al. · 2023 [cited by applicant]
US 20230007309A1 · Wang · 2023 [cited by applicant]
US 20230164371A1 · Denoual et al. · 2023 [cited by applicant]
CN 101998142A · 2011 [cited by applicant]
CN 107431819A · 2017 [cited by applicant]
CN 108141583A · 2018 [cited by applicant]
CN 111526368A · 2020 [cited by applicant]
EP 3975569A1 · 2022 [cited by applicant]
JP 2015527813A · 2015 [cited by applicant]
JP 2017522766A · 2017 [cited by applicant]
JP 2017525175A · 2017 [cited by applicant]
JP 2022056417A · 2022 [cited by applicant]
JP 7623496B2 · 2025 [cited by applicant]
JP 7626862B2 · 2025 [cited by applicant]
JP 7629101B2 · 2025 [cited by applicant]
WO 2015192990A1 · 2015 [cited by applicant]
WO 2020058494A1 · 2020 [cited by applicant]
WO 2022143615A1 · 2022 [cited by applicant]
WO 2022148269A1 · 2022 [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 18/343,370, mailed Oct. 15, 2024, 35 Pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 18/343,307 mailed Nov. 20, 2024, 27 pages. [cited by applicant]
Decision of Refusal for Japanese Application No. 2023-539838, mailed Jan. 21, 2025, 8 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 18/343,335, mailed Jan. 8, 2025, 25 pages. [cited by applicant]
Document: JVET-M0360-v4, Yu, H., et al., “Video coding based on cross RAP referencing (CRR),” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 13th Meeting: Marrakech, MA, Jan. 9-18,… [cited by applicant]
Document: JVET-N0119-v2, Wang, Y., et al., “AHG17: EDR—external decoding refresh,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 14th Meeting: Geneva, CH, Mar. 19-27, 2019, 4 pages. [cited by applicant]
Document: JVET-O0149-v1, Wang, Y., et al., “AHG17: On external decoding refresh (EDR),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, 6 … [cited by applicant]
Document: JVET-P0114-v1, Wang, Y., et al., “AHG17: On external decoding refresh (EDR),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 16th Meeting: Geneva, CH, Oct. 1-11, 2019, 7 page… [cited by applicant]
Document: JVET-T0071-v1, Wang, Y., et al., “AHG9: SEI messages for support of cross RAP referencing based video coding,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29 20th Meeting, by telec… [cited by applicant]
“Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services—Coding of moving video High efficiency video coding,” Rec. ITU-T H.265 | ISO/IEC 23008-2 (in force edition), Feb. 2018, 692 pages. [cited by applicant]
Document: JVET-G1001-v1, Chen, J., et al., “Algorithm Description of Joint Exploration Test Model 7 (JEM 7),” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 7th Meeting: Torino, IT… [cited by applicant]
“Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services—Coding of moving video Versatile video coding,” Rec. ITU-T H.266 | ISO/IEC 23090-3, Aug. 2020, 516 pages. [cited by applicant]
Document: JVET-S2001-vH, Bross, B., et al., “Versatile Video Coding (Draft 10),” Joint Video Experts Team (JVET) , of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 19th Meeting: by teleconference, Jun. 22-Jul. 1, 2020,… [cited by applicant]
“Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services—Coding of moving video Versatile supplemental enhancement information messages for coded video bitstreams,” Rec. ITU-T Rec. H.274 | IS… [cited by applicant]
Document: JVET-S2007-v7, Boyce, J., et al., “Versatile supplemental enhancement information messages for coded video bitstreams (Draft 5),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG … [cited by applicant]
“Information technology—Coding of audio-visual objects—Part 12: ISO base media file format, Technical Corrigendum,” ISO/IEC 14496-12:2015/Cor.1, ISO/IEC JTC 1/SC 29/WG 11, Jun. 3, 2016, 303 pages. [cited by applicant]
“Information technology—Coding of audio-visual objects—Part 15: Carriage of network abstraction layer (NAL) unit structured video in the ISO base media file format,” ISO/IEC 14496-15, First edition, Apr. 15, 2004, 29 pa… [cited by applicant]
“Information technology—Coding of audio-visual objects—Part 15: Carriage of network abstraction layer (NAL) unit structured video in the ISO base media file format—Amendment 2: Carriage of VVC and EVC in ISOBMFF,” ISO/I… [cited by applicant]
“Information technology—High efficiency coding and media delivery in heterogeneous environments—Part 12: Image File Format—Amendment 3: Support for VVC, EVC, slideshows and other improvements,” ISO/IEC JTC 1/SC 29/WG 03… [cited by applicant]
Document: JVET-N0494-v3, Pettersson, M., et al., “AHG17 : Dependent random access I-25 point pictures in VVC,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 14th Meeting: Geneva, CH, Mar. … [cited by applicant]
Document: JVET-U0084-v2, Wang, Y., et al., “AHG9: Cross RAP referencing (CRR) SEI message,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 21st Meeting, by teleconference Jan. 6-15, 2021, 6… [cited by applicant]
Document: JVET-O0393, Hannuksela M., et al., “AHG17: On cross-RAP referencing,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29 15th Meeting: Gothenburg, SE Jul. 3-12, 2019, 5 pages. [cited by applicant]
Document: JVET-T0066, Skupin, R., et al., “AHG9: IRAP only HRD SEI message,” Joint Video Experts Team(JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29 20th Meeting, by teleconference Oct. 7-16, 2020 , 16 pages. [cited by applicant]
International Search Report from PCT Application No. PCT/CN2021/141937 dated Mar. 29, 2022, 9 pages. [cited by applicant]
International Search Report from PCT Application No. PCT/CN2021/141939 dated Mar. 29, 2022, 11 pages. [cited by applicant]
International Search Report from PCT Application No. PCT/CN2021/141940 dated Mar. 16, 2022, 10 pages. [cited by applicant]
International Search Report from PCT Application No. PCT/CN2021/141942 dated Mar. 29, 2022, 9 pages. [cited by applicant]
Document: JVET-N0072-v1, Chen, L., et al., “AHG17: New NAL unit types for VVC,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 14th Meeting: Geneva, Switzerland, Mar. 19-27, 2019, 6 pa… [cited by applicant]
Japanese Office Action from Japanese Patent Application No. 2023-539540 dated Aug. 6, 2024, 18 pages. [cited by applicant]
Japanese Office Action from Japanese Patent Application No. 2023-539839 dated Jul. 30, 2024, 5 pages. [cited by applicant]
Document: JVET-U0084-v1, Wang, Y.K., et al., “AHG9: Cross RAP referencing (CRR) SEI message,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29 21st Meeting, by teleconference, Jan. 6-15, 2021,… [cited by applicant]
Document: JVET-V0065-v1, Wang, Y.K., et al., “AHG9: On the DRAP and EDRAP indication SEI messages,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and 1S0/IEC JTC 1/SC 29 22nd Meeting, by teleconference, Apr. 20-28… [cited by applicant]
Extended European Search Report from European Application No. 21914349.2 dated Apr. 11, 2024, 9 pages. [cited by applicant]
Extended European Search Report from European Application No. 21917309.3 dated Jun. 6, 2024, 11 pages. [cited by applicant]
Indian Office Action from Indian Patent Application No. 202327043648 dated Feb. 24, 2025, 8 pages. [cited by applicant]