IP Library › Granted Patent US 12,439,064
Granted Patent B2
US 12,439,064 · App. 18/307,161 · Granted Oct 7, 2025

Roll sample group in VVC video coding

Inventors: Ye-Kui Wang (San Diego, CA); Li Zhang (San Diego, CA); Haibin Yin (Beijing, CN); Lingyu Li (Beijing, CN)
Assignees: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.; BYTEDANCE INC.
H04N19/30H04N19/107H04N19/172H04N19/188H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,439,064
App. No.
18/307,161
Granted
Oct 7, 2025
Kind
B2
Abstract

A mechanism for processing video data is disclosed. A conversion is performed between a visual media data and a visual media data file. The visual media data file includes pictures in layers, a random access recovery point (roll) sample group specifying access points into the layers, and a grouping type parameter. The grouping type parameter specifies a correspondence between the access points and relevant layers of the layers. The grouping type parameter includes a layer identifier method identification code parameter specifying that the access points include one or more of: one or more gradual decoding refresh (GDR) pictures; and one or more mixed network abstraction layer (NAL) unit pictures with both an intra random access point (IRAP) subpicture and a non-IRAP subpicture.

Claims (53)

1. A method for processing visual media data comprising:

performing a conversion between visual media data and a visual media data file that includes pictures in layers, based on a random access recovery point (roll) sample group specifying access points into the layers, and a grouping type parameter, the grouping type parameter specifying a correspondence between the access points and relevant layers of the layers and including a layer identifier method identification code parameter specifying that the access points include one or more of:

one or more gradual decoding refresh (GDR) pictures; and

one or more mixed network abstraction layer (NAL) unit pictures with both an intra random access point (IRAP) subpicture and a non-IRAP subpicture,

wherein the layer identifier method identification code parameter is denoted as layer_id_method_idc, and wherein the layer_id_method_idc equal to one or three specifies that: the access points apply to only the relevant layers and each bit in a target layer parameter included in the grouping type parameter specifies one of the relevant layers.

2. The method of claim 1 , wherein the conversion includes:

encoding the pictures into the layers in the visual media data file;

determining the roll sample group specifying access points into the layers;

encoding the grouping type parameter into the visual media data file; and

storing the visual media data file.

3. The method of claim 1 , wherein the conversion includes:

receiving the visual media data file comprising the pictures coded into the layers;

obtaining the roll sample group from the visual media data file, the roll sample group specifying the access points into the layers;

obtaining the grouping type parameter from the visual media data file; and

decoding the visual media data file based on the grouping type parameter.

4. The method of claim 1 , wherein a value of the layer identifier method identification code parameter specifies that the access points apply to only the relevant layers or the access points apply to all of the layers.

5. The method of claim 1 , wherein the grouping type parameter is denoted as group_type_parameter, and wherein the target layer parameter is denoted as target_layers.

6. The method of claim 1 , wherein the layer_id_method_idc is set to zero when specifying all access points in the relevant layers are GDR pictures and the access points apply to all the layers.

7. The method of claim 1 , wherein the layer_id_method_idc is set to one when specifying all access points in the relevant layers are GDR pictures and the access points apply to only the relevant layers.

8. The method of claim 1 , wherein the layer_id_method_idc is set to two when specifying the access points in the relevant layers are GDR pictures, mixed NAL unit pictures, or combinations thereof and the access points apply to all the layers.

9. The method of claim 1 , wherein the layer_id_method_idc is set to three when specifying the access points in the relevant layers are GDR pictures, mixed NAL unit pictures, or combinations thereof and the access points apply to only the relevant layers.

10. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

perform a conversion between visual media data and a visual media data file that includes pictures in layers, based on a random access recovery point (roll) sample group specifying access points into the layers, and a grouping type parameter, the grouping type parameter specifying a correspondence between the access points and relevant layers of the layers and including a layer identifier method identification code parameter specifying that the access points include one or more of:

one or more gradual decoding refresh (GDR) pictures; and

one or more mixed network abstraction layer (NAL) unit pictures with both an intra random access point (IRAP) subpicture and a non-IRAP subpicture,

wherein the layer identifier method identification code parameter is denoted as layer_id_method_idc, and wherein the layer_id_method_idc equal to one or three specifies that: the access points apply to only the relevant layers and each bit in a target layer parameter included in the grouping type parameter specifies one of the relevant layers.

11. The apparatus of claim 10 , wherein the conversion includes:

encoding the pictures into the layers in the visual media data file;

determining the roll sample group specifying access points into the layers;

encoding the grouping type parameter into the visual media data file; and

storing the visual media data file.

12. The apparatus of claim 10 , wherein the conversion includes:

receiving the visual media data file comprising the pictures coded into the layers;

obtaining the roll sample group from the visual media data file, the roll sample group specifying the access points into the layers;

obtaining the grouping type parameter from the visual media data file; and

decoding the visual media data file based on the grouping type parameter.

13. The apparatus of claim 10 , wherein a value of the layer identifier method identification code parameter specifies that the access points apply to only the relevant layers or the access points apply to all of the layers.

14. The apparatus of claim 10 , wherein the grouping type parameter is denoted as group_type_parameter, wherein the target layer parameter is denoted as target_layers,

wherein the layer_id_method_idc is set to zero when specifying all access points in the relevant layers are GDR pictures and the access points apply to all the layers,

wherein the layer_id_method_idc is set to one when specifying all access points in the relevant layers are GDR pictures and the access points apply to only the relevant layers,

wherein the layer_id_method_idc is set to two when specifying the access points in the relevant layers are GDR pictures, mixed NAL unit pictures, or combinations thereof and the access points apply to all the layers, and

wherein the layer_id_method_idc is set to three when specifying the access points in the relevant layers are GDR pictures, mixed NAL unit pictures, or combinations thereof and the access points apply to only the relevant layers.

15. A non-transitory computer readable medium comprising a computer program product for use by a video coding device, the computer program product comprising computer executable instructions stored on the non-transitory computer readable medium such that when executed by a processor cause the video coding device to:

perform a conversion between visual media data and a visual media data file that includes pictures in layers, based on a random access recovery point (roll) sample group specifying access points into the layers, and a grouping type parameter, the grouping type parameter specifying a correspondence between the access points and relevant layers of the layers and including a layer identifier method identification code parameter specifying that the access points include one or more of:

one or more gradual decoding refresh (GDR) pictures; and

one or more mixed network abstraction layer (NAL) unit pictures with both an intra random access point (IRAP) subpicture and a non-IRAP subpicture,

wherein the layer identifier method identification code parameter is denoted as layer_id_method_idc, and wherein the layer_id_method_idc equal to one or three specifies that: the access points apply to only the relevant layers and each bit in a target layer parameter included in the grouping type parameter specifies one of the relevant layers.

16. The non-transitory computer readable medium of claim 15 , wherein a value of the layer identifier method identification code parameter specifies that the access points apply to only the relevant layers or the access points apply to all of the layers.

17. The non-transitory computer readable medium of claim 15 , wherein the grouping type parameter is denoted as group_type_parameter, wherein the target layer parameter is denoted as target_layers,

wherein the layer_id_method_idc is set to zero when specifying all access points in the relevant layers are GDR pictures and the access points apply to all the layers,

wherein the layer_id_method_idc is set to one when specifying all access points in the relevant layers are GDR pictures and the access points apply to only the relevant layers,

wherein the layer_id_method_idc is set to two when specifying the access points in the relevant layers are GDR pictures, mixed NAL unit pictures, or combinations thereof and the access points apply to all the layers, and

wherein the layer_id_method_idc is set to three when specifying the access points in the relevant layers are GDR pictures, mixed NAL unit pictures, or combinations thereof and the access points apply to only the relevant layers.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 12, 2023
From: WANG, YE-KUI; ZHANG, LI
To: BYTEDANCE INC.
Reel/Frame 063919/0749 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 12, 2023
From: YIN, HAIBIN; LI, LINGYU
To: SHANGHAI SUIXUNTONG ELECTRONIC TECHNOLOGY CO., LTD.
Reel/Frame 063919/0800 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 12, 2023
From: SHANGHAI SUIXUNTONG ELECTRONIC TECHNOLOGY CO., LTD.
To: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 063919/0848 →
Priority Claims (1)
WO PCT/CN2020/123540 · Oct 26, 2020 · international
Continuity (2)
Continuation PCTCN2021126299 · Oct 26, 2021
Related Publication 20230269384A1 · Aug 24, 2023
References Cited (61)
US 20050220441A1 · Comer · 2005 [cited by applicant]
US 20160212437A1 · Tsukuba · 2016 [cited by applicant]
US 20170111642A1 · Hendry · 2017 [cited by applicant]
US 20200177923A1 · Chen · 2020 [cited by applicant]
US 20200302650A1 · Aksu · 2020 [cited by applicant]
US 20220070496A1 · Wang et al. · 2022 [cited by applicant]
US 20220086430A1 · Wang · 2022 [cited by applicant]
US 20220086494A1 · Wang · 2022 [cited by applicant]
US 20230015840A1 · Sánchez La Fuente · 2023 [cited by examiner]
US 20230336761A1 · Hendry · 2023 [cited by applicant]
US 20240040169A1 · Hendry · 2024 [cited by examiner]
CN 101416149A · 2009 [cited by applicant]
CN 105103561A · 2015 [cited by applicant]
CN 105556981A · 2016 [cited by applicant]
CN 107211159A · 2017 [cited by applicant]
CN 107750458A · 2018 [cited by applicant]
CN 108141616A · 2018 [cited by applicant]
CN 108353191A · 2018 [cited by applicant]
CN 109155861A · 2019 [cited by applicant]
GB 202007552 · 2020 [cited by applicant]
IN 201747023656A · 2017 [cited by applicant]
IN 201827009114A · 2018 [cited by applicant]
JP 2008518516A · 2008 [cited by applicant]
JP 2016507988A · 2016 [cited by applicant]
JP 2018510546A · 2018 [cited by applicant]
JP 2018511208A · 2018 [cited by applicant]
JP 2018524891A · 2018 [cited by applicant]
JP 2022050360A · 2022 [cited by applicant]
JP 2023508736A · 2023 [cited by applicant]
WO 2015105399A1 · 2015 [cited by applicant]
WO WO2017066075A1 · 2017 [cited by examiner]
WO 2019141901A1 · 2019 [cited by applicant]
“Information technology—High efficiency coding and media delivery in heterogeneous environments—Part 2: High Efficiency Video Coding”, ISO/IEC JTC 1/SC 29/WG 11 N17661, Rec. ITU-T H.265 | ISO/IEC 23008-2 (in force editi… [cited by applicant]
Document: JVET-G1001-v1, Chen, J., et al., “Algorithm Description of Joint Exploration Test Model 7 (JEM7),” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 7th Meeting: Torino, IT,… [cited by applicant]
“Information technology—Coded representation of immersive media—Part 3: Versatile video coding, ISO/IEC JTC 1/SC 29/WG 11 N18692,” Rec. Itu-T H.266 | ISO/IEC 23090-3, Jul. 12, 2019, 445 pages. [cited by applicant]
Document: JVET-S2001-vH, Bross, B., et al., “Versatile Video Coding (Draft 10),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 19th Meeting: by teleconference, Jun. 22-Jul. 1, 2020, 5… [cited by applicant]
“Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services—Coding of moving video Versatile supplemental enhancement information messages for coded video bitstreams” Rec. ITU-T Rec. H.274 | ISO… [cited by applicant]
Document: JVET-S2007-v7, Boyce, J., et al., “Versatile supplemental enhancement information messages for coded video bitstreams (Draft 5),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG … [cited by applicant]
ISO/IEC 14496-12: “Information technology—Coding of audio-visual objects—Part 12: ISO base media file format, Technical Corrigendum 1,” ISO/IEC JTC 1/SC 29 N, Jun. 3, 2016, 303 pages. [cited by applicant]
ISO/IEC 23009-1: “Information technology—Dynamic adaptive streaming over HTTP (DASH)—Part 1: Media presentation description and segment formats,” ISO/IEC JTC1/SC29/WG11 MPEG2019/m52458, Jan. 2020, 287 pages. [cited by applicant]
ISO/IEC 14496-15: “Information technology—Coding of audio-visual objects—Part 15: Carriage of network abstraction layer (NAL) unit structured video in the ISO base media file format,” ISO/IEC 14496-15: 2021, 282 pages. [cited by applicant]
ISO/IEC 23008-12: “Information technology—High efficiency coding and media delivery in heterogeneous environments—Part 12: Image File Format—Amendment 2: Support for VVC, EVC, slideshows and other improvements,” ISO/IEC… [cited by applicant]
MPEG output document N19454, “Information technology—Coding of audio-visual objects—Part 15: Carriage of network abstraction layer (NAL) unit structured video in the ISO base media file format—Amendment 2: Carriage of V… [cited by applicant]
MPEG output document N19460, “Information technology—High efficiency coding and media delivery in heterogeneous environments—Part 12: Image File Format—Amendment 3: Support for VVC, EVC, slideshows and other improvement… [cited by applicant]
Wang, Y., “Signalling of decoder configuration information and subpicture entity groups,” ISO/IEC JTC 1/SC 29/WG 3 m55177, Oct. 2020, 9 pages. [cited by applicant]
Hendry, “On signalling operating point related information in sample entry of VVC file format,” ISO/IEC JTC 1/SC 29/WG 3 m55015, Oct. 2020, 6 pages. [cited by applicant]
Sanchez, Y., “[14496-15]: On Random Access for mixed NALUs in VVC with shifted IRAPs,” ISO/IEC JTC 1/SC 29/WG 3 m55163, Oct. 2020, 4 pages. [cited by applicant]
Document: JVET-R2001-vA, Bross, B., et al., “Versatile Video Coding (Draft 9),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 18th Meeting: by teleconference, Apr. 15-24, 2020, 524 pa… [cited by applicant]
Foreign Communication From A Related Counterpart Application, PCT Application No. PCT/CN2021/126297, International Search Report dated Dec. 30, 2021, 10 pages. [cited by applicant]
Foreign Communication From A Related Counterpart Application, PCT Application No. PCT/CN2021/126299, International Search Report dated Jan. 17, 2022, 12 pages. [cited by applicant]
“Technologies under Consideration for NAL File Format”, ISO/IEC JTC 1/SC 29/WG 03 N0036, Oct. 16, 2020, XP030291529, Retrieved from the Internet: URL: https://dms.mpeg.expert/doc_end_user/documents/132_OnLine/wg11/MDS19… [cited by applicant]
Extended European Search Report from U.S. Patent Application No. 21885124.4 dated Feb. 13, 2024, 10 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 18/306,886, mailed Oct. 7, 2024, 18 Pages. [cited by applicant]
Japanese Notification of Reasons for Refusal from Japanese Patent Application No. 2023-524965 dated Nov. 26, 2024, 6 pages. [cited by applicant]
Japanese Office Action from Japanese Patent Application No. 2023-524966 dated Jul. 30, 2024, 6 pages. [cited by applicant]
Japanese Office Action from Japanese Patent Application No. 2023-524965 dated Jul. 30, 2024, 6 pages. [cited by applicant]
Document: JVET-M0101, Skupin, R., et al., “AHG17: On VVC HLS,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, 11 pages. [cited by applicant]
Indian Office Action from Indian Patent Application No. 202327030432 dated Oct. 1, 2025, 6 pages. [cited by applicant]
Japanese Decision of Registration for Japanese Patent Application No. 2023-524965 dated Mar. 11, 2025, 5 pages. [cited by applicant]
Chinese Office Action from Chinese Patent Application No. 202180073375.9 dated Jul. 8, 2025, 22 pages. [cited by applicant]
Chinese Notice of Allowance from Chinese Patent Application No. 202180073375.9 dated Aug. 20, 2025, 5 pages. [cited by applicant]