IP Library Granted Patent US 12,268,570
Granted Patent B2
US 12,268,570 · App. 18/521,714 · Granted Apr 8, 2025

Reference picture management in video coding

Inventors: FNU Hendry (San Diego, CA); Ye-Kui Wang (San Diego, CA)
Assignee: Huawei Technologies Co., Ltd.
A61C7/08A61C7/002B29C51/30B29C51/46B29L2031/753
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,268,570
App. No.
18/521,714
Granted
Apr 8, 2025
Kind
B2
Abstract

A method of decoding a coded video bitstream includes obtaining a reference picture list structure for a current slice represented in the coded video bitstream, wherein the reference picture list structure contains a number of entries; obtaining a default number of active entries in a reference picture list for the current slice; constructing a reference picture list for the current slice, the reference picture list containing a number of active and inactive entries; setting the number of active entries in the reference picture list equal to the number of entries in the reference picture list structure when the default number of active entries in the reference picture list is greater than the number of entries in the reference picture list structure; and obtaining, based on at least one active entry of the reference picture list, at least one reconstructed block of the current slice.

Claims (81)

1. A method of decoding a coded video bitstream implemented by a video decoder for use in video decoding based on the Versatile Video Coding (VVC) standard, comprising:

obtaining a reference picture list structure for a current slice from the coded video bitstream, wherein the reference picture list structure contains a number of entries;

obtaining a default number of active entries in a reference picture list for the current slice;

constructing the reference picture list for the current slice, wherein the reference picture list contains a number of active entries and a number of inactive entries;

setting the number of active entries in the reference picture list equal to the number of entries in the reference picture list structure when the default number of active entries in the reference picture list is greater than the number of entries in the reference picture list structure;

obtaining, based on at least one active entry of the reference picture list, at least one reconstructed block of the current slice; and

obtaining an override flag from the coded video bitstream, wherein the setting the number of active entries in the reference picture list equal to the number of entries in the reference picture list structure is performed on condition that a value of the override flag is false,

wherein an order of entries in the reference picture list structure is the same as an order of corresponding reference pictures in the reference picture list,

wherein the at least one active entry of the reference picture list is used for inter prediction of the at least one reconstructed block,

wherein the inactive entries are not used for inter prediction of a current picture, and

wherein the reference picture list structure, ref_pic_list_struct (listIdx, rplsIdx), includes two separate indexes, listIdx and rplsIdx.

2. The method of claim 1 , wherein the current slice is a P slice or a B slice.

3. A decoding device, comprising:

a memory storing instructions; and

one or more processors coupled to the memory, the one or more processors configured to execute the instructions stored in the memory to cause the decoding device to:

obtain a reference picture list structure for a current slice from a coded video bitstream, wherein the reference picture list structure contains a number of entries;

obtain a default number of active entries in a reference picture list for the current slice;

construct the reference picture list for the current slice, wherein the reference picture list contains a number of active entries and a number of inactive entries;

set the number of active entries in the reference picture list equal to the number of entries in the reference picture list structure when the default number of active entries in the reference picture list is greater than the number of entries in the reference picture list structure;

obtain, based on at least one active entry of the reference picture list, at least one reconstructed block of the current slice; and

obtain an override flag from the coded video bitstream, wherein the setting the number of active entries in the reference picture list equal to the number of entries in the reference picture list structure is performed on condition that a value of the override flag is false,

wherein an order of entries in the reference picture list structure is the same as an order of corresponding reference pictures in the reference picture list,

wherein the at least one active entry of the reference picture list is used for inter prediction of the at least one reconstructed block,

wherein the inactive entries are not used for inter prediction of a current picture, and

wherein the reference picture list structure, ref_pic_list_struct (listIdx, rplsIdx), includes two separate indexes, listIdx and rplsIdx.

4. The decoding device of claim 3 , wherein the current slice is a P slice or a B slice.

5. A non-transitory computer program product comprising program code for performing a method when executed on a computer or one or more processors, the method comprising:

obtaining a reference picture list structure for a current slice from a coded video bitstream, wherein the reference picture list structure contains a number of entries;

obtaining a default number of active entries in a reference picture list for the current slice;

constructing the reference picture list for the current slice, wherein the reference picture list contains a number of active entries and a number of inactive entries;

setting the number of active entries in the reference picture list equal to the number of entries in the reference picture list structure when the default number of active entries in the reference picture list is greater than the number of entries in the reference picture list structure;

obtaining, based on at least one active entry of the reference picture list, at least one reconstructed block of the current slice; and

obtaining an override flag from the coded video bitstream, wherein the setting the number of active entries in the reference picture list equal to the number of entries in the reference picture list structure is performed on condition that a value of the override flag is false,

wherein an order of entries in the reference picture list structure is the same as an order of corresponding reference pictures in the reference picture list,

wherein the at least one active entry of the reference picture list is used for inter prediction of the at least one reconstructed block,

wherein the inactive entries are not used for inter prediction of a current picture, and

wherein the reference picture list structure, ref_pic_list_struct (listIdx, rplsIdx), includes two separate indexes, listIdx and rplsIdx.

6. The non-transitory computer program product of claim 5 , wherein the current slice is a P slice or a B slice.

7. The method of claim 1 , wherein the listIdx has a value to specify a reference picture list, and wherein the rplsIdx specifies an index of a reference picture list syntax structure.

8. The decoding device of claim 3 , wherein the listIdx has a value to specify a reference picture list, and wherein the rplsIdx specifies an index of a reference picture list syntax structure.

9. The non-transitory computer program product of claim 5 , wherein the listIdx has a value to specify a reference picture list, and wherein the rplsIdx specifies an index of a reference picture list syntax structure.

10. The method of claim 1 , wherein the reference picture list structure, ref_pic_list_struct (listIdx, rplsIdx), is defined as:

Descriptor

ref_pic_list_struct( listIdx, rplsIdx ) {

 num_strp_entries[ listIdx ][ rplsIdx ]

ue(v)

 for( i = 0; i < NumEntriesInList[ listIdx ][ rplsIdx ]; i++) {

  delta_poc_st[ listIdx ][ rplsIdx ][ i ]

se(v)

 }

}

wherein the ref_pic_list_struct (listIdx, rplsIdx) comprises a first parameter listIdx and a second parameter rplsIdx, wherein rplsIdx specifies an index of a reference picture list syntax structure, listIdx has a value to specify a reference picture list; wherein the value of the listIdx equals to 0 or 1,

wherein the reference picture list syntax structure is represented in a sequence parameter set (SPS) and specifies a candidate for short-term reference picture list listIdx, and the current picture refers to each picture that 1) has one or more slices containing ref_pic_list_idx [listIdx] equal to an index into the list of the ref_pic_list_struct (listIdx, rplsIdx) syntax structures included in the SPS, and 2) is in a coded video sequence (CVS) that has the SPS as the active SPS,

wherein num_strp_entries [listIdx][rplsIdx] specifies the number of short term reference picture (STRP) entries in the ref_pic_list_struct (listIdx, rplsIdx) syntax structure, and

wherein delta_poc_st [listIdx][rplsIdx][i] specifies the difference between the picture order count values of the current picture and the picture referred to by the i-th entry when the i-th entry is the first STRP entry in ref_pic_list_struct (listIdx, rplsIdx) syntax structure, or delta_poc_st [listIdx][rplsIdx][i] specifies the difference between the picture order count values of the pictures referred to by the i-th entry and by the previous STRP entry in the ref_pic_list_struct (listIdx, rplsIdx) syntax structure when the i-th entry is an STRP entry but not the first STRP entry in the ref_pic_list_struct (listIdx, rplsIdx) syntax structure.

11. The decoding device of claim 3 , wherein the reference picture list structure, ref_pic_list_struct (listIdx, rplsIdx), is defined as:

Descriptor

ref_pic_list_struct( listIdx, rplsIdx ) {

 num_strp_entries[ listIdx ][ rplsIdx ]

ue(v)

 for( i = 0; i < NumEntriesInList[ listIdx ][ rplsIdx ]; i++) {

  delta_poc_st[ listIdx ][ rplsIdx ][ i ]

se(v)

 }

}

wherein the ref_pic_list_struct (listIdx, rplsIdx) comprises a first parameter listIdx and a second parameter rplsIdx, wherein rplsIdx specifies an index of a reference picture list syntax structure, listIdx has a value to specify a reference picture list; wherein the value of the listIdx equals to 0 or 1,

wherein the reference picture list syntax structure is represented in a sequence parameter set (SPS) and specifies a candidate for short-term reference picture list listIdx, and the current picture refers to each picture that 1) has one or more slices containing ref_pic_list_idx [listIdx] equal to an index into the list of the ref_pic_list_struct (listIdx, rplsIdx) syntax structures included in the SPS, and 2) is in a coded video sequence (CVS) that has the SPS as the active SPS,

wherein num_strp_entries [listIdx][rplsIdx] specifies the number of short term reference picture (STRP) entries in the ref_pic_list_struct (listIdx, rplsIdx) syntax structure, and wherein delta_poc_st [listIdx][rplsIdx][i] specifies the difference between the picture order count values of the current picture and the picture referred to by the i-th entry when the i-th entry is the first STRP entry in ref_pic_list_struct (listIdx, rplsIdx) syntax structure, or delta_poc_st [listIdx][rplsIdx][i] specifies the difference between the picture order count values of the pictures referred to by the i-th entry and by the previous STRP entry in the ref_pic_list_struct (listIdx, rplsIdx) syntax structure when the i-th entry is an STRP entry but not the first STRP entry in the ref_pic_list_struct (listIdx, rplsIdx) syntax structure.

12. The non-transitory computer program product of claim 5 , wherein the reference picture list structure, ref_pic_list_struct (listIdx, rplsIdx), is defined as:

Descriptor

ref_pic_list_struct( listIdx, rplsIdx ) {

 num_strp_entries[ listIdx ][ rplsIdx ]

ue(v)

 for( i = 0; i < NumEntriesInList[ listIdx ][ rplsIdx ]; i++) {

  delta_poc_st[ listIdx ][ rplsIdx ][ i ]

se(v)

 }

}

wherein the ref_pic_list_struct (listIdx, rplsIdx) comprises a first parameter listIdx and a second parameter rplsIdx, wherein rplsIdx specifies an index of a reference picture list syntax structure, listIdx has a value to specify a reference picture list; wherein the value of the listIdx equals to 0 or 1,

wherein the reference picture list syntax structure is represented in a sequence parameter set (SPS) and specifies a candidate for short-term reference picture list listIdx, and the current picture refers to each picture that 1) has one or more slices containing ref_pic_list_idx [listIdx] equal to an index into the list of the ref_pic_list_struct (listIdx, rplsIdx) syntax structures included in the SPS, and 2) is in a coded video sequence (CVS) that has the SPS as the active SPS,

wherein num_strp_entries [listIdx][rplsIdx] specifies the number of short term reference picture (STRP) entries in the ref_pic_list_struct (listIdx, rplsIdx) syntax structure, and wherein delta_poc_st [listIdx][rplsIdx][i] specifies the difference between the picture order count values of the current picture and the picture referred to by the i-th entry when the i-th entry is the first STRP entry in ref_pic_list_struct (listIdx, rplsIdx) syntax structure, or delta_poc_st [listIdx][rplsIdx][i] specifies the difference between the picture order count values of the pictures referred to by the i-th entry and by the previous STRP entry in the ref_pic_list_struct (listIdx, rplsIdx) syntax structure when the i-th entry is an STRP entry but not the first STRP entry in the ref_pic_list_struct (listIdx, rplsIdx) syntax structure.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 30, 2025
From: FUTUREWEI TECHNOLOGIES, INC.
To: HUAWEI TECHNOLOGIES CO., LTD.
Reel/Frame 070059/0498 →
Continuity (4)
Continuation 17176595 · Feb 16, 2021
Continuation PCTUS2019046934 · Aug 16, 2019
Provisional Application 62719360 · Aug 17, 2018
Related Publication 20240130835A1 · Apr 25, 2024
References Cited (114)
US 9521418B2 · Chen et al. · 2016 [cited by applicant]
US 10003817B2 · Sullivan et al. · 2018 [cited by applicant]
US 10158881B2 · Chen et al. · 2018 [cited by applicant]
US 10432928B2 · Li et al. · 2019 [cited by applicant]
US 10743024B2 · Yamaoto et al. · 2020 [cited by applicant]
US 20050123056A1 · Wang et al. · 2005 [cited by applicant]
US 20070019724A1 · Tourapis et al. · 2007 [cited by applicant]
US 20090262804A1 · Pandit et al. · 2009 [cited by applicant]
US 20110096835A1 · Lim et al. · 2011 [cited by applicant]
US 20120056981A1 · Tian et al. · 2012 [cited by applicant]
US 20120183076A1 · Boyce · 2012 [cited by examiner]
US 20120189053A1 · Chen et al. · 2012 [cited by applicant]
US 20130057646A1 · Chen et al. · 2013 [cited by applicant]
US 20130077677A1 · Wang et al. · 2013 [cited by applicant]
US 20130077685A1 · Chen et al. · 2013 [cited by applicant]
US 20130077687A1 · Wang et al. · 2013 [cited by applicant]
US 20130089134A1 · Wang et al. · 2013 [cited by applicant]
US 20130107953A1 · Chen · 2013 [cited by examiner]
US 20130107958A1 · Shimada et al. · 2013 [cited by applicant]
US 20130114710A1 · Park et al. · 2013 [cited by applicant]
US 20130114741A1 · Sullivan et al. · 2013 [cited by applicant]
US 20130114742A1 · Hannuksela · 2013 [cited by examiner]
US 20130155184A1 · Chen et al. · 2013 [cited by applicant]
US 20130202034A1 · Yu et al. · 2013 [cited by applicant]
US 20130202035A1 · Chen et al. · 2013 [cited by applicant]
US 20130208792A1 · He et al. · 2013 [cited by applicant]
US 20130215975A1 · Samuelsson et al. · 2013 [cited by applicant]
US 20130279589A1 · Gu et al. · 2013 [cited by applicant]
US 20130322531A1 · Chen et al. · 2013 [cited by applicant]
US 20130329787A1 · Ramasubramonian et al. · 2013 [cited by applicant]
US 20140010291A1 · He et al. · 2014 [cited by applicant]
US 20140016699A1 · Chen et al. · 2014 [cited by applicant]
US 20140049604A1 · Chen et al. · 2014 [cited by applicant]
US 20140050270A1 · Lim et al. · 2014 [cited by applicant]
US 20140086324A1 · Ramasubramonian et al. · 2014 [cited by applicant]
US 20140198857A1 · Deshpande · 2014 [cited by applicant]
US 20140286416A1 · Jeon et al. · 2014 [cited by applicant]
US 20140301453A1 · Deshpande · 2014 [cited by applicant]
US 20150016505A1 · Sjoberg et al. · 2015 [cited by applicant]
US 20150023422A1 · Zhang et al. · 2015 [cited by applicant]
US 20150071351A1 · Lee et al. · 2015 [cited by applicant]
US 20150092839A1 · Chien et al. · 2015 [cited by applicant]
US 20150103881A1 · Hendry et al. · 2015 [cited by applicant]
US 20150124877A1 · Choi et al. · 2015 [cited by applicant]
US 20150172655A1 · Hendry et al. · 2015 [cited by applicant]
US 20150195506A1 · Lin et al. · 2015 [cited by applicant]
US 20150271487A1 · Li et al. · 2015 [cited by applicant]
US 20150312591A1 · Takahashi et al. · 2015 [cited by applicant]
US 20150373315A1 · Chen et al. · 2015 [cited by applicant]
US 20150373361A1 · Wang et al. · 2015 [cited by applicant]
US 20160057420A1 · Pang et al. · 2016 [cited by applicant]
US 20160100189A1 · Pang et al. · 2016 [cited by applicant]
US 20160241850A1 · Deshpande · 2016 [cited by applicant]
US 20160353117A1 · Seregin et al. · 2016 [cited by applicant]
US 20170094270A1 · Zhao et al. · 2017 [cited by applicant]
US 20170150159A1 · Lee et al. · 2017 [cited by applicant]
US 20170214938A1 · Zhang et al. · 2017 [cited by applicant]
US 20180160122A1 · Xu et al. · 2018 [cited by applicant]
US 20180270504A1 · Park et al. · 2018 [cited by applicant]
US 20190230376A1 · Hu et al. · 2019 [cited by applicant]
US 20200296392A1 · Choi · 2020 [cited by examiner]
US 20210092448A1 · Choi et al. · 2021 [cited by applicant]
US 20210168359A1 · Wang et al. · 2021 [cited by applicant]
US 20210195236A1 · Hendry et al. · 2021 [cited by applicant]
US 20210203923A1 · Sjoberg et al. · 2021 [cited by applicant]
US 20210266600A1 · Seregin et al. · 2021 [cited by applicant]
US 20210360290A1 · Seregin et al. · 2021 [cited by applicant]
US 20210368163A1 · Chen et al. · 2021 [cited by applicant]
US 20210385436A1 · Hsiang et al. · 2021 [cited by applicant]
US 20210409691A1 · Hendry et al. · 2021 [cited by applicant]
US 20220038685A1 · Deshpande · 2022 [cited by applicant]
US 20220060712A1 · Zhang et al. · 2022 [cited by applicant]
US 20220394243A1 · Deng et al. · 2022 [cited by applicant]
US 20220394301A1 · Deshpande et al. · 2022 [cited by applicant]
US 20230008778A1 · Wang et al. · 2023 [cited by applicant]
US 20230048070A1 · Deshpande · 2023 [cited by applicant]
US 20230084006A1 · Wang et al. · 2023 [cited by applicant]
US 20230111484A1 · Wang et al. · 2023 [cited by applicant]
US 20230120726A1 · Deng et al. · 2023 [cited by applicant]
CN 103814574A · 2014 [cited by applicant]
CN 103828374A · 2014 [cited by applicant]
CN 103959793A · 2014 [cited by applicant]
CN 104662920A · 2015 [cited by applicant]
CN 105122816A · 2015 [cited by applicant]
CN 106664426A · 2017 [cited by applicant]
CN 112585974A · 2021 [cited by applicant]
GB 2548739A · 2019 [cited by applicant]
RU 2571410C2 · 2015 [cited by applicant]
WO 2013043886A2 · 2013 [cited by applicant]
WO 2013043887A1 · 2013 [cited by applicant]
WO 2013111605A1 · 2013 [cited by applicant]
WO 2014004201A1 · 2014 [cited by applicant]
WO 2014006854A1 · 2014 [cited by applicant]
WO 2014097913A1 · 2014 [cited by applicant]
Document: JCTVC-U1005_v2_defects, Gerhard Tech, et al., “JCT-3V AHG Report: 3D-HEVC Draft and MV-HEVC / 3D-HEVC Test Model editing (AGH2) Defect report,” Joint Collaborative Team on 3D Video Coding Extension of ITU-T SG… [cited by applicant]
Sakae Okubo (et al.), Supervising Editor, “H.265/HEVC Textbook,” 1st Ed., Impress Japan Co., Ltd., Oct. 21, 2013, pp. 130-135, 208-214. [cited by applicant]
Sjoberg, “Overview of HEVC High-Level Syntax and Reference Picture Management,” IEEE Transactions on Circuits and Systems for Video Technology, XP011487804, vol. 22, No. 12, Dec. 2012, pp. 1858-1870. [cited by applicant]
ITU-T H.261, “Line Transmission of Non-Telephone Signals, Video Codec for Audiovisual Services at p x 64 kbits,” Mar. 1993, 29 pages. [cited by applicant]
ITU-T H.262, “Transmission of Non-Telephone Signals, Information Technology—Generic Coding of Moving Pictures and Associated Audio Information: Video,” Feb. 2000, 220 pages. [cited by applicant]
ITU-T H.263, “Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video, Video coding for low bit rate communication,” Feb. 1998, 167 pages. [cited by applicant]
ITU-T H.265, “Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video, High Efficiency Video Coding,” Apr. 2013, 317 pages. [cited by applicant]
ITU-T H.265, “Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services—Coding of moving video High Efficiency Video Coding,” Recommendation Dec. 2016, 664 pages. [cited by applicant]
Hannuksela, M., et al., “AHG21: On reference picture list construction and reference picture marking,” JCTVC-G643, 7th Meeting: Geneva, CH, Nov. 21-30, 2011, JCTVC-G643, 10 pages. [cited by applicant]
Wang, Y., et al., “On reference picture management for VVC,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-M0128-v1, 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, 9 pages. [cited by applicant]
Hannuksela et al.,“AHG21: On reference picture list construction and reference picture marking,” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/ SC29/WG11, JCTVC-G643r1, 7th Meeting… [cited by applicant]
Wang, Y., et al., “On reference picture management for VVC,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IECJTC 1/SC 29/WG11, JVET-L0112-v1, 12th Meeting: Macao, CN, Oct. 3-12, 2018, 11 pages. [cited by applicant]
Wang, Y.K., et al., “On reference picture management for VVC,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Document: JVET-L0112-v2, 12th Meeting: Macao, CN, Oct. 3-12, 2018, 13 pag… [cited by applicant]
Wang, Y., et al., “On reference picture management for VVC,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Document: JVET-L0112-v4, 12th Meeting: Macao, CN, Oct. 3-12, 2018, 13 pages. [cited by applicant]
Bross, B., et al., “WD4: Working Draft 4 of High-Efficiency Video Coding”, JCTVC-F803 d5, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG116th Meeting: Torino, IT, Jul. 14-22… [cited by applicant]
Document: JVET-K1001-v1, “Versatile Video Coding (Draft 21),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 11th Meeting: Ljubljana, SI, Jul. 10-18, 2018, 43 pages. [cited by applicant]
Document: JVET-K0325-v2, “On High Level Syntax Starting Point,” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 11th Meeting: Ljubljana, SI, Jul. 10-18, 2018, 6 pages. [cited by applicant]
Wahadaniah, V. et al., “AHG21: Construction and modification of predefined reference picture sets and reference picture lists,” JCTVC-G548, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC… [cited by applicant]
Deshpande, S., et al., “AHG21: Reference Picture Set Signaling Method,” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, Document: JCTVC-H0433, WG11 No. m23309, 8th Meeting… [cited by applicant]
Deshpande, S., “On Reference Pictures Signaling and Management for VVC,” Joint Video Experts Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, Document: JVET-10450-v2, 12th Meeting: Macao, CN, Oct. 3-12, 2018, 7… [cited by applicant]