IP Library › Granted Patent US 12,407,853
Granted Patent B2
US 12,407,853 · App. 18/386,778 · Granted Sep 2, 2025

Signaling of state information for a decoded picture buffer and reference picture lists

Inventors: Gary J. Sullivan (Bellevue, WA); Yongjun Wu (Bellevue, WA)
Assignee: Microsoft Technology Licensing, LLC
H04N19/58H04N19/423H04N19/46H04N19/573H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,407,853
App. No.
18/386,778
Granted
Sep 2, 2025
Kind
B2
Abstract

Innovations for signaling state of a decoded picture buffer (“DPB”) and reference picture lists (“RPLs”). In example implementations, rather than rely on internal state of a decoder to manage and update DPB and RPLs, state information about the DPB and RPLs is explicitly signaled. This permits a decoder to determine which pictures are expected to be available for reference from the signaled state information. For example, an encoder determines state information that identifies which pictures are available for use as reference pictures (optionally considering feedback information from a decoder about which pictures are available). The encoder sets syntax elements that represent the state information. In doing so, the encoder sets identifying information for a long-term reference picture (“LTRP”), where the identifying information is a value of picture order count least significant bits for the LTRB. The encoder then outputs the syntax elements as part of a bitstream.

Claims (36)

1. In a computer system that implements a video encoder, a method comprising:

determining to include long-term reference picture (“LTRP”) status information in a bitstream for pictures of a sequence;

outputting, as part of a sequence parameter set, a flag that indicates LTRP status information is present in the bitstream for the pictures of the sequence;

encoding a current picture among the pictures of the sequence, thereby producing coded data, including:

determining a set of reference pictures available for the current picture, the set of reference pictures including at least one LTRP;

determining LTRP status information for the current picture, wherein the LTRP status information for the current picture identifies which pictures, if any, among the set of reference pictures are available for use as LTRPs for the current picture; and

setting syntax elements that represent the LTRP status information for the current picture, including setting identifying information for a given LTRP in the LTRP status information for the current picture, wherein the identifying information for the given LTRP is a value of picture order count least significant bits (“POC LSBs”) for the given LTRP for the current picture, the value of the POC LSBs for the given LTRP, modulo a most significant bit wrapping point, at least in part indicating a difference between POC values of the current picture and the given LTRP; and

outputting the coded data as part of the bitstream, the coded data including the syntax elements that represent the LTRP status information for the current picture.

2. The method of claim 1 , wherein the syntax elements that represent the LTRP status information for the current picture are signaled in the bitstream for the current picture.

3. The method of claim 1 , wherein the setting the syntax elements that represent the LTRP status information for the current picture further includes:

setting a flag for the given LTRP in the LTRP status information for the current picture, the flag for the given LTRP indicating whether the given LTRP is used for decoding of the current picture.

4. The method of claim 1 , wherein the setting the syntax elements that represent the LTRP status information for the current picture further includes:

setting a flag for the given LTRP in the LTRP status information for the current picture, wherein the flag for the given LTRP facilitates reuse of the value of POC LSBs for the given LTRP as a value of POC LSBs for another reference picture while the given LTRP remains distinctly identified compared to the other reference picture.

5. The method of claim 1 , further comprising:

reusing the value of POC LSBs for the given LTRP as a value of POC LSBs for another reference picture, wherein the other reference picture is a short-term reference picture (“STRP”).

6. The method of claim 5 , further comprising:

marking the given LTRP as used for long-term reference to distinguish between the given LTRP and the STRP.

7. The method of claim 1 , wherein the determining the LTRP status information for the current picture and/or the setting the syntax elements is subject to a constraint that POC value for each picture that is in the set of reference pictures or waiting for output is computed unambiguously relative to the POC value of the current picture.

8. The method of claim 1 , wherein the determining the LTRP status information for the current picture and/or the setting the syntax elements is subject to a constraint on range of POC values for the current picture and the set of reference pictures.

9. A computer system comprising one or more processing units and memory, wherein the computer system implements a video decoder configured to perform operations comprising:

receiving a sequence parameter set;

parsing, from the sequence parameter set, a flag that indicates long-term reference picture (“LTRP”) status information is present in a bitstream for pictures of a sequence;

receiving coded data in the bitstream for a current picture among the pictures of the sequence; and

decoding the current picture, including determining a set of reference pictures available for the current picture, the set of reference pictures including at least one LTRP, wherein the decoding further includes:

parsing syntax elements from the bitstream, wherein the syntax elements represent LTRP status information for the current picture, wherein the LTRP status information for the current picture identifies which pictures, if any, among the set of reference pictures are available for use as LTRPs for the current picture, the syntax elements including identifying information for a given LTRP in the LTRP status information for the current picture, and wherein the identifying information for the given LTRP is a value of picture order count least significant bits (“POC LSBs”), modulo a most significant bit wrapping point, for the given LTRP for the current picture; and

using the LTRP status information to identify which pictures, if any, among the set of reference pictures are available for use as LTRPs for the current picture, wherein the value of the POC LSBs for the given LTRP, modulo the most significant bit wrapping point, is used to at least in part indicate a difference between POC values of the current picture and the given LTRP.

10. The computer system of claim 9 , wherein the syntax elements that represent the LTRP status information for the current picture are signaled in the bitstream for the current picture.

11. The computer system of claim 9 , wherein the operations further comprise:

using the LTRP status information to manage a decoded picture storage area of a video decoder.

12. The computer system of claim 9 , wherein the syntax elements further include a flag for the given LTRP, the flag for the given LTRP indicating whether the given LTRP is used for decoding of the current picture.

13. The computer system of claim 9 , wherein the operations further comprise:

sending feedback information for use by a video encoder during encoding, wherein the feedback information represents decoder state information that identifies which pictures are available for use as reference pictures by a video decoder.

14. The computer system of claim 9 , wherein the operations further comprise:

reusing the value of POC LSBs for the given LTRP as a value of POC LSBs for another reference picture, wherein the other reference picture is a short-term reference picture (“STRP”).

15. The computer system of claim 14 , wherein the operations further comprise:

marking the given LTRP as used for long-term reference to distinguish between the given LTRP and the STRP.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 14, 2023
From: SULLIVAN, GARY J.; WU, YONGJUN
To: MICROSOFT CORPORATION
Reel/Frame 065558/0704 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 14, 2023
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 065558/0726 →
Continuity (7)
Continuation 17850622 · Jun 27, 2022
Continuation 17112304 · Dec 4, 2020
Continuation 16546871 · Aug 21, 2019
Continuation 15952796 · Apr 13, 2018
Continuation 13669380 · Nov 5, 2012
Provisional Application 61556813 · Nov 7, 2011
Related Publication 20240080475A1 · Mar 7, 2024
References Cited (66)
US 6956545B2 · McClintock · 2005 [cited by examiner]
US 9313500B2 · Sullivan et al. · 2016 [cited by applicant]
US 9762928B2 · Sullivan et al. · 2017 [cited by applicant]
US 10003817B2 · Sullivan et al. · 2018 [cited by applicant]
US 10165302B2 · Sullivan et al. · 2018 [cited by applicant]
US 10432964B2 · Sullivan et al. · 2019 [cited by applicant]
US 10560720B2 · Sullivan et al. · 2020 [cited by applicant]
US 10924760B2 · Sullivan et al. · 2021 [cited by applicant]
US 11012709B2 · Sullivan et al. · 2021 [cited by applicant]
US 11405636B2 · Sullivan et al. · 2022 [cited by applicant]
US 11418809B2 · Sullivan et al. · 2022 [cited by applicant]
US 11770553B2 · Sullivan et al. · 2023 [cited by applicant]
US 11770554B2 · Sullivan et al. · 2023 [cited by applicant]
US 11778229B2 · Sullivan et al. · 2023 [cited by applicant]
US 11849144B2 · Sullivan et al. · 2023 [cited by applicant]
US 12003753B2 · Lim · 2024 [cited by examiner]
US 20090238269A1 · Pandit · 2009 [cited by examiner]
US 20090262804A1 · Pandit · 2009 [cited by examiner]
US 20100189173A1 · Chen · 2010 [cited by examiner]
US 20100195721A1 · Wu · 2010 [cited by examiner]
US 20120294366A1 · Eliyahu · 2012 [cited by examiner]
US 20130003823A1 · Misra · 2013 [cited by examiner]
US 20130077679A1 · Wang · 2013 [cited by examiner]
US 20130077687A1 · Wang · 2013 [cited by examiner]
US 20130094585A1 · Misra · 2013 [cited by examiner]
US 20130094772A1 · Deshpande · 2013 [cited by examiner]
US 20130114741A1 · Sullivan et al. · 2013 [cited by applicant]
US 20130215975A1 · Samuelsson · 2013 [cited by examiner]
US 20140092991A1 · Sullivan et al. · 2014 [cited by applicant]
US 20140126640A1 · Samuelsson · 2014 [cited by examiner]
US 20150016505A1 · Sjoberg et al. · 2015 [cited by applicant]
US 20190116379A1 · Sullivan et al. · 2019 [cited by applicant]
US 20230412836A1 · Sullivan et al. · 2023 [cited by applicant]
KR 102344167 · 2021 [cited by applicant]
WO WO2013109183 · 2013 [cited by applicant]
Notice of Allowance dated Oct. 4, 2023, from U.S. Appl. No. 17/850,622, 11 pp. [cited by applicant]
Notice of Preliminary Rejection dated Feb. 6, 2024, from Korean Patent Application No. 10-2023-7020078, 13 pp. [cited by applicant]
Office Action dated Mar. 28, 2024, from U.S. Appl. No. 18/232,193, 7 pp. [cited by applicant]
Communication under Rule 71(3) EPC dated Jun. 7, 2024, from European Patent Application No. 21155916.6, 8 pp. [cited by applicant]
Decision to Grant dated Oct. 10, 2024, from European Patent Application No. 21155916.6, 2 pp. [cited by applicant]
Office Action dated Jul. 30, 2024, from U.S. Appl. No. 18/232,193, 8 pp. [cited by applicant]
Bross et al., “High Efficiency Video Coding (HEVC) Text Specification Draft 6,” JCTVC-H1003, 259 pp. (Feb. 2012). [cited by applicant]
Bross et al., “High Efficiency Video Coding (HEVC) Text Specification Draft 8,” JCTVC-J1003_d0, 284 pp. (Jul. 2012). [cited by applicant]
Communication Pursuant to Rules 70(2) and 70a(2) EPC and reference to Rule 39(1) EPC dated Jan. 27, 2025, from European Patent Application No. 24201660.8, 2 pp. [cited by applicant]
Communication pursuant to Rules 70(2) and 70a(2) EPC and reference to Rule 39(1) EPC dated Feb. 3, 2025, from European Patent Application No. 24201657.4, 2 pp. [cited by applicant]
Decision to Grant dated Aug. 23, 2018, from European Patent Application No. 12848102.5, 1 p. [cited by applicant]
Decision to Grant dated Feb. 4, 2021, from European Patent Application No. 13774010.6, 2 pp. [cited by applicant]
Examination Report dated Mar. 30, 2016, from Australian Patent Application No. 2012336043, 4 pp. [cited by applicant]
Extended European Search Report dated Dec. 23, 2024, from European Patent Application No. 24201660.8, 9 pp. [cited by applicant]
Extended European Search Report dated Jan. 8, 2025, from European Patent Application No. 24201657.4, 7 pp. [cited by applicant]
Final Office Action dated Jan. 24, 2025, from U.S. Appl. No. 18/232,193, 9 pp. [cited by applicant]
Intimation to Grant for Indian Application No. 3320/CHENP/2014, mailed on Sep. 3, 2021, 1 page. [cited by applicant]
Office Action dated Apr. 25, 2016, from U.S. Appl. No. 15/064,299, 16 pp. [cited by applicant]
Office Action dated Mar. 28, 2018, from U.S. Appl. No. 15/655,797, 8 pp. [cited by applicant]
Office Action dated Jul. 20, 2015, from U.S. Appl. No. 13/781,710, 15 pp. [cited by applicant]
Notice of Allowance dated Apr. 29, 2020, from Argentina Patent Application No. 20130103543, 2 pp. [cited by applicant]
Notice of Allowance dated Aug. 14, 2018, from U.S. Appl. No. 15/655,797, 8 pp. [cited by applicant]
Notice of Allowance dated Dec. 7, 2015, from U.S. Appl. No. 13/781,710, 7 pp. [cited by applicant]
Notice of Allowance dated Nov. 1, 2016, from U.S. Appl. No. 15/064,299, 8 pp. [cited by applicant]
Notice of Allowance dated Apr. 18, 2017, from U.S. Appl. No. 15/064,299, 9 pp. [cited by applicant]
Notice of Allowance dated Apr. 6, 2018, from U.S. Appl. No. 13/669,380, 11 pp. [cited by applicant]
Notice of Grant dated Dec. 4, 2017, from Chinese Patent Application No. 201280054626.X, 4 pp. [cited by applicant]
Notification of Reasons for Refusal dated May 16, 2017, from Japanese Patent Application No. 2015/534584, 5 pp. [cited by applicant]
Notice of Preliminary Rejection dated Apr. 10, 2019, from Korean Patent Application No. 10-2014-7012129, 9 pp. [cited by applicant]
Notice of Preliminary Rejection dated Feb. 10, 2025, from Korean Patent Application No. 10-2023-7020078, 17 pp. [cited by applicant]
Office Action and Search Report dated Jul. 17, 2017, from Taiwanese Patent Application No. 102135102, 7 pp. [No English translation available]. [cited by applicant]