IP Library Granted Patent US 11,770,553
Granted Patent B2
US 11,770,553 · App. 17/845,346 · Granted Sep 26, 2023

Conditional signalling of reference picture list modification information

Inventors: Gary J. Sullivan (Bellevue, WA); Sandeep Kanumuri (Redmond, WA)
Assignee: Microsoft Technology Licensing, LLC
H04N19/58H04N19/105H04N19/109H04N19/134H04N19/46H04N19/503H04N19/573H04N19/70H04N19/85
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,770,553
App. No.
17/845,346
Granted
Sep 26, 2023
Kind
B2
Abstract

Innovations in signaling of reference picture list (“RPL”) modification information. For example, a video encoder evaluates a condition that depends at least in part on a variable indicating a number of total reference pictures. Depending on the results of the evaluation, the encoder signals in a bitstream a flag that indicates whether an RPL is modified according to syntax elements explicitly signaled in the bitstream. A video decoder evaluates the condition and, depending on results of the evaluation, parses from a bitstream a flag that indicates whether an RPL is modified according to syntax elements explicitly signaled in the bitstream. The condition can be evaluated as part of processing for an RPL modification structure that includes the flag, or as part of processing for a slice header. The encoder and decoder can also evaluate other conditions that affect syntax elements for list entries of the RPL modification information.

Claims (61)

1. One or more computer-readable media having stored thereon computer-executable instructions for causing one or more processing units, when programmed thereby, to perform operations, the one or more computer-readable media being selected from the group consisting of volatile memory, non-volatile memory, and storage media, the operations comprising:

receiving video; and

encoding the video to produce encoded data in a bitstream, including:

determining a value of a variable that indicates a number of total reference pictures;

as part of processing for a slice header of a current slice, determining that a condition is satisfied, wherein the condition depends at least in part on the value of the variable that indicates the number of total reference pictures; and

responsive to the condition being satisfied:

signaling in the bitstream a first flag that indicates whether a first reference picture list (“RPL”) is modified according to first syntax elements explicitly signaled in the bitstream;

determining that the current slice is a B slice; and

responsive to the current slice being a B slice, signaling in the bitstream a second flag that indicates whether a second RPL is modified according to second syntax elements explicitly signaled in the bitstream.

2. The one or more computer-readable media of claim 1 , wherein the first flag and the second flag are signaled in an RPL modification structure.

3. The one or more computer-readable media of claim 1 , wherein:

if the first flag indicates the first RPL is modified, the first syntax elements, which are explicitly signaled in the bitstream, specify multiple list entries of the first RPL; and

otherwise, the first flag indicating the first RPL is not modified, the first syntax elements are missing from the bitstream.

4. The one or more computer-readable media of claim 3 , wherein:

if the second flag indicates the second RPL is modified, the second syntax elements, which are explicitly signaled in the bitstream, specify multiple list entries of the second RPL; and

otherwise, the second flag indicating the second RPL is not modified, the second syntax elements are missing from the bitstream.

5. The one or more computer-readable media of claim 1 , wherein the encoding the video further includes:

signaling in the bitstream another flag that indicates whether an RPL modification structure for the slice may be present in the bitstream.

6. The one or more computer-readable media of claim 5 , wherein the other flag is signaled as part of a picture parameter set.

7. The one or more computer-readable media of claim 1 , wherein the encoding the video further includes:

evaluating results of motion compensation for which the first RPL is modified according to the first syntax elements;

evaluating results of motion compensation for which the first RPL is not modified according to the first syntax elements; and

performing one or more of:

modifying the first RPL so as to accomplish reordering one or more reference pictures for more efficient addressing with reference indices;

modifying the first RPL so as to accomplish removing one or more reference pictures based at least in part on frequency of use during encoding; and

modifying the first RPL so as to accomplish adding one or more reference pictures based at least in part on frequency of use during encoding.

8. The one or more computer-readable media of claim 1 , wherein the encoding the video further includes:

evaluating results of motion compensation for which the second RPL is modified according to the second syntax elements;

evaluating results of motion compensation for which the second RPL is not modified according to the second syntax elements; and

performing one or more of:

modifying the second RPL so as to accomplish reordering one or more reference pictures for more efficient addressing with reference indices;

modifying the second RPL so as to accomplish removing one or more reference pictures based at least in part on frequency of use during encoding; and

modifying the second RPL so as to accomplish adding one or more reference pictures based at least in part on frequency of use during encoding.

9. The one or more computer-readable media of claim 1 , wherein the signaling in the bitstream the first flag is performed conditionally responsive to determining, as part of processing for the slice header of the current slice, that the condition is satisfied, and wherein the signaling in the bitstream the second flag is performed conditionally responsive to the current slice being a B slice and responsive to the determining, as part of processing for the slice header of the current slice, that the condition is satisfied.

10. One or more computer-readable media having stored thereon computer-executable instructions for causing one or more processing units, when programmed thereby, to perform operations, the one or more computer-readable media being selected from the group consisting of volatile memory, non-volatile memory, and storage media, the operations comprising:

receiving encoded data in a bitstream; and

decoding the encoded data to produce reconstructed video frames, including:

as part of processing for a slice header of a slice, evaluating a condition, wherein the condition depends at least in part on a value of a variable that indicates a number of total reference pictures; and

depending on whether the condition is satisfied, conditionally parsing from the bitstream:

(a) a first flag that indicates whether a first reference picture list (“RPL”) is modified according to syntax elements explicitly signaled in the bitstream; and

(b) if slice type of the slice is B, a second flag that indicates whether a second RPL is modified according to syntax elements explicitly signaled in the bitstream.

11. The one or more computer-readable media of claim 10 , wherein the first flag and, if the slice type of the slice is B, the second flag are conditionally parsed from an RPL modification structure.

12. The one or more computer-readable media of claim 10 , wherein an RPL modification structure including the first flag and, if the slice type of the slice is B, the second flag is conditionally parsed depending on whether the condition is satisfied.

13. The one or more computer-readable media of claim 10 , wherein a current picture includes the slice, the number of total reference pictures accounting for short-term reference pictures used in decoding for the current picture and accounting for long-term reference pictures used in decoding for the current picture.

14. The one or more computer-readable media of claim 10 , wherein the syntax elements explicitly signaled in the bitstream specify multiple list entries of the first RPL and, if the slice type of the slice is B, multiple list entries of the second RPL.

15. The one or more computer-readable media of claim 10 , wherein the decoding the encoded data further includes parsing from the bitstream another flag, the other flag indicating whether an RPL modification structure may be present in the bitstream.

16. The one or more computer-readable media of claim 10 , wherein the conditionally parsing is performed responsive to determining, as part of processing for the slice header of the slice, that the condition is satisfied.

17. One or more computer-readable media having stored thereon encoded data in a bitstream, the encoded data having been generated by operations, the one or more computer-readable media being selected from the group consisting of volatile memory, non-volatile memory, and storage media, the operations comprising:

determining a value of a variable that indicates a number of total reference pictures;

as part of processing for a slice header of a current slice, determining that a condition is satisfied, wherein the condition depends at least in part on the value of the variable that indicates the number of total reference pictures; and

responsive to the condition being satisfied:

signaling in the bitstream a first flag that indicates whether a first reference picture list (“RPL”) is modified according to first syntax elements explicitly signaled in the bitstream;

determining that the current slice is a B slice; and

responsive to the current slice being a B slice, signaling in the bitstream a second flag that indicates whether a second RPL is modified according to second syntax elements explicitly signaled in the bitstream.

18. The one or more computer-readable media of claim 17 , wherein the first flag and the second flag are signaled in an RPL modification structure.

19. The one or more computer-readable media of claim 17 , wherein:

if the first flag indicates the first RPL is modified, the first syntax elements, which are explicitly signaled in the bitstream, specify multiple list entries of the first RPL;

otherwise, the first flag indicating the first RPL is not modified, the first syntax elements are missing from the bitstream;

if the second flag indicates the second RPL is modified, the second syntax elements, which are explicitly signaled in the bitstream, specify multiple list entries of the second RPL; and

otherwise, the second flag indicating the second RPL is not modified, the second syntax elements are missing from the bitstream.

20. The one or more computer-readable media of claim 17 , wherein the signaling in the bitstream the first flag is performed conditionally responsive to determining, as part of processing for the slice header of the current slice, that the condition is satisfied, and wherein the signaling in the bitstream the second flag is performed conditionally responsive to the current slice being a B slice and responsive to determining, as part of processing for the slice header of the current slice, that the condition is satisfied.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 14, 2022
From: SULLIVAN, GARY J.; KANUMURI, SANDEEP
To: MICROSOFT CORPORATION
Reel/Frame 062086/0413 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 14, 2022
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 062086/0452 →
Cited By (2)
US 12,407,853 US 12,464,163