IP Library › Granted Patent US 9,369,710

Granted Patent B2

US 9,369,710 · App. 13/759,905 · Granted Jun 14, 2016

Reference picture list modification for video coding

Inventors: Ying Chen (San Diego, CA); Ye-Kui Wang (San Diego, CA); Adarsh Krishnan Ramasubramonian (San Diego, CA)

Assignee: QUALCOMM Incorporated

H04N19/00781 H04N19/105 H04N19/172 H04N19/174 H04N19/46 H04N19/573 H04N19/70

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 9,369,710
App. No.: 13/759,905
Granted: Jun 14, 2016
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

A video coder may, in some cases, signal whether one or more initial reference picture lists are to be modified. When an initial list is to be modified, the video coder can signal information indicating a starting position in the initial reference picture list. When the starting position signaled by the video coder is less than a number of pictures included in the initial reference picture list, then the video coder signals the number of pictures to be inserted into the initial reference picture list, and a reference picture source from which a picture can be retrieved to insert into the initial reference picture list to construct a modified reference picture list.

Claims (157)

1. A method for encoding video data, the method comprising:

constructing, for encoding a current picture of the video data, a plurality of reference picture subsets that together form a reference picture set, wherein the reference picture set identifies reference pictures that can be used for inter-predicting one or more pictures included in the video data, wherein the reference picture subsets includes at least two of:

a RefPicSetStCurrBefore subset, a RefPicSetStCurrAfter subset, a RefPicSetStFoll subset, a RefPicSetLtCurr subset, or a RefPicSetLtFoll subset,

wherein the RefPicSetStCurrBefore subset includes short-term reference pictures that are displayed before the current picture being encoded and can be used to inter-predict the current picture and pictures following the current picture in decoding order,

wherein the RefPicSetStCurrAfter subset includes short-term reference pictures that are displayed after the current picture being encoded and can be used to inter-predict the current picture and pictures following the current picture in decoding order,

wherein the RefPicSetStFoll subset includes short-term reference pictures that can be used to inter-predict pictures following the current picture in decoding order and not the current picture being encoded, wherein the RefPicSetLtCurr subset includes long-term reference pictures that can be used to inter-predict the current picture and pictures following the current picture in decoding order, and

wherein the RefPicSetLtFoll subset includes long-term reference pictures that can be used to inter-predict pictures following the current picture in decoding order and not the current picture;

constructing a plurality of reference picture supersets, wherein each of the reference picture supersets is a combination of two or more of the reference picture subsets;

constructing an initial reference picture list, different than the plurality of reference picture supersets, based on one or more of the reference picture subsets;

coding information indicating a starting position in the initial reference picture list;

coding information indicating a number of pictures to be inserted into the initial reference picture list;

coding information indicating a selected reference picture superset from the plurality of reference picture supersets and an index into the selected reference picture superset from which a picture can be retrieved for insertion into the initial reference picture list to construct a modified reference picture list; and

encoding the current picture of the video data based on the modified reference picture list.

2. The method of claim 1 , further comprising coding information indicating whether the initial reference picture list is to be modified, and wherein the starting position indicates a position at which to begin modification of the initial reference picture list.

3. The method of claim 1 , further comprising coding information indicating a number of times to repeatedly:

identify a reference picture in the selected reference picture superset; and

associate the identified reference picture with a respective index of the initial reference picture list to construct the modified reference picture list.

4. The method of claim 3 , wherein coding the information indicating the number of times to repeatedly identify the reference picture and associating the identified reference picture comprises coding a pics_inserted_flag syntax element that indicates that the number of times to repeatedly identify the reference picture and associating the identified reference picture is less than a number of pictures in the initial reference picture list.

5. The method of claim 1 , wherein coding the information indicating the starting position comprises coding a start_pos syntax element that indicates an index position at which to begin modification of the initial reference picture list.

6. The method of claim 1 ,

wherein coding the information indicating a number of pictures to be inserted into the initial reference picture list comprises coding a num_inserted_pics syntax element that indicates the number of pictures to be inserted into the initial reference picture list, and

wherein coding the information indicating the selected reference picture superset and the index into the selected reference picture superset comprises:

coding a source_list_idx syntax element that indicates the selected reference picture superset from which the picture can be retrieved; and

coding a ref_pic_set_idx syntax element that indicates the index into the selected reference picture superset at which the picture can be retrieved for insertion into the initial reference picture list to construct the modified reference picture list.

7. The method of claim 6 , wherein coding at least one of the source_list_idx and the ref_pic_set_idx syntax element comprises coding at least one of the source_list_idx and the ref_pic_set_idx syntax element as ur(v) representing an unsigned integer using v bits, wherein v varies based on a range of values for the at least one of the source_list_idx and the ref_pic_set_idx syntax element.

8. A method for decoding video data, the method comprising:

constructing, for decoding a current picture of the video data, a plurality of reference picture subsets that together form a reference picture set, wherein the reference picture set identifies reference pictures that can be used for inter-predicting one or more pictures included in the video data, wherein the reference picture subsets includes at least two of:

a RefPicSetStCurrBefore subset, a RefPicSetStCurrAfter subset, a RefPicSetStFoll subset, a RefPicSetLtCurr subset, or a RefPicSetLtFoll subset,

wherein the RefPicSetStCurrBefore subset includes short-term reference pictures that are displayed before the current picture being decoded and can be used to inter-predict the current picture and pictures following the current picture in decoding order,

wherein the RefPicSetStCurrAfter subset includes short-term reference pictures that are displayed after the current picture being decoded and can be used to inter-predict the current picture and pictures following the current picture in decoding order,

wherein the RefPicSetLtCurr subset includes long-term reference pictures that can be used to inter-predict the current picture and pictures following the current picture in decoding order, and

wherein the RefPicSetLtFoll subset includes long-term reference pictures that can be used to inter-predict pictures following the current picture in decoding order and not the current picture;

constructing a plurality of reference picture supersets, each of the reference picture supersets is a combination of two or more of the reference picture subsets;

constructing an initial reference picture list, wherein different than the plurality of reference picture supersets, based on one or more of the reference picture subsets;

setting a current index into the initial reference picture list based on a coded starting position indicating a position at which to begin modification of the initial reference picture list;

determining a number of pictures to be inserted into the initial reference picture list;

determining a selected reference picture superset from the plurality of reference picture supersets based on decoding of information indicating the selected reference picture superset;

identifying, based on the determined number of pictures, one or more reference pictures in the selected reference picture superset using one or more indices into the selected reference picture superset;

associating the identified reference pictures with respective indices of the initial reference picture list, including the current index, to construct the modified reference picture list; and

decoding the current picture of the video data based on the modified reference picture list.

9. The method of claim 8 , wherein constructing the plurality of reference picture supersets comprises:

combining the RefPicSetStCurrBefore subset and the RefPicSetStCurrAfter subset of the reference picture set to construct a first reference picture superset; and

constructing a second reference picture superset based on an inter-view reference picture subset of the reference picture set.

10. The method of claim 8 , wherein constructing one of the plurality of reference picture supersets comprises:

combining the RefPicSetStCurrBefore subset, the RefPicSetStCurrAfter subset, the RefPicSetStFoll subset, the RefPicSetLtCurr subset, and the RefPicSetLtFoll subset of the reference picture set to construct a first candidate reference picture superset.

11. The method of claim 8 , further comprising truncating the modified reference picture list such that any pictures associated with an index that is greater than a maximum number of pictures allowed in a reference picture list are removed from the modified reference picture list.

12. A video encoding device comprising:

a memory unit configured to store video data; and

a video encoder configured to:

construct, for encoding a current picture of the video data, a plurality of reference picture subsets that together form a reference picture set, wherein the reference picture set identifies reference pictures that can be used for inter-predicting one or more pictures included in the video data, wherein the reference picture subsets includes at least two of:

a RefPicSetStCurrBefore subset, a RefPicSetStCurrAfter subset, a RefPicSetStFoll subset, a RefPicSetLtCurr subset, or a RefPicSetLtFoll subset,

wherein the RefPicSetLtCurr subset includes long-term reference pictures that can be used to inter-predict the current picture and pictures following the current picture in decoding order, and

wherein the RefPicSetLtFoll subset includes long-term reference pictures that can be used to inter-predict pictures following the current picture in decoding order and not the current picture;

construct a plurality of reference picture supersets, wherein each of the reference picture supersets is a combination of two or more of the reference picture subsets;

construct an initial reference picture list, different than the plurality of reference picture supersets, based on one or more of the reference picture subsets;

code information indicating a starting position in an initial reference picture list;

code information indicating a number of pictures to be inserted into the initial reference picture list;

code information indicating a selected reference picture superset from the plurality of reference picture supersets and an index into the selected reference picture superset from which a picture can be retrieved for insertion into the initial reference picture list to construct a modified reference picture list; and

encode the current picture of the video data based on the modified reference picture list.

13. The video encoding device of claim 12 , wherein the video encoder is configured to code information indicating a number of times to repeatedly:

identify a reference picture in the selected reference picture superset; and

associate the identified reference picture with a respective index of the initial reference picture list to construct the modified reference picture list.

14. The video encoding device of claim 13 , wherein the video encoder is configured to code a pics_inserted_flag syntax element that indicates that the number of times to repeatedly identify the reference picture and associating the identified reference picture is less than a number of pictures in the initial reference picture list.

15. The video encoding device of claim 12 , wherein the video encoder is configured to code a start_pos syntax element that indicates an index position at which to begin modification of the initial reference picture list.

16. The video encoding device of claim 12 , wherein the video encoder is configured to:

code a num_inserted_pics syntax element that indicates the number of pictures to be inserted into the initial reference picture list; and

code a source_list_idx syntax element that indicates the selected reference picture superset from which the picture can be retrieved; and

code a ref_pic_set_idx syntax element that indicates the index into the selected reference picture superset at which the picture can be retrieved for insertion into the initial reference picture list to construct the modified reference picture list.

17. The video encoding device of claim 16 , wherein the video encoder is configured to code at least one of the source_list_idx and the ref_pic_set_idx syntax element as ur(v) representing an unsigned integer using v bits, wherein v varies based on a range of values for the at least one of the source_list_idx and the ref_pic_set_idx syntax element.

18. A video decoding device comprising:

a memory unit configured to store video data; and

a video decoder configured to:

construct, for decoding a current picture of the video data, a plurality of reference picture subsets that together form a reference picture set, wherein the reference picture set identifies reference pictures that can be used for inter-predicting one or more pictures included in the video data, wherein the reference picture subsets includes at least two of:

a RefPicSetStCurrBefore subset, a RefPicSetStCurrAfter subset, a RefPicSetStFoll subset, a RefPicSetLtCurr subset, or a RefPicSetLtFoll subset,

wherein the RefPicSetLtCurr subset includes long-term reference pictures that can be used to inter-predict the current picture and pictures following the current picture in decoding order, and

wherein the RefPicSetLtFoll subset includes long-term reference pictures that can be used to inter-predict pictures following the current picture in decoding order and not the current picture;

construct a plurality of reference picture supersets, wherein each of the reference picture supersets is a combination of two or more of the reference picture subsets;

construct an initial reference picture list, different than the plurality of reference picture supersets, based on one or more of the reference picture subsets;

set a current index into the initial reference picture list based on a coded starting position indicating a position at which to begin modification of the initial reference picture list;

determine a number of pictures to be inserted into the initial reference picture list;

determine a selected reference picture superset from the plurality of reference picture supersets based on decoding of information indicating the selected reference picture superset;

identify, based on the determined number of pictures, one or more reference pictures in the selected reference picture superset using one or more indices into the selected reference picture superset;

associate the identified reference pictures with respective indices of the initial reference picture list, including the current index, to construct the modified reference picture list; and

decode the current picture of the video data based on the modified reference picture list.

19. The video decoding device of claim 18 , wherein the video decoder is configured to construct the plurality of reference picture supersets at least by:

combining the RefPicSetStCurrBefore subset and the RefPicSetStCurrAfter subset of the reference picture set to construct a first candidate reference picture superset; and

constructing a second candidate reference picture superset based on an inter-view reference picture subset of the reference picture set.

20. The video decoding device of claim 18 , wherein the video decoder is configured to construct one of the plurality of reference picture supersets at least by:

21. The video decoding device of claim 18 , wherein the video decoder is configured to truncate the modified reference picture list such that any pictures associated with an index that is greater than a maximum number of pictures allowed in a reference picture list are removed from the modified reference picture list.

22. A video encoding device comprising:

means for constructing, for encoding a current picture of video data, a plurality of reference picture subsets that together form a reference picture set, wherein the reference picture set identifies reference pictures that can be used for inter-predicting one or more pictures included in the video data, wherein the reference picture subsets includes at least two of:

a RefPicSetStCurrBefore subset, a RefPicSetStCurrAfter subset, a RefPicSetStFoll subset, a RefPicSetLtCurr subset, or a RefPicSetLtFoll subset,

wherein the RefPicSetLtCurr subset includes long-term reference pictures that can be used to inter-predict the current picture and pictures following the current picture in decoding order, and

wherein the RefPicSetLtFoll subset includes long-term reference pictures that can be used to inter-predict pictures following the current picture in decoding order and not the current picture;

means for constructing a plurality of reference picture supersets, wherein each of the reference picture supersets is a combination of two or more of the reference picture subsets;

means for constructing an initial reference picture list, different than the plurality of reference picture supersets, based on one or more of the reference picture subsets;

means for coding information indicating a starting position in the initial reference picture list;

means for coding information indicating a number of pictures to be inserted into the initial reference picture list;

means for coding information indicating a selected reference picture superset from the plurality of reference picture supersets and an index into the selected reference picture superset from which a picture can be retrieved for insertion into the initial reference picture list to construct a modified reference picture list; and

means for encoding the current picture of the video data based on the modified reference picture list.

23. A video decoding device comprising:

means for constructing, for decoding a current picture of video data, a plurality of reference picture subsets that together form a reference picture set, wherein the reference picture set identifies reference pictures that can be used for inter-predicting one or more pictures included in the video data, wherein the reference picture subsets includes at least two of:

a RefPicSetStCurrBefore subset, a RefPicSetStCurrAfter subset, a RefPicSetStFoll subset, a RefPicSetLtCurr subset, or a RefPicSetLtFoll subset,

wherein the RefPicSetLtCurr subset includes long-term reference pictures that can be used to inter-predict the current picture and pictures following the current picture in decoding order, and

wherein the RefPicSetLtFoll subset includes long-term reference pictures that can be used to inter-predict pictures following the current picture in decoding order and not the current picture;

means for constructing a plurality of reference picture supersets, wherein each of the reference picture supersets is a combination of two or more of the reference picture subsets;

means for constructing an initial reference picture list, different than the plurality of reference picture supersets, based on one or more of the reference picture subsets;

means for setting a current index into the initial reference picture list based on a coded starting position indicating a position at which to begin modification of the initial reference picture list;

means for determining a number of pictures to be inserted into the initial reference picture list;

means for determining a selected reference picture superset from the plurality of reference picture supersets based on decoding of information indicating the selected reference picture superset;

means for identifying, based on the determined number of pictures, one or more reference pictures in the selected reference picture superset using one or more indices into the selected reference picture superset;

means for associating the identified reference pictures with the respective indices of the initial reference picture list, including the current index, to construct the modified reference picture list; and

means for decoding the current picture of the video data based on the modified reference picture list.

24. A non-transitory computer-readable storage medium having stored thereon instructions that when executed cause one or more processors to:

construct, for encoding a current picture of video data, a plurality of reference picture subsets that together form a reference picture set, wherein the reference picture set identifies reference pictures that can be used for inter-predicting one or more pictures included in the video data, wherein the reference picture subsets includes at least two of:

a RefPicSetStCurrBefore subset, a RefPicSetStCurrAfter subset, a RefPicSetStFoll subset, a RefPicSetLtCurr subset, or a RefPicSetLtFoll subset,

wherein the RefPicSetLtCurr subset includes long-term reference pictures that can be used to inter-predict the current picture and pictures following the current picture in decoding order, and

wherein the RefPicSetLtFoll subset includes long-term reference pictures that can be used to inter-predict pictures following the current picture in decoding order and not the current picture;

construct a plurality of reference picture supersets, wherein each of the reference picture supersets is a combination of two or more of the reference picture subsets;

construct an initial reference picture list, different than the plurality of reference picture supersets, based on one or more of the reference picture subsets;

code information indicating a starting position in the initial reference picture list;

code information indicating a number of pictures to be inserted into the initial reference picture list;

encode the current picture of the video data based on the modified reference picture list.

25. A non-transitory computer-readable storage medium having stored thereon instructions that when executed cause one or more processors to:

construct, for decoding a current picture of video data, a plurality of reference picture subsets that together form a reference picture set, wherein the reference picture set identifies reference pictures that can be used for inter-predicting one or more pictures included in the video data, wherein the reference picture subsets includes at least two of: