IP Library Granted Patent US 11,303,933
Granted Patent B2
US 11,303,933 · App. 16/588,992 · Granted Apr 12, 2022

Constraints and unit types to simplify video random access

Inventors: Gary J. Sullivan (Bellevue, WA); Sandeep Kanumuri (Redmond, WA)
Assignee: Microsoft Technology Licensing, LLC
H04N19/65H04N19/15H04N19/573H04N19/70H04N19/88H04N19/172H04N19/184
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,303,933
App. No.
16/588,992
Granted
Apr 12, 2022
Kind
B2
Abstract

Disclosed herein are innovations for bitstreams having clean random access (CRA) pictures and/or other types of random access point (RAP) pictures. New type definitions and strategic constraints on types of RAP pictures can simplify mapping of units of elementary video stream data to a container format. Such innovations can help improve the ability for video coding systems to more flexibly perform adaptive video delivery, production editing, commercial insertion, and the like.

Claims (32)

1. A computer-readable storage medium storing computer-executable instructions, which when executed by a computer cause the computer to perform operations, the operations comprising:

generating at least a portion of a bitstream comprising encoded data for plural pictures, the plural pictures comprising a random access point (RAP) picture, one or more associated potentially non-decodable leading pictures, and one or more non-leading pictures associated with the RAP picture, the RAP picture being at a bitstream order position after an initial picture of the bitstream, the at least a portion of the bitstream further comprising an indication signaling that the RAP picture is a picture from which a decoder can begin decoding, wherein the generating the at least a portion of the bitstream includes ordering the one or more leading pictures and the one or more non-leading pictures associated with the RAP picture such that all of the one or more leading pictures precede, in decoding order, the one or more non-leading pictures associated with the RAP picture; and

outputting the at least a portion of the bitstream.

2. The computer-readable storage medium of claim 1 , wherein the indication signaling that the RAP picture is a picture from which the decoder can begin decoding is a syntax element signaling that the RAP picture is one of a plurality of RAP picture types.

3. The computer-readable storage medium of claim 2 , wherein the plurality of RAP picture types include a broken link access (BLA) picture type and a clean random access (CRA) picture type.

4. The computer-readable storage medium of claim 2 , wherein the syntax element indicates a network abstraction layer (NAL) unit type value for a NAL unit that includes at least some encoded data for the RAP picture.

5. The computer-readable storage medium of claim 1 , wherein the operations further comprise:

receiving the at least a portion of a bitstream; and

decoding the RAP picture such that the RAP picture is the first picture decoded.

6. The computer-readable storage medium of claim 1 , wherein the at least a portion of the bitstream further includes, for each of the one or more potentially non-decodable leading pictures, an indication that the potentially non-decoding leading picture is a random access skipped leading (RASL) picture.

7. The computer-readable storage medium of claim 6 , wherein the indication that the potentially non-decoding leading picture is a RASL picture is a syntax element that indicates a network abstraction layer (NAL) unit type value for a NAL unit that includes at least some encoded data for the RASL picture.

8. The computer-readable storage medium of claim 1 , wherein the one or more leading pictures are potentially non-decodable when decoding begins at the RAP picture due to missing reference pictures for the one or more leading pictures when decoding begins at the RAP picture.

9. In a computer system that implements a video decoder, a method comprising:

receiving at least a portion of a bitstream comprising encoded data for plural pictures, the plural pictures comprising a random access point (RAP) picture, one or more associated potentially non-decodable leading pictures, and one or more non-leading pictures associated with the RAP picture, the RAP picture being at a bitstream order position after an initial picture of the bitstream, the at least a portion of the bitstream further comprising an indication signaling that the RAP picture is a picture from which the video decoder can begin decoding, the at least a portion of the bitstream having been generated by operations that include ordering the one or more leading pictures and the one or more non-leading pictures associated with the RAP picture such that all of the one or more leading pictures precede, in decoding order, the one or more non-leading pictures associated with the RAP picture; and

decoding the at least a portion of the bitstream.

10. The method of claim 9 , wherein the indication signaling that the RAP picture is a picture from which the video decoder can begin decoding is a syntax element signaling that the RAP picture is one of a plurality of RAP picture types.

11. The method of claim 10 , wherein the plurality of RAP picture types include a broken link access (BLA) picture type and a clean random access (CRA) picture type.

12. The method of claim 10 , wherein the syntax element indicates a network abstraction layer (NAL) unit type value for a NAL unit that includes at least some encoded data for the RAP picture.

13. The method of claim 9 , wherein the decoding the at least a portion of the bitstream includes:

decoding the RAP picture such that the RAP picture is the first picture decoded; and

discarding the one or more non-leading pictures associated with the RAP picture.

14. The method of claim 9 , wherein the decoding the at least a portion of the bitstream includes:

decoding the RAP picture; and

decoding the one or more non-leading pictures associated with the RAP picture.

15. The method of claim 9 , wherein the at least a portion of the bitstream further includes, for each of the one or more potentially non-decodable leading pictures, an indication that the potentially non-decoding leading picture is a random access skipped leading (RASL) picture.

16. The method of claim 15 , wherein the indication that the potentially non-decoding leading picture is a RASL picture is a syntax element that indicates a network abstraction layer (NAL) unit type value for a NAL unit that includes at least some encoded data for the RASL picture.

17. The method of claim 9 , wherein the one or more leading pictures are potentially non-decodable when decoding begins at the RAP picture due to missing reference pictures for the one or more leading pictures when decoding begins at the RAP picture.

18. A computer-readable storage medium storing at least a portion of a bitstream comprising encoded data for plural pictures, the plural pictures comprising a random access point (RAP) picture, one or more associated potentially non-decodable leading pictures, and one or more non-leading pictures associated with the RAP picture, the RAP picture being at a bitstream order position after an initial picture of the bitstream, the at least a portion of the bitstream further comprising an indication signaling that the RAP picture is a picture from which a video decoder can begin decoding, the at least a portion of the bitstream having been generated by operations that include ordering the one or more leading pictures and the one or more non-leading pictures associated with the RAP picture such that all of the one or more leading pictures precede, in decoding order, the one or more non-leading pictures associated with the RAP picture, the encoded data being organized to facilitate decoding, with the video decoder, by operations that include:

receiving the at least a portion of the bitstream; and

decoding the at least a portion of the bitstream.

19. The computer-readable storage medium of claim 18 , wherein the indication signaling that the RAP picture is a picture from which the video decoder can begin decoding is a syntax element signaling that the RAP picture is one of a plurality of RAP picture types, the syntax element indicating a network abstraction layer (NAL) unit type value for a NAL unit that includes at least some encoded data for the RAP picture.

20. The computer-readable storage medium of claim 18 , wherein the bitstream further includes, for each of the one or more potentially non-decodable leading pictures, an indication that the potentially non-decoding leading picture is a random access skipped leading (RASL) picture, and wherein the indication that the potentially non-decoding leading picture is a RASL picture is a syntax element that indicates a network abstraction layer (NAL) unit type value for a NAL unit that includes at least some encoded data for the RASL picture.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 31, 2019
From: SULLIVAN, GARY J.; KANUMURI, SANDEEP
To: MICROSOFT CORPORATION
Reel/Frame 050876/0922 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 31, 2019
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 050876/0965 →