IP Library Granted Patent US 8,081,680
Granted Patent B2
US 8,081,680 · App. 11/564,088 · Granted Dec 20, 2011

Selective inter-layer prediction in layered video coding

Assignee: Microsoft Corporation
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,081,680
App. No.
11/564,088
Granted
Dec 20, 2011
Kind
B2
Abstract

Systems and methods for using selective inter-layer prediction during layered video coding operations are described. In one aspect, a layered coded video structure is generated by selectively employing only objectively efficient inter-layer predictions between respective frames. Responsive to receiving a request for coded video data, one or more layers of the layered coded video structure are communicated to an application for decoding and presentation of video data to a user.

Claims (54)

1. A method at least partially implemented by a computing device, the method comprising:

selectively employing only objectively efficient inter-layer predictions to generate a layered coded video structure from an input video data sequence; and

responsive to receiving a request from an application for an entire set of layers of the layered coded video structure, communicating the entire set of layers of the layered coded video structure to the application for decoding and presentation to a user, the entire set of layers being free of objectively inefficient inter-layer predictions coded in the entire set of layers, wherein an inter-layer prediction is objectively inefficient if an error associated with coding a frame of a first layer of the layered coded video structure with the inter-layer prediction using one or more frames of a second layer is larger than an error associated with coding the frame of the first layer of the layered coded video structure with an intra-layer prediction using one or more frames of the first layer.

2. The method of claim 1 , wherein the layered coded video structure is a hierarchical-B structure.

3. The method of claim 1 , wherein selectively employing further comprises comparing how an inter-layer frame prediction improves frame coding efficiency as compared to not using the inter-layer frame prediction to code respective frames.

4. The method of claim 1 , wherein selectively employing represents only objectively efficient inter-layer predictions, each inter-layer prediction being based on a first frame positioned on a higher layer of the layered coded video structure using a prediction based on a second frame positioned on a lower layer of the layered coded video structure.

5. The method of claim 1 , wherein selectively employing comprises deciding whether to use the inter-layer predictions based on configuration of the layered coded video structure.

6. The method of claim 1 , wherein selectively employing only efficient inter-layer predictions further comprises:

for each frame of the layered coded video structure, measuring an inter-layer frame prediction efficiency;

if the inter-layer frame prediction efficiency is determined to be objectively efficient, coding the frame with an inter-layer frame prediction; and

if the inter-layer frame prediction efficiency is determined to be objectively inefficient, coding the frame independent of the inter-layer frame prediction.

7. The method of claim 1 , wherein selectively employing only efficient inter-layer predictions further comprises:

responsive to determining that an inter-layer frame prediction for a particular frame is inefficient, generating a prediction to code the frame from information associated with other frame(s) on a same layer as the frame; and

responsive to determining that the inter-layer frame protection for the particular frame is efficient, generating a prediction to code the frame from information associated with one or more other frames on a next lower and adjacent layer to a layer associated with the frame.

8. The method of claim 1 , wherein selectively employing only efficient inter-layer predictions further comprises:

generating a first value based on a particular frame on a specific layer being coded using a first prediction based on information from other frames on the specific layer;

calculating a second value based on the particular frame being coded using a second prediction based on information from frames on a different layer than the specific layer; and

measuring efficiency of the first and second predictions in view of the first and second values to determine whether to code the particular frame using inter-layer frame prediction.

9. The method of claim 8 , wherein measuring the efficiency further comprises:

determining a reduction of mean squared errors between the first and second values;

calculating reduction in distortion between the first and second values for a particular bit rate;

evaluating the reduction of mean squared errors in view of the reduction in distortion to determine whether a result is greater than or equal to a predetermined threshold value;

if the result is less than the predetermined threshold value, coding the particular frame independent of inter-layer frame prediction; and

if the result is greater than or equal to the predetermined threshold value, coding the particular frame using inter-layer frame prediction.

10. The method of claim 1 , wherein the communicating further comprises:

if the request indicates that not all layers of the layered coded video structure are required, and if a layer corresponding to a requested bit rate was coded independent of any inter-layer frame predictions, communicating only the layer to the application, the layer being a simulcast layer.

11. The method of claim 1 , wherein the communicating further comprises:

if the request indicates that all coded video layers are desired, communicating all layers of the layered coded video structure to the application, each layer representing only inter-layer frame predictions previously determined to be objectively efficient.

12. A tangible computer-readable data storage medium storing computer-program instructions executable by a processor, the computer-program instructions when executed by the processor for performing operations comprising:

generating a hybrid layered coded video structure from an input video data sequence;

for each frame of a layer of the layered coded video structure:

evaluating efficiency of coding the frame using intra-layer prediction as compared to coding the frame using intra-layer prediction, the evaluating comprising determining whether the inter-layer prediction is objectively efficient compared to the intra-layer prediction, wherein the inter-layer prediction is objectively efficient compared to the intra-layer prediction if an error associated with coding the frame of the layer of the layered coded video structure using the intra-layer prediction with one or more frames of the layer is greater than an error associated with coding the frame of the layer of the layered coded video structure using the inter-layer prediction with one or more frames of another layer, and

responsive to determining that the inter-layer prediction is objectively efficient compared to the intra-layer prediction, representing the frame of the layer of the layered coded video structure using the inter-layer prediction with the one or more frames of the another layer without coding the frame of the layer using the inter-layer prediction; and

responsive to receiving a request from an application for coded video data, communicating at least a subset of the layers to the application.

13. The computer-readable data storage medium of claim 12 , wherein communicating at least the subset:

if a subset of all layers in the layered coded video structure is communicated to the application, a layer of the subset is a simulcast layer; and

if each layer of the layers is communicated to the application, multiple ones of the layers are based on inter-layer frame predictions that have been determined to be objectively efficient.

14. The computer-readable data storage medium of claim 12 , wherein operations for evaluating the efficiency are based on a configuration of the layered coded video structure.

15. The computer-readable data storage medium of claim 12 , wherein evaluating the efficiency, inter-layer prediction between frames is based on a first frame positioned on a higher layer of the layered coded video structure using a prediction based on a second frame positioned on a lower layer of the layered coded video structure.

16. The computer-readable data storage medium of claim 12 , wherein evaluating the efficiency further comprises:

responsive to determining that inter-layer frame prediction for a particular frame is inefficient, generating a prediction to code the frame from information associated with other frame(s) on a same layer as the frame; and

responsive to determining that inter-layer frame protection for a particular frame is efficient, generating a prediction to code the frame from information associated with one or more other frames on a next lower and adjacent layer to a layer associated with the frame.

17. The computer-readable data storage medium of claim 12 , wherein evaluating the efficiency further comprises:

generating a first value based on a particular frame on a specific layer being coded using a first prediction based on information from other frames on the specific layer;

calculating a second value based on the particular frame being coded using a second prediction based on information from frames on a different layer than the specific layer; and

measuring efficiency of the first and second predictions in view of the first and second values to determine whether to code the particular frame using inter-layer frame prediction.

18. A computing device comprising:

one or more processors; and

memory coupled to the one or more processors, the memory comprising computer-program instructions executable by the processor for performing operations comprising:

calculating a first error associated with coding a frame of a first layer of a video frame sequence when an intra-layer prediction from one or more frames of the first layer is used for the frame of the first layer;

calculating a second error associated with coding the frame of the first layer of the video frame sequence when an inter-layer prediction from one or more frames of a second layer that is different from the first layer is used for the frame of the first layer;

calculating an efficiency of the inter-layer prediction by comparing the second error with the first error;

determining whether the efficiency of the inter-layer prediction is greater than a predetermined threshold; and

in response to determining that the efficiency of the inter-layer prediction is greater than the predetermined threshold, coding the frame of the first layer of the video frame sequence using the inter-layer prediction from the one or more frames of the second layer.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2014
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 034542/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 5, 2006
From: XU, JI-ZHENG; ZHANG, KAI; WU, FENG
To: MICROSOFT CORPORATION
Reel/Frame 018607/0640 →
Continuity (1)
Related Publication 20080123742A1 · May 29, 2008