System and method for robust video coding using progressive fine-granularity scalable (PFGS) coding
View Patent ↗A video encoding scheme employs progressive fine-granularity layered coding to encode video data frames into multiple layers, including a base layer of comparatively low quality video and multiple enhancement layers of increasingly higher quality video. Some of the enhancement layers in a current frame are predicted from at least one lower quality layer in a reference frame, whereby the lower quality layer is not necessarily the base layer.
1. A method for coding video data according to layered coding techniques in which the video data is represented as multi-layered frames, each frame having multiple layers ranging from a base layer of low quality to enhancement layers of increasingly higher quality, the method comprising:
forming a base layer for frames in the video data; and
forming multiple enhancement layers for the frames by (1) predicting even frames from even enhancement layers, but not odd enhancement layers, of preceding odd frames and (2) predicting odd frames from odd enhancement layers, but not even enhancement layers, of preceding even frames.
2. A method as recited in claim 1 , further comprising storing the base layer and the enhancement layers in memory.
3. A method as recited in claim 1 , further comprising:
transmitting the base layer over a network; and
transmitting one or more of the enhancement layers over the network according to bandwidth availability on the network.
4. A method as recited in claim 1 , further comprising recovering the video data from the base layer and any enhancement layer.
5. A method as recited in claim 4 , further comprising reconstructing a missing enhancement layer from an enhancement layer of a reference reconstructed frame.
6. A computer-readable medium having computer-executable instructions, which when executed on a processor, direct a computer to perform the steps of claim 1 .
7. A method for coding video data according to layered coding techniques in which the video data is represented as multi-layered frames, each frame having multiple layers ranging from a base layer of low quality to enhancement layers of increasingly higher quality, the method comprising:
forming a base layer for frames in the video data; and
forming at least first, second, and third enhancement layers by (1) predicting even frames from the base layer and the second enhancement layer, but not the first enhancement layer or the third enhancement layer, of preceding odd frames and (2) predicting odd frames from the base layer and the third enhancement layer, but not the second enhancement layer, of preceding even frames.
8. A method as recited in claim 7 , further comprising storing the base layer and the enhancement layers in memory.
9. A method as recited in claim 7 , further comprising:
transmitting the base layer over a network; and
transmitting one or more of the enhancement layers over the network according to bandwidth availability on the network.
10. A method as recited in claim 7 , further comprising recovering the video data from the base layer and any enhancement layer.
11. A method as recited in claim 10 , further comprising reconstructing a missing enhancement layer from an enhancement layer of a reference reconstructed frame.
12. A computer-readable medium having computer-executable instructions, which when executed on a processor, direct a computer to perform the steps of claim 7 .
13. A method for coding video data, comprising:
encoding frames of the video data into a base layer of low quality; and
encoding the frames of the video data into multiple enhancement layers of increasingly higher quality such that the enhancement layers of even frames are predicted from even layers, but not odd layers, of preceding odd frames and the enhancement layers of odd frames are predicted from odd layers, but not even layers, of preceding even frames.
14. A method as recited in claim 13 , further comprising storing the base layer and the enhancement layers in memory.
15. A method as recited in claim 13 , further comprising:
transmitting the base layer over a network; and
transmitting one or more of the enhancement layers over the network according to bandwidth availability on the network.
16. A method as recited in claim 13 , further comprising decoding the base layer and the one or more enhancement layers into the video data.
17. A method as recited in claim 16 , further comprising reconstructing a missing enhancement layer from an enhancement layer of a reference reconstructed frame.
18. A computer-readable medium having computer-executable instructions, which when executed on a processor, direct a computer to perform the steps of claim 13 .
19. A method comprising:
encoding video data into multi-layered frames where each frame has a base layer of low quality to enhancement layers of increasingly higher quality and selected enhancement layers in a current frame are predicted from at least one lower quality layer in a reference frame that is not the base layer, wherein the encoding comprises predicting even frames from even layers of preceding odd frames and predicting odd frames from odd layers of preceding even frames; and
transmitting the base layer and one or more of the enhancement layers over a network.
20. A method comprising:
encoding video data into multi-layered frames where each frame has a base layer of low quality to enhancement layers of increasingly higher quality, the enhancement layers including at least first, second, and third layers, and the encoding comprises predicting even frames from the base and second layers of preceding odd frames and predicting odd frames from the base and third layers of preceding even frames; and
transmitting the base layer and one or more of the enhancement layers over a network.
21. A method as recited in claim 20 , further comprising:
receiving the base layer and the one or more enhancement layers from the network; and
decoding the base layer and the one or more enhancement layers to recover the video data.
22. A method as recited in claim 20 , further comprising reconstructing a missing enhancement layer from an enhancement layer of a reference reconstructed frame.
23. A computer-readable medium having computer-executable instructions, which when executed on a processor, direct a computer to:
construct a base layer for frames in the video data; and
construct multiple enhancement layers for the frames in the video data by predicting even frames from even layers, and not odd layers, of preceding odd frames and predicting odd frames from odd layers, and not even layers, of preceding even frames.
24. A computer-readable medium as recited in claim 23 , further having instructions that direct a computer to store the base layer and the enhancement layers in memory.
25. A computer-readable medium as recited in claim 23 , further having instructions that direct a computer to:
transmit the base layer over a network; and
transmit one or more of the enhancement layers over the network according to bandwidth availability on the network.
26. A computer-readable medium as recited in claim 23 , further having instructions that direct a computer to recover the video data from the base layer and any of the enhancement layers.
27. A computer-readable medium as recited in claim 26 , further having instructions that direct a computer to reconstruct a missing enhancement layer from an enhancement layer of a reference reconstructed frame.
28. A computer-readable medium having computer-executable instructions, which when executed on a processor direct a computer to:
construct a base layer for frames in the video data; and
construct multiple enhancement layers for the frames in the video data, where the enhancement layers include at least first, second, and third layers of increasingly higher quality video data than the base layer, by predicting even frames from the base and second layers of preceding odd frames and predicting odd frames from the base and third layers of preceding even frames.
29. A computer-readable medium as recited in claim 28 , further having instructions that direct a computer to store the base layer and the enhancement layers in memory.
30. A computer-readable medium as recited in claim 28 , further having instructions that direct a computer to:
transmit the base layer over a network; and
transmit one or more of the enhancement layers over the network according to bandwidth availability on-the network.
31. A computer-readable medium as recited in claim 28 , further having instructions that direct a computer to recover the video data from the base layer and any of the enhancement layers.
32. A computer-readable medium as recited in claim 31 , further having instructions that direct a computer to reconstruct a missing enhancement layer from an enhancement layer of a reference reconstructed frame.
33. A video coding system comprising:
a base layer encoder to encode frames of video data into a base layer;
an enhancement layer encoder to encode the frames into multiple enhancement layers of higher quality than the base layer; and
wherein the enhancement layer encoder predicts even frames from even layers, and not odd layers, of preceding odd frames and predicts odd frames from odd layers, and not even layers, of preceding even frames.
34. A video coding system, comprising:
a base layer encoder to encode frames of video data into a base layer;
an enhancement layer encoder to encode the frames into multiple enhancement layers of higher quality than the base layer, the multiple enhancement layers including at least first, second, and third layers; and
wherein the enhancement layer encoder predicts even frames from the base and second layers of preceding odd frames and predicts odd frames from the base and third layers of preceding even frames.