Scalable predictive coding method and apparatus
A scalable predictive coder in which the current frame of data is predicted at the enhancement-layer by processing and combining the reconstructed signal at: (i) the current base-layer (or lower layers) frame; and (ii) the previous enhancement-layer frame. The combining rule takes into account the compressed prediction error of the base-layer, and the parameters used for its compression.
1. An encoder for performing scalable predictive coding, the encoder comprising:
a base layer encoder configured to reconstruct base layer data representing a current base layer frame to create reconstructed base layer data, the reconstruction performed using a base layer compression parameter, the base layer compression parameter comprising at least one of a quantized base layer residual, a quantization interval, or a partition region; and
an enhancement layer encoder configured to:
store, in a delay component, previous frame reconstructed enhancement layer data generated from processing a previous frame of enhancement layer data;
receive, distinct from the base layer data representing the base layer frame, the base layer compression parameter;
obtain, from the delay component, the previous frame reconstructed enhancement layer data;
reconstruct enhancement layer data representing the current frame based at least in part on the previous frame reconstructed enhancement layer data; and
predict enhancement layer data representing the current enhancement layer frame by processing and combining the reconstructed base layer data and the previous frame reconstructed enhancement layer data, wherein the predicting enhancement layer data representing the current enhancement layer frame uses the received base layer compression parameter.
2. The encoder of claim 1 wherein the reconstructed base layer data comprises residual data representing a difference between a reconstruction of the current base layer frame and a prediction of the current base layer frame.
3. The encoder of claim 2 wherein the residual data comprises a quantized base layer residual.
4. The encoder of claim 1 wherein the reconstructed base layer data comprises frame data representing a reconstruction of the current base layer frame.
5. A method of performing scalable predictive coding, the method comprising:
in a base layer, reconstructing base layer data representing a current base layer frame to create reconstructed base layer data, the reconstruction performed using a base layer compression parameter, the base layer compression parameter comprising at least one of a quantized base layer residual, a quantization interval, or a partition region;
in an enhancement layer—
storing, in a delay component, previous frame reconstructed enhancement layer data generated from processing a previous frame of enhancement layer data;
obtaining, from the delay component, the previous frame reconstructed enhancement layer data;
reconstructing enhancement layer data representing the current frame based at least in part on the previous frame reconstructed enhancement layer data;
receiving, distinct from the base layer data representing the base layer frame, the base layer compression parameter; and
predicting enhancement layer data representing a current enhancement layer frame by processing and combining the reconstructed base layer data and the previous frame reconstructed enhancement layer data, wherein the predicting enhancement layer data representing a current enhancement layer frame uses the received base layer compression parameter.
6. The method of claim 5 wherein at least a portion of the method is performed during decoding of the current enhancement layer frame.
7. The method of claim 5 wherein at least a portion of the method is performed during encoding of the current enhancement layer frame.
8. The method of claim 5 wherein the reconstructed base layer data comprises residual data representing a difference between a reconstruction of the current base layer frame and a prediction of the current base layer frame.
9. The method of claim 8 wherein the residual data comprises a quantized base layer residual.
10. The method of claim 5 wherein the reconstructed base layer data comprises frame data representing a reconstruction of the current base layer frame.
11. An encoder for performing scalable predictive coding, the encoder comprising:
a base layer encoder configured to:
reconstruct base layer data representing a current base layer frame to create reconstructed base layer data; and
provide additional base layer information regarding the current base layer frame, the additional base layer information comprising a base layer compression parameter, the base layer compression parameter comprising at least one of a quantized base layer residual, a quantization interval, or a partition region; and
an enhancement layer encoder configured to:
store, in a delay component, previous frame reconstructed enhancement layer data generated from processing a previous frame of enhancement layer data;
receive, distinct from the base layer data representing the base layer frame, the base layer compression parameter;
obtain, from the delay component, the previous frame reconstructed enhancement layer data;
predict enhancement layer data representing a current enhancement layer frame by processing and combining the reconstructed base layer data and the previous frame reconstructed enhancement layer data wherein the predicting enhancement layer data representing a current enhancement layer frame uses the received base layer compression parameter; and
reconstruct enhancement layer data for the current frame based at least in part on the predicted enhancement layer data to obtain current frame reconstructed enhancement layer data.
12. The encoder of claim 11 wherein the reconstructed base layer data comprises residual data representing a difference between a reconstruction of the current base layer frame and a prediction of the current base layer frame.
13. The encoder of claim 12 wherein the residual data comprises a quantized base layer residual.
14. The encoder of claim 11 wherein the base layer compression parameter comprises residual data representing a difference between a reconstruction of the current base layer frame and a prediction of the current base layer frame.
15. The encoder of claim 14 wherein the residual data comprises a quantized base layer residual.
16. A method of performing scalable predictive coding, the method comprising:
in a base layer, reconstructing base layer data representing a current base layer frame to create reconstructed base layer data and providing additional base layer information regarding the current base layer frame, the additional base layer information comprising a base layer compression parameter, the base layer compression parameter comprising at least one of a quantized base layer residual, a quantization interval, or a partition region;
in an enhancement layer—
storing, in a delay component, previous frame reconstructed enhancement layer data generated from processing a previous frame of enhancement layer data;
obtaining the previous frame reconstructed enhancement layer data from the delay component;
reconstructing enhancement layer data representing the current frame based at least in part on the previous frame reconstructed enhancement layer data;
receiving, distinct from the base layer data representing the base layer frame, the additional base layer information; and
predicting enhancement layer data representing a current enhancement layer frame by processing and combining the reconstructed base layer data and the reconstructed previous frame enhancement layer data, wherein predicting enhancement layer data representing a current enhancement layer frame uses the received additional base layer information.
17. The method of claim 16 wherein at least a portion of the method is performed during decoding of the current enhancement layer frame.
18. The method of claim 16 wherein at least a portion of the method is performed during encoding of the current enhancement layer frame.
19. The method of claim 16 wherein the reconstructed base layer data comprises residual data representing a difference between a reconstruction of the current base layer frame and a prediction of the current base layer frame.
20. The method of claim 19 wherein the residual data comprises a quantized base layer residual.
21. The method of claim 16 wherein the base layer compression parameter comprises residual data representing a difference between a reconstruction of the current base layer frame and a prediction of the current base layer frame.
22. The method of claim 21 wherein the residual data comprises a quantized base layer residual.
23. A tangible computer-readable storage medium having instructions stored thereon, the instructions, when executed by a computing device, cause the computing device to perform operations comprising:
reconstructing, in a base layer, base layer data representing a current base layer frame to create reconstructed base layer data comprising a base layer compression parameter, the base layer compression parameter comprising at least one of a quantized base layer residual, a quantization interval, or a partition region;
store in a delay component, using, in an enhancement layer, reconstructed previous frame enhancement layer data generated from processing a previous frame of enhancement layer data;
obtain from the delay component, using, in the enhancement layer, the reconstructed previous frame enhancement layer data;
reconstructing, in the enhancement layer, enhancement layer data representing the current frame based at least in part on the previous frame reconstructed enhancement layer data;
receiving, in the enhancement layer, the base layer compression parameter wherein the received compression parameter is distinct from the base layer data representing the base layer frame; and
predicting, in the enhancement layer, enhancement layer data representing a current enhancement layer frame by processing and combining the reconstructed base layer data and the reconstructed previous frame enhancement layer data, wherein the predicting, in the enhancement layer, enhancement layer data representing a current enhancement layer frame uses the received base layer compression parameter.
24. The tangible computer-readable storage medium of claim 23 wherein the instructions are executed during decoding of the current enhancement layer frame.
25. The tangible computer-readable storage medium of claim 23 wherein the instructions are executed during encoding of the current enhancement layer frame.
26. The tangible computer-readable storage medium of claim 23 wherein the reconstructed base layer data comprises residual data representing a difference between a reconstruction of the current base layer frame and a prediction of the current base layer frame.
27. The tangible computer-readable storage medium of claim 26 wherein the residual data comprises a quantized base layer residual.
28. The encoder of claim 1 wherein the base layer compression parameter comprises a base layer residual.
29. The encoder of claim 1 wherein:
the enhancement layer encoder is a first enhancement layer encoder,
the first enhancement layer encoder reconstructs enhancement layer data representing the current frame based on a first enhancement layer compression parameter, and wherein the encoder further comprises a second enhancement layer encoder, the second enhancement layer encoder configured to:
receive the first enhancement layer compression parameter, the received first enhancement layer compression parameter being distinct from the reconstructed first enhancement layer data; and
predict, based on the received first enhancement layer compression parameter, second enhancement layer data representing the current second enhancement layer frame by processing and combining the reconstructed first enhancement layer data and previous frame reconstructed second enhancement layer data.
30. The encoder of claim 1 wherein the enhancement layer encoder further comprises a motion compensator, and wherein the enhancement layer encoder is further configured to reconstruct enhancement layer data representing a previous enhancement layer frame to create previous frame reconstructed enhancement layer data in part by receiving, from the motion compensator, motion compensated data for the previous enhancement layer frame.
31. The method of claim 5 wherein the base layer compression parameter comprises a base layer residual.
32. The method of claim 5 wherein the enhancement layer is a first enhancement layer, and, in the first enhancement layer, reconstructing the enhancement layer data representing the current frame is based on a first enhancement layer compression parameter, and wherein the method further comprises:
in a second enhancement layer:
receiving the first enhancement layer compression parameter, the received first enhancement layer compression parameter being distinct from the reconstructed first enhancement layer data; and
predicting, based on the received first enhancement layer compression parameter, second enhancement layer data representing the current second enhancement layer frame by processing and combining the reconstructed first enhancement layer data and previous frame reconstructed second enhancement layer data.
33. The method of claim 5 wherein the reconstructing enhancement layer data representing a previous enhancement layer frame to create previous frame reconstructed enhancement layer data is performed in part by receiving motion compensated data for the previous enhancement layer frame.
34. The encoder of claim 11 wherein the base layer compression parameter comprises a base layer residual.
35. The encoder of claim 11 wherein:
the enhancement layer encoder is a first enhancement layer encoder,
the first enhancement layer encoder reconstructs enhancement layer data representing the current frame based on a first enhancement layer compression parameter, and wherein the encoder further comprises a second enhancement layer encoder, the second enhancement layer encoder configured to:
receive the first enhancement layer compression parameter, the received first enhancement layer compression parameter being distinct from the reconstructed first enhancement layer data; and
predict, based on the received first enhancement layer compression parameter, second enhancement layer data representing the current second enhancement layer frame by processing and combining the reconstructed first enhancement layer data and previous frame reconstructed second enhancement layer data.
36. The encoder of claim 11 wherein the enhancement layer encoder further comprises a motion compensator, and wherein the enhancement layer encoder is further configured to reconstruct enhancement layer data representing a previous enhancement layer frame to create previous frame reconstructed enhancement layer data in part by receiving, from the motion compensator, motion compensated data for the previous enhancement layer frame.
37. The method of claim 16 wherein the base layer compression parameter comprises a base layer residual.
38. The method of claim 16 wherein the enhancement layer is a first enhancement layer, and, in the first enhancement layer, reconstructing the enhancement layer data representing the current frame is based on a first enhancement layer compression parameter, and wherein the method further comprises:
in a second enhancement layer:
receiving the first enhancement layer compression parameter, the received first enhancement layer compression parameter being distinct from the reconstructed first enhancement layer data; and
predicting, based on the received first enhancement layer compression parameter, second enhancement layer data representing the current second enhancement layer frame by processing and combining the reconstructed first enhancement layer data and previous frame reconstructed second enhancement layer data.
39. The method of claim 16 wherein the reconstructing enhancement layer data representing a previous enhancement layer frame to create previous frame reconstructed enhancement layer data is performed in part by receiving motion compensated data for the previous enhancement layer frame.
40. The tangible computer-readable storage medium of claim 23 wherein the base layer compression parameter comprises a base layer residual.
41. The tangible computer-readable storage medium of claim 23 wherein the enhancement layer is a first enhancement layer, and, in the first enhancement layer, reconstructing the enhancement layer data representing the current frame is based on a first enhancement layer compression parameter, and wherein the method further comprises:
in a second enhancement layer:
receiving the first enhancement layer compression parameter, the received first enhancement layer compression parameter being distinct from the reconstructed first enhancement layer data; and
predicting, based on the received first enhancement layer compression parameter, second enhancement layer data representing the current second enhancement layer frame by processing and combining the reconstructed first enhancement layer data and previous frame reconstructed second enhancement layer data.
42. The tangible computer-readable storage medium of claim 23 wherein the reconstructing, in an enhancement layer, of enhancement layer data representing a previous enhancement layer frame to create reconstructed previous frame enhancement layer data is performed in part by receiving motion compensated data for the previous enhancement layer frame.
43. The method of claim 5 wherein the method is performed during decoding of the current frame.
44. The method of claim 5 wherein the method is performed during encoding of the current frame.
45. The method of claim 16 wherein the method is performed during decoding of the current frame.
46. The method of claim 16 wherein the method is performed during encoding of the current frame.
47. A method of performing scalable predictive coding, the method comprising:
in a base layer, reconstructing base layer data representing a current base layer frame to create reconstructed base layer data, the reconstruction performed using a base layer compression parameter, the base layer compression parameter comprising at least one of a quantized base layer residual, a quantization interval, or a partition region;
in an enhancement layer—
storing, in a delay component, previous frame reconstructed enhancement layer data generated from processing a previous frame of enhancement layer data;
obtaining, from the delay component, the previous frame reconstructed enhancement layer data;
reconstructing enhancement layer data representing the current frame based at least in part on the previous frame reconstructed enhancement layer data;
receiving the base layer compression parameter processed separately from the base layer data representing the base layer frame; and
predicting enhancement layer data representing a current enhancement layer frame by processing and combining the reconstructed base layer data and the previous frame reconstructed enhancement layer data, wherein the predicting enhancement layer data representing a current enhancement layer frame uses the received base layer compression parameter.
48. A method of performing scalable predictive coding, the method comprising:
in a base layer, reconstructing base layer data representing a current base layer frame to create reconstructed base layer data and providing additional base layer information regarding the current base layer frame, the additional base layer information comprising a base layer compression parameter, the base layer compression parameter comprising at least one of a quantized base layer residual, a quantization interval, or a partition region;
in an enhancement layer—
storing, in a delay component, previous frame reconstructed enhancement layer data generated from processing a previous frame of enhancement layer data;
obtaining, from the delay component, the previous frame reconstructed enhancement layer data;
reconstructing enhancement layer data representing the current frame based at least in part on the previous frame reconstructed enhancement layer data;
receiving the additional base layer information processed separately from the base layer data representing the base layer frame; and
predicting enhancement layer data representing a current enhancement layer frame by processing and combining the reconstructed base layer data and the reconstructed previous frame enhancement layer data, wherein predicting enhancement layer data representing a current enhancement layer frame uses the received additional base layer information.
49. A decoder for performing scalable predictive coding, the decoder comprising:
a base layer decoder configured to:
reconstruct base layer data representing a current base layer frame to create reconstructed base layer data; and
provide additional base layer information regarding the current base layer frame, the additional base layer information comprising a base layer compression parameter, the base layer compression parameter comprising at least one of a quantized base layer residual, a quantization interval, or a partition region; and
an enhancement layer decoder configured to:
store, in a delay component, previous frame reconstructed enhancement layer data generated from processing a previous frame of enhancement layer data;
receive, distinct from the base layer data representing the base layer frame, the base layer compression parameter;
obtain, from the delay component, the previous frame reconstructed enhancement layer data;
predict enhancement layer data representing a current enhancement layer frame by processing and combining the reconstructed base layer data and the previous frame reconstructed enhancement layer data wherein the predicting enhancement layer data representing a current enhancement layer frame uses the received base layer compression parameter; and
reconstruct enhancement layer data for the current frame based at least in part on the predicted enhancement layer data to obtain current frame reconstructed enhancement layer data.
50. A method of performing scalable predictive decoding, the method comprising:
in a base layer decoder, reconstructing base layer data representing a current base layer frame to create reconstructed base layer data and providing additional base layer information regarding the current base layer frame, the additional base layer information comprising a base layer compression parameter, the base layer compression parameter comprising at least one of a quantized base layer residual, a quantization interval, or a partition region;
in an enhancement layer decoder—
storing, in a delay component, previous frame reconstructed enhancement layer data generated from processing a previous frame of enhancement layer data;
obtaining, from the delay component, the previous frame reconstructed enhancement layer data;
reconstructing enhancement layer data representing the current frame based at least in part on the previous frame reconstructed enhancement layer data;
receiving, distinct from the base layer data representing the base layer frame, the additional base layer information; and
predicting enhancement layer data representing a current enhancement layer frame by processing and combining the reconstructed base layer data and the reconstructed previous frame enhancement layer data, wherein predicting enhancement layer data representing a current enhancement layer frame uses the received additional base layer information.