Lossy data compression with conditional reconstruction refinement
View Patent ↗Methods of encoding and decoding media, for example video, using conditional reconstruction refinement are described. The encoding and decoding relies upon predictions that are based on previously encoded and decoded samples. Prediction information identifies the previously reconstructed samples upon which a prediction is to be based and is available to both the encoder and decoder. On the basis that a previously-reconstructed sample is going to be used as source data for a prediction, the encoder refines that reconstructed sample, uses the refined reconstructed sample for the prediction, and encodes refinement data to enable the decoder to also refine the previously-reconstructed sample during decoding. In some cases, a reconstructed sample may be refined more than once. Various flags or bits may be used to signal when refinement is enabled or disabled and may be signaled at various hierarchical points in the coding structure.
1. A method of decoding data from a bitstream of encoded data using a decoder, the method comprising:
decoding residual data for a first set of samples and reconstructing the first set of samples based on a prediction to produce a set of reconstructed samples;
decoding prediction information for a second set of samples wherein the prediction information specifies a prediction operation involving one of the reconstructed samples from the set of reconstructed samples; and,
determining to refine the reconstructed sample, and based on that determination,
decoding refinement data for that reconstructed sample,
modifying that reconstructed sample using the refinement data to produce a refined reconstructed sample, and
storing the refined reconstructed sample for subsequent prediction operations.
2. The method claimed in claim 1 , further comprising:
generating a predicted second set of samples based on the refined reconstructed sample and the prediction information; and
reconstructing the second set of samples based on the predicted second set of samples.
3. The method claimed in claim 1 , wherein the prediction information identifies source data for predicting the second set of samples, and wherein the source data includes said one of the reconstructed samples.
4. The method claimed in claim 3 , wherein the data comprises video, and wherein the prediction information identifies one of an intra-coding mode and a motion vector.
5. The method claimed in claim 1 , wherein determining to refine includes determining that conditional refinement is enabled based on at least one decoded flag from the bitstream.
6. The method claimed in claim 5 , wherein the at least one decoded flag indicates that conditional refinement is enabled with respect to said one of the reconstructed samples.
7. The method claimed in claim 5 , wherein the at least one decoded flag indicates that refinement of reconstructed samples is permitted in association with reconstruction of the second set of samples.
8. The method claimed in claim 1 , wherein the refinement data comprises refined residual data, and wherein modifying said one of the reconstructed samples includes adding the refined residual data to that reconstructed sample.
9. The method claimed in claim 1 , further comprising:
decoding further prediction information for a third set of samples; and
determining that the further prediction information specifies a prediction operation involving said one of the reconstructed samples and, based on that determination,
decoding further-refinement data for that reconstructed sample, and
modifying said refined reconstructed sample using the further-refinement data to produce a further-refined reconstructed sample.
10. A method of encoding source data using an encoder, the method comprising:
encoding residual data for a first set of samples from the source data;
decoding the residual data and reconstructing the first set of samples based on its prediction to produce a set of reconstructed samples;
determining prediction information for a second set of samples, wherein the prediction information specifies a prediction operation involving one of the reconstructed samples from the set of reconstructed samples; and,
determining to refine the reconstructed sample, and based on that determination,
generating refinement data for that reconstructed sample from said first set of samples,
encoding and decoding that refinement data to obtain reconstructed refinement data,
modifying that reconstructed sample using the reconstructed refinement data to produce a refined reconstructed sample, and
storing the refined reconstructed sample for subsequent prediction operations.
11. The method claimed in claim 10 , further comprising:
generating a predicted second set of samples based on the refined reconstructed sample and the prediction information;
determining second residual data from the predicted second set of samples and the second set of samples; and
encoding the second residual data, the refinement data, and the prediction information.
12. The method claimed in claim 10 , wherein the prediction information identifies source data for predicting the second set of samples, and wherein the source data includes said one of the reconstructed samples.
13. The method claimed in claim 12 , wherein the data comprises video, and wherein the prediction information identifies one of an intra-coding mode and a motion vector.
14. The method claimed in claim 10 , wherein determining to refine includes determining that conditional refinement is enabled and encoding at least one flag indicating that conditional refinement is enabled.
15. The method claimed in claim 14 , wherein the at least one flag indicates that conditional refinement is enabled with respect to said one of the reconstructed samples.
16. The method claimed in claim 14 , wherein the at least one flag indicates that refinement of reconstructed samples is permitted in association with reconstruction of the second set of samples.
17. The method claimed in claim 10 , wherein the reconstructed refinement data comprises refined residual data, and wherein modifying said one of the reconstructed samples includes adding the refined residual data to that reconstructed sample.
18. The method claimed in claim 10 , further including:
selecting further prediction information for a third set of samples; and
determining that the further prediction information specifies a prediction operation involving said one of the reconstructed samples and, based on that determination,
generating further-refinement data for that reconstructed sample from said first set of samples,
encoding and decoding that further-refinement data to obtain reconstructed further-refinement data, and
modifying that refined reconstructed sample using the reconstructed further-refinement data to produce a further-refined reconstructed sample.
19. A decoder for decoding a bitstream of encoded data, the decoder comprising:
a processor;
a memory; and
a decoding application stored in memory and containing instructions for causing the processor to perform the method claimed in claim 1 .
20. The decoder claimed in claim 19 , wherein the instructions are further for causing the processor to:
generate a predicted second set of samples based on the refined reconstructed sample and the prediction information; and
reconstruct the second set of samples based on the predicted second set of samples.
21. The decoder claimed in claim 19 , wherein the prediction information identifies source data for predicting the second set of samples, and wherein the source data includes said one of the reconstructed samples.
22. The decoder claimed in claim 21 , wherein the data comprises video, and wherein the prediction information identifies one of an intra-coding mode and a motion vector.
23. The decoder claimed in claim 19 , wherein the processor determines to refine by determining that conditional refinement is enabled based on at least one decoded flag from the bitstream.
24. The decoder claimed in claim 23 , wherein the at least one decoded flag indicates that conditional refinement is enabled with respect to said one of the reconstructed samples.
25. The decoder claimed in claim 23 , wherein the at least one decoded flag indicates that refinement of reconstructed samples is permitted in association with reconstruction of the second set of samples.
26. The decoder claimed in claim 19 , wherein the refinement data comprises refined residual data, and wherein the processor modifies said one of the reconstructed samples by adding the refined residual data to that reconstructed sample.
27. The decoder claimed in claim 19 , wherein the instructions are further for causing the processor to:
decode further prediction information for a third set of samples; and
determine that the further prediction information specifies a prediction operation involving said one of the reconstructed samples and, based on that determination,
decode further-refinement data for that reconstructed sample, and
modify said refined reconstructed sample using the further-refinement data to produce a further-refined reconstructed sample.
28. An encoder for encoding data, the encoder comprising:
a processor;
a memory; and
an encoding application stored in memory and containing instructions for causing the processor to perform the method claimed in claim 10 .
29. The encoder claimed in claim 28 , wherein the instructions are further for causing the processor to:
generate a predicted second set of samples based on the refined reconstructed sample and the prediction information;
determine second residual data from the predicted second set of samples and the second set of samples; and
encode the second residual data, the refinement data, and the prediction information.
30. The encoder claimed in claim 28 , wherein the prediction information identifies source data for predicting the second set of samples, and wherein the source data includes said one of the reconstructed samples.
31. The encoder claimed in claim 30 , wherein the data comprises video, and wherein the prediction information identifies one of an intra-coding mode and a motion vector.
32. The encoder claimed in claim 28 , wherein the processor determines to refine by determining that conditional refinement is enabled and encoding at least one flag indicating that conditional refinement is enabled.
33. The encoder claimed in claim 32 , wherein the at least one flag indicates that conditional refinement is enabled with respect to said one of the reconstructed samples.
34. The encoder claimed in claim 32 , wherein the at least one flag indicates that refinement of reconstructed samples is permitted in association with reconstruction of the second set of samples.
35. The encoder claimed in claim 28 , wherein the reconstructed refinement data comprises refined residual data, and wherein the processor modifies said one of the reconstructed samples by adding the refined residual data to that reconstructed sample.
36. The encoder claimed in claim 28 , wherein the instructions are further for causing the processor to:
select further prediction information for a third set of samples; and
determine that the further prediction information specifies a prediction operation involving said one of the reconstructed samples and, based on that determination,
generate further-refinement data for that reconstructed sample from said first set of samples,
encode and decode that further-refinement data to obtain reconstructed further-refinement data, and
modify that refined reconstructed sample using the reconstructed further-refinement data to produce a further-refined reconstructed sample.
37. A non-transitory processor-readable medium storing processor-executable instructions which, when executed, configure one or more processors to perform the method claimed in claim 1 .
38. A non-transitory processor-readable medium storing processor-executable instructions which, when executed, configure one or more processors to perform the method claimed in claim 10 .