Inter-layer reference picture enhancement for multiple layer video coding
A video device may generate an enhanced inter-layer reference (E-ILR) picture io assist in predicting an enhancement layer picture of a. scalable bitstream. An E-ILR picture may include one or more E-ILR blocks. An E-ILR block may be generated using a differential method, a residual method, a bi-prediction method, and/or a uni-prediction method. The video device may determine a first time instance. The video device may subtract a block of a first base layer picture characterized by the first time instance from a block of an enhancement layer picture characterized by the first time instance to generate a differential block characterized by the first time instance. The video device may perform motion compensation on the differential block and add the motion compensated differential picture to a block of the second base layer picture characterized by the second time instance to generate an E-ILR. block.
1. A device comprising:
a processor configured to:
determine a first time instance based on motion information of a block of a base layer picture characterized by a second time instance;
generate, using the motion information, a motion compensated differential block characterized by the first time instance that indicates a difference between a block of a base layer reference picture characterized by the first time instance and a block of an enhancement layer picture characterized by the first time instance;
determine a weight using a reference index associated with the base layer reference picture and a prediction mode;
apply the weight to the motion compensated differential block;
add the weighted motion compensated differential block to the block of the base layer picture characterized by the second time instance to generate an enhanced inter-layer reference (E-ILR) block characterized by the second time instance;
generate an E-ILR picture characterized by the second time instance that comprises the E-ILR block; and
add the E-ILR picture to a reference picture list for the enhancement layer picture characterized by the second time instance.
2. The device of claim 1 , wherein the block of the base layer reference picture is an upsampled block of the base layer reference picture, and wherein the block of the base layer picture is an upsampled block of the base layer picture.
3. The device of claim 1 , wherein the processor is configured to determine the prediction mode based on whether the motion compensated differential block is generated using uni-prediction or bi-prediction.
4. The device of claim 3 , wherein the processor is further configured to:
select a lookup table out of a plurality of lookup tables based on the reference index from the motion information and the prediction mode; and
determine the weight using the selected lookup table.
5. The device of claim 1 , wherein the processor is configured to add an offset to the motion compensated differential block.
6. The device of claim 5 , wherein the processor is further configured to subtract the offset from the motion compensated differential block before adding the motion compensated differential block to the block of the base layer picture to generate the E-ILR block.
7. The device of claim 1 , wherein the processor is further configured to determine an offset such that when the offset is added to the motion compensated differential block, a dynamic range of the motion compensated differential block is shifted to a dynamic range of the enhancement layer picture characterized by the second time instance.
8. The device of claim 1 , wherein the processor is configured to predict the enhancement layer picture characterized by the second time instance using the E-ILR picture.
9. A video device comprising:
a processor configured to:
determine a first time instance based on motion information of a block of a base layer picture characterized by a second time instance;
generate a residual block that indicates a difference between a motion compensated block of a base layer reference picture characterized by the first time instance and a block of the base layer picture;
determine a weight using a reference index associated with the base layer reference picture and a prediction mode;
apply the weight to the residual block;
add the weighted residual block to a motion compensated block of an enhancement layer picture characterized by the first time instance to generate an enhanced inter-layer reference (E-ILR) block characterized by the second time instance; and
generate an E-ILR picture characterized by the second time instance that comprises the E-ILR block; and
add the E-ILR picture to a reference picture list for the enhancement layer picture characterized by the second time instance.
10. The device of claim 9 , wherein the block of the base layer reference picture is an upsampled block of the base layer reference picture, and wherein the block of the base layer picture is an upsampled block of the base layer picture.
11. The device of claim 9 , wherein the prediction mode is a uni-prediction mode or a bi-prediction mode.
12. The device of claim 9 , wherein the processor is configured to add an offset to the residual block after applying the weight to the residual block.
13. The device of claim 12 , wherein the processor is configured to subtract the offset from the residual block before adding the residual block to the motion compensated block of the enhancement layer picture to generate the E-ILR block.
14. The device of claim 9 , wherein the processor is configured to predict the enhancement layer picture characterized by the second time instance using the E-ILR picture.
15. A device comprising:
a processor configured to:
determine a first time instance based on motion information of a block of a base layer picture characterized by a second time instance;
generate, using the motion information, a motion compensated differential block characterized by the first time instance that indicates a difference between a block of a base layer reference picture characterized by the first time instance and a block of an enhancement layer picture characterized by the first time instance;
determine a weight using a reference index associated with the base layer reference picture and a prediction mode;
apply the weight to the motion compensated differential block; and
add the weighted motion compensated differential block to a block of the base layer picture characterized by the second time instance to generate an enhanced inter-layer reference (E-ILR) block characterized by the second time instance.
16. The device of claim 15 , wherein the block of the base layer reference picture is an upsampled block of the base layer reference picture, and wherein the block of the base layer picture is an upsampled block of the base layer picture.
17. The device of claim 15 , wherein the processor is configured to:
select a lookup table out of a plurality of lookup tables based on the reference index from the motion information and the prediction mode; and
determine the weight using the selected lookup table.
18. The device of claim 15 , wherein the processor is configured to add an offset to the motion compensated differential block.
19. The device of claim 18 , wherein the processor is further configured to subtract the offset from the motion compensated differential block before adding the motion compensated differential block to the block of the base layer picture to generate the E-ILR block.
20. The device of claim 15 , wherein the processor is configured to:
generate an E-ILR picture characterized by the second time instance that comprises the E-ILR block;
add the E-ILR picture to a reference picture list for the enhancement layer picture characterized by the second time instance; and
predict the enhancement layer picture characterized by the second time instance using the E-ILR picture.
21. A method for decoding a video, the method comprising determining, by a device, a first time instance based on motion information of a block of a base layer picture characterized by a second time instance;
generating, using the motion information, a motion compensated differential block characterized by the first time instance that indicates a difference between a block of a base layer reference picture characterized by the first time instance and a block of an enhancement layer picture characterized by the first time instance;
determining a weight using a reference index associated with the base layer reference picture and a prediction mode;
applying the weight to the motion compensated differential block;
adding the weighted motion compensated differential block to the block of the base layer picture characterized by the second time instance to generate an enhanced inter-layer reference (E-ILR) block characterized by the second time instance;
generating an E-ILR picture characterized by the second time instance that comprises the E-ILR block; and
adding the E-ILR picture to a reference picture list for the enhancement layer picture characterized by the second time instance.
22. A method for decoding a video, the method comprising: a processor configured to:
determining a first time instance based on motion information of a block of a base layer picture characterized by a second time instance;
generating a residual block that indicates a difference between a motion compensated block of a base layer reference picture characterized by the first time instance and a block of the base layer picture;
determining a weight using a reference index associated with the base layer reference picture and a prediction mode;
applying the weight to the residual block;
adding adding the weighted residual block to a motion compensated block of an enhancement layer picture characterized by the first time instance to generate an enhanced inter-layer reference (E-ILR) block characterized by the second time instance; and
generating an E-ILR picture characterized by the second time instance that comprises the E-ILR block; and
adding the E-ILR picture to a reference picture list for the enhancement layer picture characterized by the second time instance.