Perceptual three-dimensional (3D) video coding based on depth information
A method for encoding a multi-view frame in a video encoder is provided that includes computing a depth quality sensitivity measure for a multi-view coding block in the multi-view frame, computing a depth-based perceptual quantization scale for a 2D coding block of the multi-view coding block, wherein the depth-based perceptual quantization scale is based on the depth quality sensitive measure and a base quantization scale for the 2D frame including the 2D coding block, and encoding the 2D coding block using the depth-based perceptual quantization scale.
1. A method comprising:
receiving a depth map for a frame, the frame comprises a coding block and the coding block comprises a 2D coding block;
computing a depth-based perceptual quantization scale for the coding block, wherein the depth-based perceptual quantization scale is based on the received depth map and a base quantization scale for the frame; and
encoding the 2D coding block using the depth-based perceptual quantization scale.
2. The method of claim 1 , wherein the 2D coding block is a macroblock.
3. The method of claim 1 , wherein the frame is a stereoscopic frame comprising a left 2D frame and a right 2D frame.
4. The method of claim 1 , further computing a depth quality sensitivity measure for the coding block in the frame based on the received depth map.
5. The method of claim 4 , wherein computing the depth quality sensitive measure comprising computing the depth quality sensitive measure as a function of a depth of the coding block and a depth with a least quality sensitivity.
6. The method of claim 5 , wherein the depth with the least quality sensitivity is the farthest depth of a coding block in the frame.
7. The method of claim 5 , wherein the depth with the least quality sensitivity is user-specified.
8. The method of claim 1 , wherein computing a depth-based perceptual quantization scale comprises applying a depth-based perceptual scaling factor to the base quantization scale, wherein the depth-based perceptual scaling factor is determined based on the depth quality sensitive measure, an average of depth quality sensitivity measures of all coding blocks in the frame, and a number of coding blocks in the frame.
9. The method of claim 8 , wherein the depth-based perceptual scaling factor is computed as
N
_
·
DQSavg
_
N
·
J
(
d
1
,
d
0
)
+
DQSavg
,
wherein N is the number of coding blocks in the frame, DQSavg is the average of depth quality sensitivity measures, and f(dj,do) is the depth quality sensitivity measure.
10. An apparatus comprising:
means for receiving a depth map for a frame, the frame comprises a coding block and the coding block comprises a 2D coding block;
means for computing a depth-based perceptual quantization scale the coding block, wherein the depth-based perceptual quantization scale is based on the received depth map and a base quantization scale for the frame; and
means for encoding the 2D coding block using the depth-based perceptual quantization scale.
11. The apparatus of claim 10 , wherein the 2D coding block is a macroblock.
12. The apparatus of claim 10 , wherein the frame is a stereoscopic frame comprising a left 2D frame and a right 2D frame.
13. The apparatus of claim 10 , further comprising means for computing a depth quality sensitivity measure for a coding block in the frame based in part on the received depth map.
14. The apparatus of claim 10 , wherein the means for computing a depth quality sensitivity measure comprises means for computing the depth quality sensitivity measure as a function of a depth of the coding block and a depth with a least quality sensitivity.
15. The apparatus of claim 14 , wherein the depth with the least quality sensitivity is the farthest depth of a coding block in the frame.
16. The apparatus of claim 14 , wherein the depth with the least quality sensitivity is user-specified.
17. The apparatus of claim 10 , wherein computing a depth-based perceptual quantization scale comprises applying a depth-based perceptual scaling factor to the base quantization scale, wherein the depth-based perceptual scaling factor is determined based on the depth quality sensitive measure, an average of depth quality sensitivity measures of all coding blocks in the frame, and a number of coding blocks in the frame.
18. The apparatus of claim 17 , wherein the depth-based perceptual scaling factor is computed as
N
_
·
DQSavg
_
N
·
J
(
d
1
,
d
0
)
+
DQSavg
,
wherein N is the number of coding blocks in the frame, DQSavg is the average of depth quality sensitivity measures, and f(dj,do) is the depth quality sensitivity measure.
19. A non-transitory computer-readable medium storing software instructions that, when executed by a processor, perform a method, the method comprising:
receiving a depth map for a frame, the frame comprises a multi-view coding block and the coding block comprises a 2D coding block;
computing a depth-based perceptual quantization scale for the coding block, wherein the depth-based perceptual quantization scale is based on the received depth map and a base quantization scale for the frame; and
encoding the 2D coding block using the depth-based perceptual quantization scale.
20. The computer readable medium of claim 19 , wherein computing a depth quality sensitivity measure comprises computing the depth quality sensitivity measure as a function of a depth of the coding block and a depth with a least quality sensitivity.