Perceptual three-dimensional (3D) video coding based on depth information
A method for encoding a multi-view frame in a video encoder is provided that includes computing a depth quality sensitivity measure for a multi-view coding block in the multi-view frame, computing a depth-based perceptual quantization scale for a 2D coding block of the multi-view coding block, wherein the depth-based perceptual quantization scale is based on the depth quality sensitive measure and a base quantization scale for the 2D frame including the 2D coding block, and encoding the 2D coding block using the depth-based perceptual quantization scale.
1. A method comprising:
receiving a depth map for a multi-view frame, the multi-view frame comprises a multi-view coding block and the multi-view coding block comprises a 2D coding block;
computing a depth quality sensitivity measure for the multi-view coding block in the multi-view frame based in part on the received depth map;
computing a depth-based perceptual quantization scale for the multi-view coding block, wherein the depth-based perceptual quantization scale is based on the depth quality sensitive measure and a base quantization scale for the multi-view frame; and
encoding the 2D coding block using the depth-based perceptual quantization scale.
2. The method of claim 1 , wherein the 2D coding block is a macroblock.
3. The method of claim 1 , wherein the multi-view frame is a stereoscopic frame comprising a left 2D frame and a right 2D frame.
4. The method of claim 1 , wherein computing a depth quality sensitivity measure comprises computing the depth quality sensitivity measure as a function of a depth of the multi-view coding block and a depth with a least quality sensitivity.
5. The method of claim 4 , wherein the function is f(d,d 0 )=c 0 ·|d−d 0 |+c 1 , wherein c 0 and c 1 are tuning parameters, d is the depth of the multi-view coding block, and d 0 is the depth with the least quality sensitivity.
6. The method of claim 5 , wherein the depth with the least quality sensitivity is the farthest depth of a multi-view coding block in the multi-view frame.
7. The method of claim 5 , wherein the depth with the least quality sensitivity is user-specified.
8. The method of claim 1 , wherein computing a depth-based perceptual quantization scale comprises applying a depth-based perceptual scaling factor to the base quantization scale, wherein the depth-based perceptual scaling factor is determined based on the depth quality sensitive measure, an average of depth quality sensitivity measures of all multi-view coding blocks in the multi-view frame, and a number of multi-view coding blocks in the multi-view frame.
9. The method of claim 8 , wherein the depth-based perceptual scaling factor is computed as
N
·
DQS
avg
N
·
f
(
d
j
,
d
0
)
+
DQS
avg
,
wherein N is the number of multi-view coding blocks in the multi-view frame, DQS avg is the average of depth quality sensitivity measures, and f(d j ,d 0 ) is the depth quality sensitivity measure.
10. An apparatus comprising:
Means for receiving a depth map for a multi-view frame, the multi-view frame comprises a multi-view coding block and the multi-view coding block comprises a 2D coding block;
means for computing a depth quality sensitivity measure for a multi-view coding block in the multi-view frame based in part on the received depth map;
means for computing a depth-based perceptual quantization scale the multi-view coding block, wherein the depth-based perceptual quantization scale is based on the depth quality sensitive measure and a base quantization scale for the multi-view frame; and
means for encoding the 2D coding block using the depth-based perceptual quantization scale.
11. The apparatus of claim 10 , wherein the 2D coding block is a macroblock.
12. The apparatus of claim 10 , wherein the multi-view frame is a stereoscopic frame comprising a left 2D frame and a right 2D frame.
13. The apparatus of claim 10 , wherein the means for computing a depth quality sensitivity measure comprises means for computing the depth quality sensitivity measure as a function of a depth of the multi-view coding block and a depth with a least quality sensitivity.
14. The apparatus of claim 10 , wherein the function is f(d,d 0 )=c 0 ·|d−d 0 |+c 1 , wherein c 0 and c 1 are tuning parameters, d is the depth of the multi-view coding block, and d 0 is the depth with the least quality sensitivity.
15. The apparatus of claim 14 , wherein the depth with the least quality sensitivity is the farthest depth of a multi-view coding block in the multi-view frame.
16. The apparatus of claim 14 , wherein the depth with the least quality sensitivity is user-specified.
17. The apparatus of claim 10 , wherein computing a depth-based perceptual quantization scale comprises applying a depth-based perceptual scaling factor to the base quantization scale, wherein the depth-based perceptual scaling factor is determined based on the depth quality sensitive measure, an average of depth quality sensitivity measures of all multi-view coding blocks in the multi-view frame, and a number of multi-view coding blocks in the multi-view frame.
18. The apparatus of claim 17 , wherein the depth-based perceptual scaling factor is computed as
N
·
DQS
avg
N
·
f
(
d
j
,
d
0
)
+
DQS
avg
,
wherein N is the number of multi-view coding blocks in the multi-view frame, DQS avg is the average of depth quality sensitivity measures, and f(d j ,d 0 ) is the depth quality sensitivity measure.
19. A non-transitory computer-readable medium storing software instructions that, when executed by a processor, perform a method, the method comprising:
receiving a depth map for a multi-view frame, the multi-view frame comprises a multi-view coding block and the multi-view coding block comprises a 2D coding block;
computing a depth quality sensitivity measure for the multi-view coding block in the multi-view frame based in part on the received depth map;
computing a depth-based perceptual quantization scale for the multi-view coding block, wherein the depth-based perceptual quantization scale is based on the depth quality sensitive measure and a base quantization scale for the multi-view frame; and
encoding the 2D coding block using the depth-based perceptual quantization scale.
20. The computer readable medium of claim 19 , wherein computing a depth quality sensitivity measure comprises computing the depth quality sensitivity measure as a function of a depth of the multi-view coding block and a depth with a least quality sensitivity.