System and method for transmission, processing, and rendering of stereoscopic and multi-view images

1. A computer implemented method comprising:

receiving a reduced bandwidth 3D image comprising a reduced bandwidth depth map and color data;

applying, using a processor, a filter employing a statistical domain of the hue of the color data in a color space comprising a plurality of color channels to restore bandwidth of the reduced bandwidth depth map, the filter employing two radiosity weighting functions, the first radiosity weighting function provided by the reduced bandwidth depth map as a first domain, the second radiosity weighting function provided by a multi-channel color difference measure of the color data represented in the color space with at least one channel of the plurality of color channels incorporating a hue value of the color data and incorporating a gradient of the hue value of the color data as a second domain, the filter employing a spatial weighting function; and

repeatedly subjecting the reduced bandwidth depth map to iterations of one-octave bicubic up-scaling followed by application of the filter until a first octave up-scaled depth map is produced, wherein repeatedly subjecting the reduced bandwidth depth map to iterations of one-octave bicubic up-scaling followed by application of the filter minimizes an error function comprising a difference between an edge gradient of the depth map and a color edge gradient of the color data to regularize output of the up-scaled depth map.

2. The method of claim 1 , wherein the filter is configured to restore discontinuities in the reduced bandwidth depth map by matching discontinuities of the color data containing at least one object boundary.

3. The method of claim 1 , further comprising applying a depth-image-based rendering (DIBR) method to warp the restored depth map and the color image to construct at least one view suitable for stereoscopic or auto-stereoscopic 3D displays.

4. The method of claim 3 , wherein applying a DIBR method further comprises:

applying an optical flow method to the color data;

applying motion compensation and image warping to the color to produce a table of motion compensated pixels;

applying one of temporal predictions and spatial predictions of candidate occluded pixels from the table of motion compensated pixels;

applying a statistical in-painting procedure to the candidate occluded pixels; and

warping pixels obtained from the statistical in-painting procedure to obtain left and right eye views of images for display.

5. The method of claim 4 , further comprising:

classifying disocclusions from the depth map to inform spatial predictions of candidate occluded pixels; and

applying a Z smoothing method to the depth map to produce a processed depth map.

6. The method of claim 5 , wherein the warping is informed by the processed depth map.

7. The method of claim 1 , further comprising applying a lossless decoding method to the reduced bandwidth depth map before repeatedly subjecting the reduced bandwidth depth map to iterations of one-octave bicubic up-scaling followed by application of the filter.

8. A non-transitory computer-readable data storage medium comprising computer-executable instructions for processing three-dimensional (3D) imagery, the instructions to:

receive a reduced bandwidth 3D image comprising a reduced bandwidth depth map and color data;

apply a filter employing a statistical domain of the hue of the color data in a color space comprising a plurality of color channels to restore bandwidth of the reduced bandwidth depth map, the filter employing two radiosity weighting functions, the first radiosity weighting function provided by the reduced bandwidth depth map as a first domain, the second radiosity weighting function provided by a multi-channel color difference measure of the color data represented in the color space with at least one channel of the plurality of color channels incorporating a hue value of the color data and incorporating a gradient of the hue value of the color data as a second domain, the filter employing a spatial weighting function; and

repeatedly subject the reduced bandwidth depth map to iterations of one-octave bicubic up-scaling followed by application of the filter until a first octave up-scaled depth map is produced, wherein repeatedly subjecting the reduced bandwidth depth map to iterations of one-octave bicubic up-scaling followed by application of the filter minimizes an error function comprising a difference between an edge gradient of the depth map and a color edge gradient of the color data to regularize output of the up-scaled depth map.

9. The non-transitory computer-readable data storage medium of claim 8 , wherein the instructions are further to:

apply a depth-image-based rendering (DIBR) method to warp the restored depth map and the color image to construct at least one view suitable for stereoscopic or auto-stereoscopic 3D displays.

10. The non-transitory computer-readable data storage medium of claim 8 , wherein the at least one filter is operable to employ a statistical domain of the color data to restore the bandwidth of the reduced bandwidth depth map.

11. A receiver for receiving and restoring at least one reduced bandwidth image comprising a reduced bandwidth depth map, the receiver to:

receive a reduced bandwidth 3D image comprising a reduced bandwidth depth map and color data;

apply a filter employing a statistical domain of the hue of the color data in a color space comprising a plurality of color channels to restore bandwidth of the reduced bandwidth depth map, the filter employing two radiosity weighting functions, the first radiosity weighting function provided by the reduced bandwidth depth map as a first domain, the second radiosity weighting function provided by a multi-channel color difference measure of the color data represented in the color space with at least one channel of the plurality of color channels incorporating a hue value of the color data and incorporating a gradient of the hue value of the color data as a second domain, the filter employing a spatial weighting function; and

repeatedly subject the reduced bandwidth depth map to iterations of one-octave bicubic up-scaling followed by application of the filter until a first octave up-scaled depth map is produced, wherein repeatedly subjecting the reduced bandwidth depth map to iterations of one-octave bicubic up-scaling followed by application of the filter minimizes an error function comprising a difference between an edge gradient of the depth map and a color edge gradient of the color data to regularize output of the up-scaled depth map.

12. The receiver of claim 11 , wherein restoring bandwidth of the reduced bandwidth depth map is based on restoring discontinuities in the reduced bandwidth depth map by matching discontinuities of the color data containing at least one object boundary.

13. The receiver of claim 11 , further comprising a depth-image-based rendering (DIBR) module for warping the restored depth map and the color data to construct at least one view suitable for stereoscopic or auto-stereoscopic 3D displays.

14. The receiver of claim 11 , wherein the receiver is implemented using an application-specific integrated circuit (ASIC).

15. A system for transmitting, receiving, and rendering 3D imagery, comprising:

a transmitter for:

extracting a depth map and color data from at least one 3D image,

reducing spatial bandwidth of the depth map by applying one or more one-octave decimations to the depth map using a bicubic reduction filter,

generating a histogram of depth values in the depth map,

removing depth values associated with a modality obtained from the histogram,

selecting the modality by computing the largest of a plurality of maxima values of the histogram to reduce the bit-per-pixel bandwidth of the depth map to produce a reduced bandwidth depth map,

inserting the reduced bandwidth depth map into the color data to produce a reduced bandwidth 3D image, and

transmitting the reduced bandwidth 3D image into a transmission channel; and

a receiver for:

receiving a reduced bandwidth 3D image comprising a reduced bandwidth depth map and color data,

applying a filter employing a statistical domain of the hue of the color data in a color space comprising a plurality of color channels to restore bandwidth of the reduced bandwidth depth map, the filter employing two radiosity weighting functions, the first radiosity weighting function provided by the reduced bandwidth depth map as a first domain, the second radiosity weighting function provided by a multi-channel color difference measure of the color data represented in the color space with at least one channel of the plurality of color channels incorporating a hue value of the color data and incorporating a gradient of the hue value of the color data as a second domain, the filter employing a spatial weighting function,

repeatedly subject the reduced bandwidth depth map to iterations of one-octave bicubic up-scaling followed by application of the filter until a first octave up-scaled depth map is produced, wherein repeatedly subjecting the reduced bandwidth depth map to iterations of one-octave bicubic up-scaling followed by application of the filter minimizes an error function comprising a difference between an edge gradient of the depth map and a color edge gradient of the color data to regularize output of the up-scaled depth map, and

rendering the restored depth map and color data on a display.

16. The system of claim 15 , wherein reducing bandwidth of the depth map comprises retaining at least one region of the depth map comprising at least one discontinuity that corresponds to at least one object boundary in the color data.

17. The system of claim 15 , wherein the filter is configured to restore discontinuities in the reduced bandwidth depth map by matching discontinuities of the color data containing at least one object boundary.