System and method for transmission, processing, and rendering of stereoscopic and multi-view images
A digital image processing system takes color plus Z channel data as input, preprocesses, decimates, and codes the Z channel in-band as digital watermark data embedded within the color data prior to encoding and transmission. A second digital image processing system receives, decodes, and extracts the decimated Z channel data before applying statistical regularization to restore a full-resolution Z channel prior to depth-image-based rendering.
1. A method, comprising:
extracting, by a computer processor, a depth map and color data from at least one 3D image;
reducing, by the computer processor, bandwidth of the depth map to produce a reduced bandwidth depth map comprising:
generating a histogram of depth values in the depth map;
performing a modal analysis on the histogram;
identifying one or more depth values in the histogram associated with a modality in a distribution representing depth values in the depth map, wherein the modality is the most frequently occurring depth value of the one or more depth values in the distribution; and
removing the identified one or more depth values associated with the modality in the depth map to produce the reduced bandwidth depth map;
inserting, by the computer processor, the reduced bandwidth depth map into the color data to produce a reduced bandwidth 3D image; and
transmitting, by the computer processor, the reduced bandwidth 3D image into a transmission channel for delivery to a display.
2. The method of claim 1 , wherein reducing bandwidth of the depth map comprises retaining at least one region of the depth map comprising at least one discontinuity that corresponds to at least one object boundary in the color data.
3. The method of claim 1 , further comprising applying a grayscale morphological closing operation on the histogram to remove 0.0 or 1.0 depth values.
4. The method of claim 1 , further comprising filtering the depth map with a median image processing filter.
5. The method of claim 4 , wherein the median image processing filter is a 5×5 median filter.
6. The method of claim 1 , further comprising:
performing a spatial decimation operation on the depth map, and
applying a lossless method of statistical coding on the depth map.
7. The method of claim 6 , wherein performing a spatial decimation operation on the depth map comprises at least one of applying a cubic reduction filter to the depth map or performing a repeated succession of one-octave bicubic reductions.
8. The method of claim 6 , wherein applying a lossless method of statistical coding on the depth map further comprises at least one of:
transforming the depth map to a 7-bit-per-pixel representation and encoding statistically with a Huffmann encoding scheme,
applying arithmetic coding to the depth map, or
applying a two-dimensional codebook encoding scheme to the depth map.
9. The method of claim 1 , wherein inserting the reduced bandwidth depth map into the color data comprises adding the depth map to color data as a watermark.
10. A non-transitory computer-readable storage medium including instructions that, when accessed by a computer processor, cause the computer processor to perform operations, comprising:
extracting, by the computer processor, a depth map and color data from at least one 3D image;
reducing, by the computer processor, bandwidth of the depth map to produce a reduced bandwidth depth map comprising:
generating a histogram of depth values in the depth map;
performing a modal analysis on the histogram;
identifying one or more depth values in the histogram associated with a modality in a distribution representing depth values in the depth map, wherein the modality is the most frequently occurring depth value of the one or more depth values in the distribution; and
removing the identified one or more depth values associated with the modality in the depth map to produce the reduced bandwidth depth map;
inserting, by the computer processor, the reduced bandwidth depth map into the color data to produce a reduced bandwidth 3D image; and
transmitting, by the computer processor, the reduced bandwidth 3D image into a transmission channel for delivery to a display.
11. The non-transitory computer-readable storage medium of claim 10 , wherein reducing bandwidth of the depth map comprises retaining at least one region of the depth map comprising at least one discontinuity that corresponds to at least one object boundary in the color data.
12. A system, comprising:
a memory;
a computer processor, operatively coupled to the memory, the computer processor to:
extract a depth map and color data from at least one 3D image;
reduce bandwidth of the depth map to produce reduced a bandwidth depth map comprising:
generating a histogram of depth values in the depth map;
performing a modal analysis on the histogram;
identifying one or more depth values in the histogram associated with a modality in a distribution representing depth values in the depth map, wherein the modality is the most frequently occurring depth value of the one or more depth values in the distribution; and
removing the identified one or more depth values associated with the modality in the depth map to produce the reduced bandwidth depth map;
insert the reduced bandwidth depth map into the color data to produce a reduced bandwidth 3D image; and
transmit the reduced bandwidth 3D image into a transmission channel for delivery to a display.
13. The system of claim 12 , wherein reducing bandwidth of the depth map comprises retaining at least one region of the depth map comprising at least one discontinuity that corresponds to at least one object boundary in the color data.