Method and apparatus for point cloud color processing

1. A method for reducing color leaking artefacts in an image formed by projection processing from a 3D point cloud, the method comprising:

receiving an input image comprising the 3D point cloud;

classifying the 3D point cloud into a plurality of 3D surface patches;

projecting the 3D surface patches onto a 2D image plane to form a first 2D image;

processing the first 2D image, by coding, transmitting and decoding, to form a final 2D image; and

providing the final 2D image as an output;

wherein processing comprises coding comprising background filling of pixels between patches to reduce inter-patch color leakage in the final 2D image;

wherein background filling of pixels between patches comprises:

for each patch, displacing active pixels at a patch border by a safeguard distance towards the patch interior; and

for each patch, filling the safeguard distance with a background fill.

2. The method of claim 1 , wherein the safeguard distance is selected to be less than or equal to 3 pixels.

3. The method of claim 1 , additionally comprising processing the final 2D image to reconstruct an output 3D point cloud;

wherein the background-filled pixels in the safeguard distance are used to replicate information across patches, such that patch seam mismatch artefacts are minimized in the reconstruction.

4. A method for reducing color leaking artefacts in an image formed by projection processing from a 3D point cloud, the method comprising:

receiving an input image comprising the 3D point cloud;

classifying the 3D point cloud into a plurality of 3D surface patches;

projecting the 3D surface patches onto a 2D image plane to form a first 2D image;

processing the first 2D image, by coding, transmitting and decoding, to form a final 2D image; and

providing the final 2D image as an output;

wherein processing comprises coding comprising background filling of pixels between patches to reduce inter-patch color leakage in the final 2D image;

wherein background filling of pixels between patches comprises:

dilating each patch to increase the number of apparently active pixels;

for each patch, displacing apparently active pixels at a patch border by a safeguard distance towards the patch interior; and

for each patch, filling the safeguard distance with a background fill.

5. A method for reducing color leaking artefacts in an image formed by projection processing from a 3D point cloud, the method comprising:

receiving an input image comprising the 3D point cloud;

classifying the 3D point cloud into a plurality of 3D surface patches;

projecting the 3D surface patches onto a 2D image plane to form a first 2D image;

processing the first 2D image, by coding, transmitting and decoding, to form a final 2D image; and

providing the final 2D image as an output;

wherein processing comprises coding comprising background filling of pixels between patches to reduce inter-patch color leakage in the final 2D image;

wherein background filling of pixels between patches comprises an optimized background filling scheme comprising:

calculating an average value for each blob of pixels in the first 2D image;

normalizing values for pixels in each blob relative to the calculated average value; setting all background pixels to zero; and

adding a global bias to all normalized pixels and zeroed background pixels to yield an image with no negative pixel values.

6. An apparatus comprising:

one or more processors; and

logic encoded in one or more non-transitory media for execution by the one or more processors and when executed operable to:

receive an input image comprising a 3D point cloud;

classify the 3D point cloud into a plurality of 3D surface patches;

project the 3D surface patches onto a 2D image plane to form a first 2D image;

process the first 2D image, by coding, transmitting and decoding, to form a final 2D image; and

provide the final 2D image as an output;

wherein processing comprises coding comprising background filling of pixels between patches to reduce inter-patch color leakage in the final 2D image;

wherein background filling of pixels between patches comprises:

for each patch, displacing active pixels at a patch border by a safeguard distance towards the patch interior; and

for each patch, filling the safeguard distance with a background fill.

7. The apparatus of claim 6 , wherein the safeguard distance is selected to be less than or equal to 3 pixels.

8. An apparatus comprising:

one or more processors; and

logic encoded in one or more non-transitory media for execution by the one or more processors and when executed operable to:

receive an input image comprising a 3D point cloud;

classify the 3D point cloud into a plurality of 3D surface patches;

project the 3D surface patches onto a 2D image plane to form a first 2D image;

process the first 2D image, by coding, transmitting and decoding, to form a final 2D image; and

provide the final 2D image as an output;

wherein processing comprises coding comprising background filling of pixels between patches to reduce inter-patch color leakage in the final 2D image;

wherein background filling of pixels between patches comprises:

dilating each patch to increase the number of apparently active pixels;

for each patch, displacing apparently active pixels at a patch border by a safeguard distance towards the patch interior; and

for each patch, filling the safeguard distance with a background fill.

9. An apparatus comprising:

one or more processors; and

logic encoded in one or more non-transitory media for execution by the one or more processors and when executed operable to:

receive an input image comprising a 3D point cloud;

classify the 3D point cloud into a plurality of 3D surface patches;

project the 3D surface patches onto a 2D image plane to form a first 2D image;

process the first 2D image, by coding, transmitting and decoding, to form a final 2D image; and

provide the final 2D image as an output;

wherein processing comprises coding comprising background filling of pixels between patches to reduce inter-patch color leakage in the final 2D image;

wherein background filling of pixels between patches comprises an optimized background filling scheme comprising:

calculating an average value for each blob of pixels in the first 2D image;

normalizing values for pixels in each blob relative to the calculated average value; setting all background pixels to zero; and

adding a global bias to all normalized pixels and zeroed background pixels to yield an image with no negative pixel values.

10. An apparatus comprising:

one or more processors; and

software encoded in one or more non-transitory computer-readable media for execution by the one or more processors and when executed operable to reduce color leaking artefacts in an image formed by projection processing from a 3D point cloud, by:

receiving an input image comprising the 3D point cloud;

classifying the 3D point cloud into a plurality of 3D surface patches;

projecting the 3D surface patches onto a 2D image plane to form a first 2D image;

processing the first 2D image, by coding, transmitting and decoding, to form a final 2D image; and

providing the final 2D image as an output;

wherein processing comprises coding comprising background filling of pixels between patches to reduce inter-patch color leakage in the final 2D image;

wherein background filling of pixels between patches comprises:

for each patch, displacing active pixels at a patch border by a safeguard distance towards the patch interior; and

for each patch, filling the safeguard distance with a background fill.

11. The apparatus of claim 10 , wherein the safeguard distance is selected to be less than or equal to 3 pixels.

12. An apparatus comprising:

one or more processors; and

software encoded in one or more non-transitory computer-readable media for execution by the one or more processors and when executed operable to reduce color leaking artefacts in an image formed by projection processing from a 3D point cloud, by:

receiving an input image comprising the 3D point cloud;

classifying the 3D point cloud into a plurality of 3D surface patches;

projecting the 3D surface patches onto a 2D image plane to form a first 2D image;

processing the first 2D image, by coding, transmitting and decoding, to form a final 2D image; and

providing the final 2D image as an output;

wherein processing comprises coding comprising background filling of pixels between patches to reduce inter-patch color leakage in the final 2D image;

wherein background filling of pixels between patches comprises:

dilating each patch to increase the number of apparently active pixels;

for each patch, displacing apparently active pixels at a patch border by a safeguard distance towards the patch interior; and

for each patch, filling the safeguard with a background fill.