Digital model repair system and method
A digital model repair method includes: providing a point cloud digital model of a target object as input to a generative network of a trained generative adversarial network ‘GAN’, the input point cloud comprising a plurality of points erroneously perturbed by one or more causes, and generating, by the generative network of the GAN, an output point cloud in which the erroneous perturbation of some or all of the plurality of points has been reduced; where the generative network of the GAN was trained using input point clouds comprising a plurality of points erroneously perturbed by said one or more causes, and a discriminator of the GAN was trained to distinguish point clouds comprising a plurality of points erroneously perturbed by said one or more causes and point clouds substantially without such perturbations.
1. A digital model repair method, comprising the steps of:
providing a point cloud digital model of a target object as input to a generative network of a trained generative adversarial network ‘GAN’, the input point cloud comprising a plurality of points erroneously perturbed by one or more causes; and
generating, by the generative network of the GAN, an output point cloud in which the erroneous perturbation of some or all of the plurality of points has been reduced;
wherein the generative network of the GAN was trained using input point clouds comprising a plurality of points erroneously perturbed by said one or more causes, and a discriminator of the GAN was trained to distinguish point clouds comprising a plurality of points erroneously perturbed by said one or more causes and point clouds substantially without such perturbations, and
wherein the discriminator is trained to discriminate between substantially error free target point cloud data and point cloud data comprising points erroneously perturbed by one or more causes, the substantially error free target point cloud data is generated by the application of a noise reduction technique to the point cloud data comprising points erroneously perturbed by the one or more causes, and the noise reduction technique applies winding numbers to the point cloud.
2. The method of claim 1 , comprising: rendering an image responsive to the point cloud output by the generative network of the trained GAN.
3. The method of claim 1 , in which
the discriminator of the GAN is first trained on substantially error-free target point cloud data until it can identify this to a predetermined level of accuracy; and
the discriminator is secondly trained to discriminate between substantially error free target point cloud data and point cloud data comprising points erroneously perturbed by one or more causes.
4. The method of claim 3 , in which the discriminator is trained to discriminate between substantially error free target point cloud data and point cloud data output by the generative network of the GAN.
5. The method of claim 1 , in which point cloud data comprising points erroneously perturbed by one or more causes comprises points erroneously perturbed by one or more causes selected from the list consisting of:
i. Gaussian noise
ii. depth estimation error; and
iii. over smoothing at an occlusion boundary.
6. The method of claim 1 , in which point cloud data comprising points erroneously perturbed by one or more causes was derived from data obtained by one selected from the list consisting of:
i. a stereo camera;
ii. a time-of-flight camera; and
iii. one or more cameras in conjunction with a structured light projection.
7. The method of claim 1 , in which the generative network outputs a point cloud that corresponds to a subsection of the input point cloud.
8. The method of claim 1 , in which a point cloud input to the generative network of the GAN further comprises additional random points, provided for remapping by the generative network of the GAN to fill one or more gaps in a surface distribution of the point cloud.
9. The method of claim 1 , in which target point cloud data includes representations of one or more selected from the list consisting of:
i. human faces;
ii. human bodies or parts thereof;
iii. flat surfaces; and
iv. geometric primitives or parts thereof.
10. The method of claim 1 , in which the discriminator is trained to discriminate between substantially error free target point clouds based on a plurality of different point distribution methods, and point cloud data output by the generative network of the GAN.
11. A non-transitory machine-readable medium comprising computer executable instructions adapted to cause a computer system to carry out actions, comprising:
providing a point cloud digital model of a target object as input to a generative network of a trained generative adversarial network ‘GAN’, the input point cloud comprising a plurality of points erroneously perturbed by one or more causes; and
generating, by the generative network of the GAN, an output point cloud in which the erroneous perturbation of some or all of the plurality of points has been reduced;
wherein the generative network of the GAN was trained using input point clouds comprising a plurality of points erroneously perturbed by said one or more causes, and a discriminator of the GAN was trained to distinguish point clouds comprising a plurality of points erroneously perturbed by said one or more causes and point clouds substantially without such perturbations, and
wherein the discriminator is trained to discriminate between substantially error free target point cloud data and point cloud data comprising points erroneously perturbed by one or more causes, the substantially error free target point cloud data is generated by the application of a noise reduction technique to the point cloud data comprising points erroneously perturbed by the one or more causes, and the noise reduction technique applies winding numbers to the point cloud.
12. A digital model repair apparatus, comprising:
a trained generative adversarial network ‘GAN’;
input means operable to provide a point cloud digital model of a target object as input to a generative network of the trained GAN, the input point cloud comprising a plurality of points erroneously perturbed by one or more causes; and
the generative network of the GAN being operable to generate an output point cloud in which the erroneous perturbation of some or all of the plurality of points has been reduced;
wherein the generative network of the GAN was trained using input point clouds comprising a plurality of points erroneously perturbed by said one or more causes, and a discriminator of the GAN was trained to distinguish point clouds comprising a plurality of points erroneously perturbed by said one or more causes and point clouds substantially without such perturbations, and
wherein the discriminator is trained to discriminate between substantially error free target point cloud data and point cloud data comprising points erroneously perturbed by one or more causes, the substantially error free target point cloud data is generated by the application of a noise reduction technique to the point cloud data comprising points erroneously perturbed by the one or more causes, and the noise reduction technique applies winding numbers to the point cloud.
13. The apparatus of claim 12 , comprising: a rendering processor operable to render an image responsive to the point cloud output by the generative network of the trained GAN.