Imputation of 3D data using generative adversarial networks

1. A non-transitory computer readable storage medium having stored thereon computer instructions that, when executed by one or more processors, cause the one or more processors to:

obtain one or more training three-dimensional point clouds;

extract one or more three-dimensional regions from each training three-dimensional point cloud, wherein extracting the one or more three-dimensional regions from each training three-dimensional point cloud includes creating one or more gaps in each three-dimensional point cloud corresponding to each of the one or more extracted three-dimensional regions,

train the generative adversarial network by:

analyzing the extracted three-dimensional regions and each three-dimensional point cloud including the respective one or more gaps, wherein the analyzing includes generating a loss value, and

updating one or more weights of the generative adversarial network by backpropagating the loss value throughout the generative adversarial network; and

store the updated weights of the generative adversarial network on the non-transitory computer readable storage medium as parameters for initializing the generative adversarial network.

2. The non-transitory computer readable storage medium of claim 1 , having stored thereon further instructions to:

obtain a three-dimensional point cloud having one or more gaps;

initialize the generative adversarial network using the stored weights; and

impute one or both of (i) RGB data, and (ii) elevation data into the gaps of the three-dimensional point cloud by analyzing the three-dimensional point cloud using the initialized generative adversarial network.

3. The non-transitory computer readable storage medium of claim 2 , having stored thereon further instructions to:

store the three-dimensional point cloud including the imputed data on the computer readable storage medium.

4. The non-transitory computer readable storage medium of claim 2 , having stored thereon further instructions to:

generate the three-dimensional point cloud using a structure-from-motion technique.

5. The non-transitory computer readable storage medium of claim 1 , wherein the gaps include one or both of (i) an implicit gap, and (ii) an explicit gap.

6. The non-transitory computer readable storage medium of claim 1 , having stored thereon further instructions to:

backpropagate discriminator loss to a discriminator artificial neural network.

7. The non-transitory computer readable storage medium of claim 1 , having stored thereon further instructions to:

backpropagate discriminator loss to a discriminator artificial neural network and a generator artificial neural network.

8. A computer-implemented method for training a generative adversarial network, comprising:

obtaining one or more training three-dimensional point clouds;

extracting one or more three-dimensional regions from each training three-dimensional point cloud, wherein extracting the one or more three-dimensional regions from each training three-dimensional point cloud includes creating one or more gaps in each three-dimensional point cloud corresponding to each of the one or more extracted three-dimensional regions,

training the generative adversarial network by:

analyzing the extracted three-dimensional regions and each three-dimensional point cloud including the respective one or more gaps, wherein the analyzing includes generating a loss value, and

updating one or more weights of the generative adversarial network by backpropagating the loss value throughout the generative adversarial network; and

storing the updated weights of the generative adversarial network on a non-transitory computer readable storage medium as parameters for initializing the generative adversarial network.

9. The computer-implemented method of claim 8 , further comprising:

obtaining a three-dimensional point cloud having one or more gaps;

initializing the generative adversarial network using the stored weights; and

imputing one or both of (i) RGB data, and (ii) elevation data into the gaps of the three-dimensional point cloud by analyzing the three-dimensional point cloud using the initialized generative adversarial network.

10. The computer-implemented method of claim 9 , further comprising:

storing the three-dimensional point cloud including the imputed data on the computer readable storage medium.

11. The computer-implemented method of claim 9 , wherein obtaining the three-dimensional point cloud having one or more gaps includes generating the three-dimensional point cloud using a structure-from-motion technique.

12. The computer-implemented method of claim 8 , wherein the gaps include one or both of (i) an implicit gap, and (ii) an explicit gap.

13. The computer-implemented method of claim 8 , wherein updating the one or more weights of the generative adversarial network by backpropagating the loss value throughout the generative adversarial network includes backpropagating discriminator loss to a discriminator artificial neural network.

14. The computer-implemented method of claim 8 , wherein updating the one or more weights of the generative adversarial network by backpropagating the loss value throughout the generative adversarial network includes backpropagating discriminator loss to a discriminator artificial neural network and a generator artificial neural network.

15. A server comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the server to

obtain one or more training three-dimensional point clouds;

extract one or more three-dimensional regions from each training three-dimensional point cloud, wherein extracting the one or more three-dimensional regions from each training three-dimensional point cloud includes creating one or more gaps in each three-dimensional point cloud corresponding to each of the one or more extracted three-dimensional regions,

train the generative adversarial network by:

analyzing the extracted three-dimensional regions and each three-dimensional point cloud including the respective one or more gaps, wherein the analyzing includes generating a loss value, and

updating one or more weights of the generative adversarial network by backpropagating the loss value throughout the generative adversarial network; and

store the updated weights of the generative adversarial network on a non-transitory computer readable storage medium as parameters for initializing the generative adversarial network.

16. The server of claim 15 , the memory storing further instructions that, when executed by the one or more processors, cause the server to

obtain a three-dimensional point cloud having one or more gaps;

initialize the generative adversarial network using the stored weights; and

impute one or both of (i) RGB data, and (ii) elevation data into the gaps of the three-dimensional point cloud by analyzing the three-dimensional point cloud using the initialized generative adversarial network.

17. The server of claim 15 , wherein the gaps include one or both of (i) an implicit gap, and (ii) an explicit gap.

18. The server of claim 15 , the memory storing further instructions that, when executed by the one or more processors, cause the server to backpropagate discriminator loss to a discriminator artificial neural network.

19. The server of claim 15 , the memory storing further instructions that, when executed by the one or more processors, cause the server to

backpropagate discriminator loss to a discriminator artificial neural network and a generator artificial neural network.

20. The server of claim 15 , the memory storing further instructions that, when executed by the one or more processors, cause the server to

display the three-dimensional point cloud including the imputed data in the display device of a user.