Higher-order function networks for learning composable three-dimensional (3D) object and operating method thereof
An apparatus for representing a three-dimensional (3D) object, the apparatus includes a memory storing instructions, and a processor configured to execute the instructions to transmit a two-dimensional (2D) image to an external device, based on the 2D image being transmitted, receive, from the external device, mapping function parameters that are obtained using a first neural network, set a mapping function of a second neural network, based on the received mapping function parameters, and based on 3D samples, obtain the 3D object corresponding to the 2D image, using the second neural network of which the mapping function is set.
1. An apparatus for representing a three-dimensional (3D) object, the apparatus comprising:
a memory storing instructions; and
a processor configured to execute the instructions to:
transmit a two-dimensional (2D) image to an external device;
based on the 2D image being transmitted, receive, from the external device, mapping function parameters that are obtained using a first neural network;
set a mapping function of a second neural network, based on the received mapping function parameters; and
based on 3D samples, obtain the 3D object corresponding to the 2D image, using the second neural network of which the mapping function is set.
2. The apparatus of claim 1 , wherein the 3D samples are of a surface or an interior of a 3D canonical domain.
3. The apparatus of claim 2 , wherein the 3D canonical domain is a unit sphere or a unit cube.
4. The apparatus of claim 1 , wherein the 3D object is a surface or an interior of an object included in the 2D image.
5. The apparatus of claim 1 , wherein the first neural network is a convolutional neural network (CNN) that is trained to output the mapping function parameters, based on the 2D image that is input in the CNN, and is trained in connection with the second neural network.
6. The apparatus of claim 1 , wherein the second neural network is a convolutional neural network (CNN) that is trained to output the 3D object, based on the 3D samples that are input in the CNN, and is trained in connection with the first neural network.
7. The apparatus of claim 1 , wherein the apparatus is a client device that is separate and external from the external device.
8. The apparatus of claim 1 , wherein the external device is a server device that is separate and external from the apparatus.
9. A method of representing a three-dimensional (3D) object, the method being performed by an apparatus, and the method comprising:
transmitting a two-dimensional (2D) image to an external device;
based on the 2D image being transmitted, receiving, from the external device, mapping function parameters that are obtained using a first neural network;
setting a mapping function of a second neural network, based on the received mapping function parameters; and
based on 3D samples, obtaining the 3D object corresponding to the 2D image, using the second neural network of which the mapping function is set.
10. The method of claim 9 , wherein the 3D samples are of a surface or an interior of a 3D canonical domain.
11. The method of claim 10 , wherein the 3D canonical domain is a unit sphere or a unit cube.
12. The method of claim 9 , wherein the 3D object is a surface or an interior of an object included in the 2D image.
13. The method of claim 9 , wherein the first neural network is a convolutional neural network (CNN) that is trained to output the mapping function parameters, based on the 2D image that is input in the CNN, and is trained in connection with the second neural network.
14. The method of claim 9 , wherein the second neural network is a convolutional neural network (CNN) that is trained to output the 3D object, based on the 3D samples that are input in the CNN, and is trained in connection with the first neural network.
15. The method of claim 9 , wherein the apparatus is a client device that is separate and external from the external device.
16. The method of claim 9 , wherein the external device is a server device that is separate and external from the apparatus.
17. A non-transitory computer-readable storage medium storing instructions configured to cause a processor to:
transmit a two-dimensional (2D) image to an external device;
based on the 2D image being transmitted, receive, from the external device, mapping function parameters that are obtained using a first neural network;
set a mapping function of a second neural network, based on the received mapping function parameters; and
based on 3D samples, obtain the 3D object corresponding to the 2D image, using the second neural network of which the mapping function is set.
18. The non-transitory computer-readable storage medium of claim 17 , wherein the 3D samples are of a surface or an interior of a 3D canonical domain.
19. The non-transitory computer-readable storage medium of claim 18 , wherein the 3D canonical domain is a unit sphere or a unit cube.
20. The non-transitory computer-readable storage medium of claim 17 , wherein the 3D object is a surface or an interior of an object included in the 2D image.