Rendering device, rendering method, 3D model generating method, neural network model generating method, and non-transitory computer readable medium

1. A projection data generating device for generating projection data in a 2D space, comprising:

at least one memory; and

at least one processor configured to:

acquire information about the projection data in the 2D space from information about a 3D model;

store, in the at least one memory, (i) the information about the projection data in the 2D space and (ii) information associating identification information given to a part of the 3D model with a pixel where the part of the 3D model is projected in the 2D space; and

generate the projection data in the 2D space based on the information about the projection data in the 2D space.

2. The projection data generating device according to claim 1 , wherein

the at least one processor is configured to generate the projection data using deferred shading.

3. The projection data generating device according to claim 2 , wherein

the at least one memory is a G buffer (Geometry buffer).

4. The projection data generating device according to claim 1 , wherein

the at least one processor is configured to generate the projection data by a differentiable arithmetic operation.

5. The projection data generating device according to claim 1 , wherein

the at least one processor is configured to generate the projection data by a neural network.

6. The projection data generating device according to claim 5 , wherein

the at least one processor is configured to update the neural network using information indicating an error between supervised data and the projection data.

7. The projection data generating device according to claim 1 , wherein

the part of the 3D model has a shape of a polygon.

8. The projection data generating device according to claim 1 , wherein

the information about the projection data includes information about at least one of color, normal vector, albedo, depth, distance, reflection, emission, diffusion, ambient occlusion, lightmap, or data to be used for lighting calculation.

9. The projection data generating device according to claim 1 , wherein

the at least one processor is configured to perform a backpropagation by propagating information indicating an error between supervised data and the projection data back to the 3D model using the associating information.

10. The projection data generating device according to claim 9 , wherein

the at least one processor is configured to update the 3D model based on the backpropagation.

11. The projection data generating device according to claim 1 , wherein

the at least one processor is configured to generate the projection data using at least one of a shading or a rendering.

12. The projection data generating device according to claim 11 , wherein

the at least one processor is configured to update parameters for the at least one of the shading or the rendering by using an error between supervised data and the projection data.

13. The projection data generating device according to claim 11 , wherein

the at least one processor is configured to generate the projection data by using a forward shading.

14. The projection data generating device according to claim 1 , wherein

the information about the 3D model is at least one of identifier data, or coordinate data of the part of the 3D model.

15. The projection data generating device according to claim 1 , wherein

the at least one processor is further configured to store, in the at least one memory, information associating identification information given to another part of the 3D model with another pixel where the another part of the 3D model is projected in the 2D space.

16. A method for generating projection data in a 2D space comprising:

acquiring, by at least one processor, information about the projection data in the 2D space from information about a 3D model;

storing, by the at least one processor in at least one memory, (i) the information about the projection data in the 2D space and (ii) information associating identification information given to a part of the 3D model with a pixel where the part of the 3D model is projected in the 2D space; and

generating, by the at least one processor, the projection data in the 2D space based on the information about the projection data in the 2D space.

17. The method according to claim 16 , further comprising:

performing, by the at least one processor, a backpropagation by propagating information indicating an error between supervised data and the projection data back to the 3D model using the associating information.

18. The method according to claim 17 , further comprising:

updating, by the at least one processor, the 3D model based on the backpropagation.

19. A non-transitory computer readable medium storing therein a program which, when executed by at least one processor performs a method comprising:

acquiring, by at least one processor, information about projection data in a 2D space from information about a 3D model;

storing, by the at least one processor in at least one memory, (i) the information about the projection data in the 2D space and (ii) information associating identification information given to a part of the 3D model with a pixel where the part of the 3D model is projected in the 2D space; and

generating, by the at least one processor, the projection data in the 2D space based on the information about the projection data in the 2D space.

20. The non-transitory computer readable medium according to claim 19 , wherein the method further comprises:

performing, by the at least one processor, a backpropagation by propagating information indicating an error between supervised data and the projection data back to the 3D model using the associating information.

21. The non-transitory computer readable medium according to claim 20 , wherein the method further comprises:

updating, by the at least one processor, the 3D model based on the backpropagation.