Graphics rendering using a neural network
Apparatuses, systems, and techniques to generate a surface. In at least one embodiment, one or more neural networks are used to generate a surface of an object based, at least in part, on motion of the object.
1 . One or more processors, comprising:
circuitry to use one or more neural networks to generate a first surface of an object based, at least in part, on a plurality of second surfaces generated by the one or more neural networks, wherein each of the plurality of second surfaces corresponds to a specific joint involved in a motion of the object, and wherein the one or more neural networks aggregate the plurality of second surfaces to generate the first surface.
2 . The one or more processors of claim 1 , wherein the first surface is to be generated based, at least in part, on a signed distance field.
3 . The one or more processors of claim 1 , wherein the first surface is to be generated based, at least in part, on one or more joint positions of the object.
4 . The one or more processors of claim 1 , wherein the plurality of second surfaces are to be generated based, at least in part, on one or more randomly generated three-dimensional points.
5 . The one or more processors of claim 1 , wherein the first surface is to be generated based, at least in part, on one or more three-dimensional points located within a bounding volume of the object.
6 . The one or more processors of claim 1 , wherein the one or more neural networks include at least one signed distance field neural network.
7 . The one or more processors of claim 1 , wherein the one or more neural networks include at least one aggregation neural network.
8 . The one or more processors of claim 1 , wherein the first surface is to be generated based, at least in part, on minimizing one or more loss functions of the one or more neural networks.
9 . The one or more processors of claim 1 , wherein the first surface is to be generated based, at least in part, on a subject code associated with the object.
10 . The one or more processors of claim 1 , wherein the motion of the object is to be specified using one or more joint transformations associated with the object.
11 . A computer-implemented method, comprising:
using one or more neural networks to generate a first surface of an object based, at least in part, on a plurality of second surfaces generated by the one or more neural networks, wherein each of the plurality of second surfaces corresponds to a specific joint involved in a motion of the object, and wherein the one or more neural networks aggregate the plurality of second surfaces to generate the first surface.
12 . The method of claim 11 , wherein at least one neural network of the one or more neural networks is to be trained based, at least in part, on a set of meshes associated with the object.
13 . The method of claim 11 , wherein at least one neural network of the one or more neural networks is to be trained based, at least in part, on a set of joint data associated with the object.
14 . The method of claim 11 , wherein at least one neural network of the one or more neural networks is to be trained based, at least in part, on a latent pose associated with the object.
15 . The method of claim 11 , wherein using the one or more neural networks to generate the first surface of the object comprises:
receiving a canonical pose associated with the object;
generating a test pose of the object; and
determining the motion of the object based, at least in part, on one or more transformations between the canonical pose and the test pose.
16 . The method of claim 11 , wherein using the one or more neural networks to generate the first surface of the object comprises:
determining a joint position based, at least in part, on the motion of the object;
selecting a point within a bounding volume of the object;
generating a signed distance field of the point based, at least in part on the joint position; and
generating the first surface based, at least in part, on the signed distance field of the point.
17 . The method of claim 11 , wherein at least one neural network of the one or more neural networks is to be trained based, at least in part, on a loss function of the neural network.
18 . The method of claim 11 , wherein at least one neural network of the one or more neural networks is to be trained based, at least in part, on a least squares loss function of the neural network.
19 . The method of claim 11 , wherein at least one neural network of the one or more neural networks is to be trained based, at least in part, on an eikonal loss function of the neural network.
20 . The method of claim 11 , wherein at least one neural network of the one or more neural networks is to be trained based, at least in part, on:
estimating one or more joint labels based at least in part on a geodesic distance;
determining a per-joint least squares loss function of the neural network based, at least in part, on one or more estimated joint labels; and
training the neural network of the one or more neural networks based at least in part, on the per-joint least squares loss function.
21 . A computer system, comprising:
one or more processors and memory storing executable instructions that, if performed by the one or more processors, cause the one or more processors to use one or more neural networks to generate a first surface of an object based, at least in part, on a plurality of second surfaces generated by the one or more neural networks, wherein each of the plurality of second surfaces corresponds to a specific joint involved in a motion of the object, and wherein the one or more neural networks aggregate the plurality of second surfaces to generate the first surface.
22 . The computer system of claim 21 , wherein at least one of the one or more neural networks is a joint-specific neural network.
23 . The computer system of claim 21 , wherein the instructions, if executed by the one or more processors, cause the first surface of the object to be generated based, at least in part, on a signed distance field.
24 . The computer system of claim 21 , wherein the instructions, if executed by the one or more processors, cause the first surface of the object to be generated as a signed distance field.
25 . The computer system of claim 21 , wherein the instructions, if executed by the one or more processors, cause the first surface of the object to be rendered using raytracing.
26 . The computer system of claim 21 , wherein at least one neural network of the one or more neural networks is to be trained, based at least in part, on a set of joint data associated with the object.
27 . The computer system of claim 21 , wherein at least one neural network of the one or more neural networks is to be trained, based at least in part, on a set of meshes associated with the object.
28 . A machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to use one or more neural networks to generate a first surface of an object based, at least in part, on a plurality of second surfaces generated by the one or more neural networks, wherein each of the plurality of second surfaces corresponds to a specific joint involved in a motion of the object, and wherein the one or more neural networks aggregate the plurality of second surfaces to generate the first surface.
29 . The machine-readable medium of claim 28 , wherein the instructions, if performed by the one or more processors, cause the first surface to be generated based, at least in part, on a signed distance field.
30 . The machine-readable medium of claim 28 , wherein the instructions, if performed by the one or more processors, cause the first surface to be generated based, at least in part, on one or more joint positions of the object.
31 . The machine-readable medium of claim 28 , wherein the instructions, if performed by the one or more processors, cause the first surface to be generated based, at least in part, on one or more randomly generated three-dimensional points.
32 . The machine-readable medium of claim 28 , wherein the instructions, if performed by the one or more processors, cause the first surface to be generated based, at least in part, on one or more three-dimensional points located within a bounding volume of the object.
33 . The machine-readable medium of claim 28 , wherein the one or more neural networks include at least one signed distance field neural network.
34 . The machine-readable medium of claim 28 , wherein the one or more neural networks include at least one aggregation neural network.
35 . The machine-readable medium of claim 28 , wherein the instructions, if performed by the one or more processors, cause the first surface to be generated based, at least in part, on minimizing one or more loss functions of the one or more neural networks.
36 . The machine-readable medium of claim 28 , wherein the instructions, if performed by the one or more processors, cause the first surface to be generated based, at least in part, on a subject code associated with the object.
37 . The machine-readable medium of claim 28 , wherein the set of instructions includes instructions which, if performed by the one or more processors, cause the one or more processors to:
estimate one or more joint labels based at least in part on a geodesic distance;
determine a per-joint eikonal loss function of the one or more neural networks based, at least in part, on the estimated one or more joint labels; and
train a neural network of the one or more neural networks based at least in part, on the per-joint eikonal loss function.
38 . One or more processors, comprising:
circuitry to use one or more neural networks to generate at least shading information to be applied to one or more first surfaces of one or more rendered three-dimensional (3D) objects based, at least in part, on pose information of the one or more first surfaces of one or more rendered 3D objects and a plurality of second surfaces generated by the one or more neural networks, each of the one or more neural networks corresponding to a specific joint of a plurality of joints, wherein a neural network, of the one or more neural networks, corresponding to a specific joint involved in a motion of the objects, generate each of the plurality of second surfaces using at least the neural network, and wherein the one or more neural networks aggregate the plurality of second surfaces to generate the one or more first surfaces.
39 . The one or more processors of claim 38 , wherein the circuitry is to cause the shading information to be generated, based at least in part, on a signed distance field.
40 . The one or more processors of claim 38 , wherein the shading information is to be generated based, at least in part, on the pose information.
41 . The one or more processors of claim 38 , wherein the shading information is to be generated based, at least in part, on one or more three-dimensional points.
42 . The one or more processors of claim 38 , wherein the shading information is to be generated based, at least in part, on one or more randomly generated points.
43 . The one or more processors of claim 38 , wherein the shading information is to be generated based, at least in part, on one or more points located within a bounding volume of at least one rendered 3D object of the one or more rendered 3D objects.
44 . The one or more processors of claim 38 , wherein the one or more neural networks include at least one signed distance field neural network.
45 . The one or more processors of claim 38 , wherein the one or more neural networks include at least one aggregation neural network.
46 . The one or more processors of claim 38 , wherein the shading information is to be generated based, at least in part, on one or more loss functions of the one or more neural networks.
47 . The one or more processors of claim 38 , wherein motion of the one or more first surfaces of one or more rendered 3D objects is based, at least in part, on the pose information of the one or more first surfaces of one or more rendered 3D objects.
48 . A computer-implemented method, comprising:
using one or more neural networks to generate at least shading information to be applied to one or more first surfaces of one or more rendered three-dimensional (3D) objects based, at least in part, on pose information of the one or more first surfaces of one or more rendered 3D objects and a plurality of second surfaces generated by the one or more neural networks, each of the one or more neural networks corresponding to a specific joint of a plurality of joints, wherein a neural network, of the one or more neural networks, corresponding to a specific joint involved in a motion of the objects, generate each of the plurality of second surfaces using at least the neural network, and wherein the one or more neural networks aggregate the plurality of second surfaces to generate the one or more first surfaces.
49 . The method of claim 48 , wherein at least one neural network of the one or more neural networks is to be trained based, at least in part, on a set of meshes associated with the one or more first surfaces of one or more rendered 3D objects.
50 . The method of claim 48 , wherein at least one neural network of the one or more neural networks is to be trained based, at least in part, on the pose information of the one or more first surfaces of one or more rendered 3D objects.
51 . The method of claim 48 , wherein at least one neural network of the one or more neural networks is to be trained based, at least in part, on one of more latent poses associated with the one or more first surfaces of one or more rendered 3D objects.
52 . The method of claim 48 , wherein using the one or more neural networks to generate the shading information comprises:
selecting an object of the one or more objects;
receiving a canonical pose associated with the selected object;
generating one or more test poses of the selected object; and
determining pose information of the selected object based, at least in part, on one or more transformations between the canonical pose and the one or more test poses.
53 . The method of claim 48 , wherein using the one or more neural networks to generate the shading information comprises:
selecting an object of the one or more objects;
determining a joint position based, at least in part, on pose information of the selected object;
selecting a point within a bounding volume of the selected object;
generating a signed distance field of the point based, at least in part on the joint position; and
generating shading information for the selected object based, at least in part, on the signed distance field of the point.
54 . The method of claim 48 , wherein at least one neural network of the one or more neural networks is to be trained based, at least in part, on a per-joint least squares loss function.
55 . A computer system, comprising:
one or more processors and memory storing executable instructions that, if performed by the one or more processors, cause the one or more processors to use one or more neural networks to generate at least shading information to be applied to one or more first surfaces of one or more rendered three-dimensional (3D) objects based, at least in part, on pose information of the one or more first surfaces of one or more rendered 3D objects and a plurality of second surfaces generated by the one or more neural networks, each of the one or more neural networks corresponding to a specific joint of a plurality of joints, wherein a neural network, of the one or more neural networks, corresponding to a specific joint involved in a motion of the objects, generate each of the plurality of second surfaces using at least the neural network, and wherein the one or more neural networks aggregate the plurality of second surfaces to generate the one or more first surfaces.
56 . The computer system of claim 55 , wherein the shading information is to be generated based, at least in part, on a signed distance field.
57 . The computer system of claim 55 , wherein the shading information is to be generated based, at least in part, on one or more joint positions of the one or more first surfaces of one or more rendered 3D objects.
58 . The computer system of claim 55 , wherein the one or more first surfaces of one or more rendered 3D objects are to be rendered using raytracing.
59 . The computer system of claim 55 , wherein the one or more first surfaces of one or more rendered 3D objects are to be rendered using rasterization.
60 . The computer system of claim 55 , wherein the shading information at least includes one or more colors associated with the one or more first surfaces of one or more rendered 3D objects.
61 . The computer system of claim 55 , wherein:
the shading information at least includes one or more textures associated with the one or more first surfaces of one or more rendered 3D objects; and
the shading information at least includes one or more texture coordinates associated with the first surfaces of one or more one or more rendered 3D objects.
62 . The computer system of claim 55 , wherein the shading information at least includes one or more normals associated with the one or more first surfaces of one or more rendered 3D objects.
63 . A machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to use one or more neural networks to generate at least shading information to be applied to one or more first surfaces of one or more rendered three-dimensional (3D) objects based, at least in part, on pose information of the one or more first surfaces of one or more rendered 3D objects and a plurality of second surfaces generated by the one or more neural networks, each of the one or more neural networks corresponding to a specific joint of a plurality of joints, wherein a neural network, of the one or more neural networks, corresponding to a specific joint involved in a motion of the objects, generate each of the plurality of second surfaces using at least the neural network, and wherein the one or more neural networks aggregates the plurality of second surfaces to generate the one or more first surfaces.
64 . The machine-readable medium of claim 63 , wherein the instructions, if performed by the one or more processors, cause the shading information to be generated based, at least in part, on a signed distance field.
65 . The machine-readable medium of claim 63 , wherein the instructions, if performed by the one or more processors, cause the shading information to be generated based, at least in part, on the pose information of the one or more first surfaces of one or more rendered 3D objects.
66 . The machine-readable medium of claim 63 , wherein the instructions, if performed by the one or more processors, cause the shading information to be generated based, at least in part, on one or more randomly generated points.
67 . The machine-readable medium of claim 63 , wherein the instructions, if performed by the one or more processors, cause the shading information to be generated based, at least in part, on one or more points located within a bounding volume of the one or more objects.
68 . The machine-readable medium of claim 63 , wherein the one or more neural networks include at least one signed distance field neural network.
69 . The machine-readable medium of claim 63 , wherein the one or more neural networks include at least one aggregation neural network.
70 . The machine-readable medium of claim 63 , wherein the instructions, if performed by the one or more processors, cause the shading information to be generated based, at least in part, on one or more loss functions of the one or more neural networks.
71 . The machine-readable medium of claim 63 , wherein the instructions, if performed by the one or more processors, cause the shading information to be generated based, at least in part, on a subject code associated with the one or more objects.