Convex representation of objects using neural network
Methods, systems, and apparatus including computer programs encoded on a computer storage medium, for generating convex decomposition of objects using neural network models. One of the methods includes receiving an input that depicts an object. The input is processed using a neural network to generate an output that defines a convex representation of the object. The output includes, for each of a plurality of convex elements, respective parameters that define a position of the convex element in the convex representation of the object.
1. A computer-implemented method, comprising:
receiving an input that depicts an object; and
generating a convex representation of the object, comprising processing the input using a neural network comprising an output layer that generates parameters that define a predetermined number of convex elements, wherein the parameters comprise respective parameters for each of the convex elements define a position of the convex element in the convex representation of the object, wherein each of the convex elements is defined by a predetermined number of halfspaces, and wherein the respective parameters comprise, for each of the halfspaces, parameters that define the halfspace.
2. The computer-implemented method of claim 1 , wherein the respective parameters define, for each of the halfspaces, a signed distance from a point to the halfspace defined with a normal and an offset.
3. The computer-implemented method of claim 1 , wherein the neural network comprises:
an encoder neural network that is configured to receive the input and to generate a low-dimensional latent representation of the input, and
a decoder neural network that is configured to generate the parameters.
4. The computer-implemented method of claim 1 , wherein for each of the convex elements, the respective parameters comprise:
parameters that define an indicator function for the convex element.
5. The computer-implemented method of claim 1 , wherein the respective parameters comprise, for each of the predetermined number of convex elements, pose parameters that define an affine transformation that transforms a point from world coordinates to local coordinates of the convex element.
6. The computer-implemented method of claim 1 , wherein the input is one of an image of the object, a point cloud of the object, or a voxel grid of the object.
7. The computer-implemented method of claim 1 , further comprising:
using the convex representation of the object in applications where an explicit representation of a surface is required.
8. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving an input that depicts an object; and
generating a convex representation of the object, comprising processing the input using a neural network comprising an output layer that generates parameters that define a predetermined number of convex elements, wherein the parameters comprise respective parameters for each of the convex elements define a position of the convex element in the convex representation of the object, wherein each of the convex elements is defined by a predetermined number of halfspaces, and wherein the respective parameters comprise, for each of the halfspaces, parameters that define the halfspace.
9. The system of claim 8 , wherein the respective parameters define, for each of the halfspaces, a signed distance from a point to the halfspace defined with a normal and an offset.
10. The system of claim 8 , wherein the neural network comprises:
an encoder neural network that is configured to receive the input and to generate a low-dimensional latent representation of the input, and
a decoder neural network that is configured to generate the parameters.
11. The system of claim 8 , wherein for each of the convex elements, the respective parameters comprise:
parameters that define an indicator function for the convex element.
12. The system of claim 8 , wherein the respective parameters comprise, for each of the predetermined number of convex elements, pose parameters that define an affine transformation that transforms a point from world coordinates to local coordinates of the convex element.
13. The system of claim 8 , wherein the input is one of an image of the object, a point cloud of the object, or a voxel grid of the object.
14. The system of claim 8 , the operations comprise:
using the convex representation of the object in applications where an explicit representation of a surface is required.
15. A computer program product, encoded on one or more non-transitory computer storage media, comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
receiving an input that depicts an object; and
generating a convex representation of the object, comprising processing the input using a neural network comprising an output layer that generates parameters that define a predetermined number of convex elements, wherein the parameters comprise respective parameters for each of the convex elements define a position of the convex element in the convex representation of the object, wherein each of the convex elements is defined by a predetermined number of halfspaces, and wherein the respective parameters comprise, for each of the halfspaces, parameters that define the halfspace.
16. The computer program product of claim 15 , wherein the respective parameters define, for each of the halfspaces, a signed distance from a point to the halfspace defined with a normal and an offset.
17. The computer program product of claim 15 , wherein the neural network comprises:
an encoder neural network that is configured to receive the input and to generate a low-dimensional latent representation of the input, and
a decoder neural network that is configured to generate the parameters.
18. The computer program product of claim 15 , wherein for each of the convex elements, the respective parameters comprise:
parameters that define an indicator function for the convex element.
19. The computer program product of claim 15 , wherein the respective parameters comprise, for each of the predetermined number of convex elements, pose parameters that define an affine transformation that transforms a point from world coordinates to local coordinates of the convex element.
20. The computer program product of claim 15 , wherein the input is one of an image of the object, a point cloud of the object, or a voxel grid of the object.