Wireframe generation via gaussian splatting

1 . A processing system comprising:

one or more memories comprising processor-executable instructions; and

one or more processors coupled to the one or more memories and configured to execute the processor-executable instructions and cause the processing system to:

access an image depicting an object;

generate an edge map comprising a plurality of edges based on the image;

modify a thickness of each of the plurality of edges in the edge map based on a current stage of a three-dimensional reconstruction operation;

generate a rendered image depicting a set of Gaussian distributions in a three-dimensional virtual space using Gaussian splatting;

modify one or more parameters of one or more of the set of Gaussian distributions based on comparing the rendered image and the edge map; and

after modifying the one or more parameters, generate a three-dimensional wireframe model of the object based on the set of Gaussian distributions.

2 . The processing system of claim 1 , wherein:

to modify the thickness of each of the plurality of edges based on the current stage of the three-dimensional reconstruction operation, the one or more processors are configured to execute the processor-executable instructions and cause the processing system to increase the thickness of each of the plurality of edges by a first amount; and

the first amount is less than an amount used to increase edge thickness during a prior stage of the three-dimensional reconstruction operation, relative to the current stage.

3 . The processing system of claim 1 , wherein, to modify the thickness of each of the plurality of edges, the one or more processors are configured to execute the processor-executable instructions and cause the processing system to:

generate a set of distance values for a set of pixels in the edge map using a distance transform; and

binarize the set of distance values based on a threshold value, wherein the threshold value controls the thickness of each of the plurality of edges.

4 . The processing system of claim 1 , wherein, to modify the thickness of each of the plurality of edges, the one or more processors are configured to execute the processor-executable instructions and cause the processing system to apply a Gaussian blur operation to the edge map.

5 . The processing system of claim 1 , wherein:

the one or more processors are configured to further execute the processor-executable instructions and cause the processing system to initialize the set of Gaussian distributions based on a set of initialization edge maps corresponding to the object; and

to initialize the set of Gaussian distributions, the one or more processors are configured to execute the processor-executable instructions and cause the processing system to:

back project a set of pixels corresponding to edges in the set of initialization edge maps into the three-dimensional virtual space; and

initialize the set of Gaussian distributions based on the back projected set of pixels.

6 . The processing system of claim 5 , wherein, to initialize the set of Gaussian distributions based on the back projected set of pixels, the one or more processors are configured to execute the processor-executable instructions and cause the processing system to:

identify a set of voxels in the three-dimensional virtual space that were intersected by the back projecting; and

randomly distribute the set of Gaussian distributions within the identified set of voxels.

7 . The processing system of claim 1 , wherein the one or more processors are configured to further execute the processor-executable instructions and cause the processing system to align the set of Gaussian distributions, wherein, to align the set of Gaussian distributions, the one or more processors are configured to execute the processor-executable instructions and cause the processing system to, for a first Gaussian distribution of the set of Gaussian distributions:

identify a set of nearest neighbors to the first Gaussian distribution;

generate a local covariance matrix for the first Gaussian distribution based on the set of nearest neighbors;

determine a principal axis of the set of nearest neighbors based on the local covariance matrix; and

move the first Gaussian distribution, in the three-dimensional virtual space, to the principal axis.

8 . The processing system of claim 1 , wherein, to generate the three-dimensional wireframe model, the one or more processors are configured to execute the processor-executable instructions and cause the processing system to determine a set of edge orientations for the three-dimensional wireframe model based on the set of Gaussian distributions.

9 . The processing system of claim 8 , wherein, to determine the set of edge orientations, the one or more processors are configured to execute the processor-executable instructions and cause the processing system to, for a first Gaussian distribution of the set of Gaussian distributions:

identify a set of nearest neighbors to the first Gaussian distribution;

generate a local covariance matrix for the first Gaussian distribution based on the set of nearest neighbors; and

determine a principal axis of the set of nearest neighbors based on the local covariance matrix, wherein at least one of the set of edge orientations corresponds to the principal axis.

10 . The processing system of claim 8 , wherein:

to determine the set of edge orientations, the one or more processors are configured to execute the processor-executable instructions and cause the processing system to, for a first Gaussian distribution of the set of Gaussian distributions, constrain a scale of the first Gaussian distribution during the three-dimensional reconstruction operation to be elongated along a primary axis; and

at least one of the set of edge orientations corresponds to the primary axis.

11 . A processor-implemented method for three-dimensional reconstruction using machine learning, comprising:

accessing an image depicting an object;

generating an edge map comprising a plurality of edges based on the image;

modifying a thickness of each of the plurality of edges in the edge map based on a current stage of the three-dimensional reconstruction;

generating a rendered image depicting a set of Gaussian distributions in a three-dimensional virtual space using Gaussian splatting;

modifying one or more parameters of one or more of the set of Gaussian distributions based on comparing the rendered image and the edge map; and

after modifying the one or more parameters, generating a three-dimensional wireframe model of the object based on the set of Gaussian distributions.

12 . The method of claim 11 , wherein modifying the thickness of each of the plurality of edges based on the current stage of the three-dimensional reconstruction comprises increasing the thickness of each of the plurality of edges by a first amount, wherein the first amount is less than an amount used to increase edge thickness during a prior stage of the three-dimensional reconstruction, relative to the current stage.

13 . The method of claim 11 , wherein modifying the thickness of each of the plurality of edges comprises:

generating a set of distance values for a set of pixels in the edge map using a distance transform; and

binarizing the set of distance values based on a threshold value, wherein the threshold value controls the thickness of each of the plurality of edges.

14 . The method of claim 11 , wherein modifying the thickness of each of the plurality of edges comprises applying a Gaussian blur operation to the edge map.

15 . The method of claim 11 , further comprising initializing the set of Gaussian distributions based on a set of initialization edge maps corresponding to the object, comprising:

back projecting a set of pixels corresponding to edges in the set of initialization edge maps into the three-dimensional virtual space; and

initializing the set of Gaussian distributions based on the back projected set of pixels.

16 . The method of claim 15 , wherein initializing the set of Gaussian distributions based on the back projected set of pixels comprises:

identifying a set of voxels in the three-dimensional virtual space that were intersected by the back projecting; and

randomly distributing the set of Gaussian distributions within the identified set of voxels.

17 . The method of claim 11 , further comprising aligning the set of Gaussian distributions comprising, for a first Gaussian distribution of the set of Gaussian distributions:

identifying a set of nearest neighbors to the first Gaussian distribution;

generating a local covariance matrix for the first Gaussian distribution based on the set of nearest neighbors;

determining a principal axis of the set of nearest neighbors based on the local covariance matrix; and

moving the first Gaussian distribution, in the three-dimensional virtual space, to the principal axis.

18 . The method of claim 11 , wherein generating the three-dimensional wireframe model comprises determining a set of edge orientations for the three-dimensional wireframe model based on the set of Gaussian distributions.

19 . The method of claim 18 , wherein determining the set of edge orientations comprises, for a first Gaussian distribution of the set of Gaussian distributions, constraining a scale of the first Gaussian distribution during the three-dimensional reconstruction to be elongated along a primary axis, wherein at least one of the set of edge orientations corresponds to the primary axis.

20 . An apparatus comprising:

means for accessing an image depicting an object;

means for generating an edge map comprising a plurality of edges based on the image;

means for modifying a thickness of each of the plurality of edges in the edge map based on a current stage of a three-dimensional reconstruction operation;

means for generating a rendered image depicting a set of Gaussian distributions in a three-dimensional virtual space using Gaussian splatting;

means for modifying one or more parameters of one or more of the set of Gaussian distributions based on comparing the rendered image and the edge map; and

means for generating, after modifying the one or more parameters, a three-dimensional wireframe model of the object based on the set of Gaussian distributions.