THREE-DIMENSIONAL MODELING TOOLKIT

1 . A system comprising:

one or more hardware processors; and

memory storing instructions that, when executed by the one or more hardware processors, causes the system to perform operations comprising:

accessing image data depicting an object, wherein accessing the image data includes generating the image data via a camera of a mobile device of the user, and wherein the object is a portion of the user;

identifying a set of features from the image data;

assigning labels to regions of the image data based on the set of features, wherein assigning labels to the regions of the image data comprises providing the set of features identified from each respective region of the regions as input into a model, wherein the model outputs a label for the respective region, and wherein the labels comprise one or more semantic labels or classifications that correspond with the set of features; and

generating a three-dimensional (3D) model of the object based on the labels assigned to the regions of the image data.

2 . The system of claim 1 , wherein the one or more semantic labels or classifications indicate one or more regions within the image data to scan.

3 . The system of claim 1 , wherein accessing the image data includes:

activating the camera associated with the mobile device of the user; and

generating the image data via the camera of the mobile device, wherein the image data corresponds to a first data stream.

4 . The system of claim 3 , wherein generating the 3D model based on the labels includes:

generating the 3D model based on at least one of 2D-2D correspondences of the image data or 2D-3D correspondences of the image data, wherein the 2D-2D correspondences or the 2D-3D correspondences includes a plurality of distances of points in the image data depicting the object to the camera of the mobile device, wherein the 2D-2D correspondences or the 2D-3D correspondences corresponds to a second data stream.

5 . The system of claim 1 , wherein the 3D model is a reconstruction of the object, and wherein the reconstruction is performed using one or more of a polygon mesh model, a triangle mesh model, a non-uniform rational basis spline (NURBS) surface model, or a CAD model.

6 . The system of claim 1 , wherein the image data comprises a depiction of the object, and wherein the 3D model comprises a representation of surface features of the object depicted by the image data, and wherein the representation of surface features correspond to at least a portion of the user's head or face.

7 . The system of claim 6 , wherein the image data comprises a plurality of objects, and the operations further comprise:

filtering the plurality of objects of the user's head or face corresponding to a plurality of size values based on a position of the 3D model relative to a point cloud; and

presenting, via an interface, the filtered 3D model on the mobile device of the user.

8 . The system of claim 1 , wherein the 3D model includes unlabeled regions and labeled regions, and wherein the labeled regions correspond to a different color or pattern from the labeled regions.

9 . A method comprising:

accessing, by a processing circuit, image data depicting an object, wherein accessing the image data includes generating the image data via a camera of a mobile device of the user, and wherein the object is a portion of the user;

identifying, by the processing circuit, a set of features from the image data;

assigning, by the processing circuit, labels to regions of the image data based on the set of features, wherein assigning labels to the regions of the image data comprises providing the set of features identified from each respective region of the regions as input into a model, wherein the model outputs a label for the respective region, and wherein the labels comprise one or more semantic labels or classifications that correspond with the set of features; and

generating, by the processing circuit, a three-dimensional (3D) model of the object based on the labels assigned to the regions of the image data.

10 . The method of claim 9 , wherein the one or more semantic labels or classifications indicate one or more regions within the image data to scan.

11 . The method of claim 9 , wherein accessing the image data includes:

activating the camera associated with the mobile device of the user; and

generating the image data via the camera of the mobile device, wherein the image data corresponds to a first data stream.

12 . The method of claim 11 , wherein generating the 3D model based on the labels includes:

generating the 3D model based on at least one of 2D-2D correspondences of the image data or 2D-3D correspondences of the image data, wherein the 2D-2D correspondences or the 2D-3D correspondences includes a plurality of distances of points in the image data depicting the object to the camera of the mobile device, wherein the 2D-2D correspondences or the 2D-3D correspondences corresponds to a second data stream.

13 . The method of claim 9 , wherein the 3D model is a reconstruction of the object, and wherein the reconstruction is performed using one or more of a polygon mesh model, a triangle mesh model, a non-uniform rational basis spline (NURBS) surface model, or a CAD model.

14 . The method of claim 9 , wherein the image data comprises a depiction of the object, and wherein the 3D model comprises a representation of surface features of the object depicted by the image data, and wherein the representation of surface features correspond to at least a portion of the user's head or face.

15 . The method of claim 14 , wherein the image data comprises a plurality of objects, and the method further comprising:

filtering the plurality of objects of the user's head or face corresponding to a plurality of size values based on a position of the 3D model relative to a point cloud; and

presenting, via an interface, the filtered 3D model on the mobile device of the user.

16 . The method of claim 9 , wherein the 3D model includes unlabeled regions and labeled regions, and wherein the labeled regions correspond to a different color or pattern from the labeled regions.

17 . A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors of a computer system, cause the computer system to perform operations comprising:

accessing image data depicting an object, wherein accessing the image data includes generating the image data via a camera of a mobile device of the user, and wherein the object is a portion of the user;

identifying a set of features from the image data;

assigning labels to regions of the image data based on the set of features, wherein assigning labels to the regions of the image data comprises providing the set of features identified from each respective region of the regions as input into a model, wherein the model outputs a label for the respective region, and wherein the labels comprise one or more semantic labels or classifications that correspond with the set of features; and

generating a three-dimensional (3D) model of the object based on the labels assigned to the regions of the image data.

18 . The non-transitory computer-readable storage medium of claim 17 , wherein the one or more semantic labels or classifications indicate one or more regions within the image data to scan.

19 . The non-transitory computer-readable storage medium of claim 17 , wherein the operations further comprise:

activating the camera associated with the mobile device of the user;

generating the image data via the camera of the mobile device, wherein the image data corresponds to a first data stream; and

generating the 3D model based on at least one of 2D-2D correspondences of the image data or 2D-3D correspondences of the image data, wherein the 2D-2D correspondences or the 2D-3D correspondences includes a plurality of distances of points in the image data depicting the object to the camera of the mobile device, wherein the 2D-2D correspondences or the 2D-3D correspondences corresponds to a second data stream.

20 . The non-transitory computer-readable storage medium of claim 17 , wherein the image data comprises a depiction of the object, and wherein the 3D model comprises a representation of surface features of the object depicted by the image data, and wherein the representation of surface features correspond to at least a portion of the user's head or face.