Systems and methods for retexturizing mesh-based objects using point clouds
Disclosed is a graphics system and associated methodologies for selectively increasing the level-of-detail at specific parts of a mesh model based on a point cloud that provides a higher detailed representation of the same or similar three-dimensional (“3D”) object. The graphics system receives the mesh model and the point cloud of the 3D object. The graphics system determines a region-of-interest of the 3D object based in part on differences amongst points that represent part or all of the region-of-interest. The graphics system reconstructs the region-of-interest in the mesh model and generates a modified mesh model by modifying a first set of meshes representing the region-of-interest in the mesh model to a second set of meshes based on the positional elements of the point cloud points. The second set of meshes has more meshes and represents the region-of-interest at a higher level-of-detail than the first set of meshes.
1. A method comprising:
selecting a region-of-interest in a mesh model comprising a first plurality of polygons that are connected to one another to represent the region-of-interest with a first amount of visual data;
comparing the first amount of visual data against a second amount of visual data contained in a representation of the region-of-interest formed by a plurality of disconnected points from a point cloud;
determining that the first amount of visual data differs from the second amount of visual data by more than a threshold amount; and
replacing the first plurality of polygons in the region-of-interest with a second plurality of polygons that represent the region-of-interest in the mesh model with a third amount of visual data that is greater than the first amount of visual data.
2. The method of claim 1 , wherein determining that the first amount of visual data differs from the second amount of visual data by more than the threshold amount comprises:
detecting fewer angles or features formed by the first plurality of polygons than formed by the plurality of disconnected points.
3. The method of claim 1 , wherein determining that the first amount of visual data differs from the second amount of visual data by more than the threshold amount comprises:
computing positional differences between the first plurality of polygons and the plurality of disconnected points; and
determining that the positional differences are greater than the threshold amount.
4. The method of claim 1 , wherein comparing the first amount of visual data against the second amount of visual data comprises:
comparing a first shape formed by the first plurality of polygons against a second shape formed by the plurality of disconnected points; and
determining that the first shape comprises fewer details, angles, or features than the second shape.
5. The method of claim 1 further comprising:
generating the second plurality of polygons to have greater positional variation than the first plurality of polygons.
6. The method of claim 5 , wherein generating the second plurality of polygons comprises:
defining the second plurality of polygons to have more polygons that are smaller in size than the first plurality of polygons.
7. The method of claim 5 , wherein generating the second plurality of polygons comprises:
defining the second plurality of polygons with greater angular variation than the first plurality of polygons.
8. The method of claim 5 , wherein generating the second plurality of polygons comprises:
defining the second plurality of polygons to cross more planes in a three-dimensional space than the first plurality of polygons.
9. The method of claim 5 , wherein generating the second plurality of polygons comprises:
sampling a set of points from the plurality of disconnected points based on positional variation amongst the set of points; and
defining vertices of the second plurality of polygons based on positions of the set of points.
10. The method of claim 5 , wherein generating the second plurality of polygons comprises:
sampling a set of points from the plurality of disconnected points that correspond to different features in the region-of-interest; and
defining vertices of the second plurality of polygons using the set of points.
11. The method of claim 5 , wherein generating the second plurality of polygons comprises:
sampling a set of points from the plurality of disconnected points with a greater difference in their respective positions than other set of points from the plurality of disconnected points; and
defining vertices of the second plurality of polygons based on positions of the set of points.
12. The method of claim 1 further comprising:
presenting a visualization of the region-of-interest by rendering the second plurality of polygons using software and hardware that are optimized for polygons of the mesh model.
13. The method of claim 1 , wherein selecting the region-of-interest comprises:
selecting the region-of-interest based on a density or a variety associated with the plurality of disconnected points representing the region-of-interest in the point cloud.
14. The method of claim 1 , wherein the first plurality of polygons are generated from a first sampling of the plurality of disconnected points, and wherein the second plurality of polygons are generated from a second sampling of the plurality of disconnected points that includes more points than the first sampling of the plurality of disconnected points.
15. The method of claim 1 , wherein the first amount of visual data and the second amount of visual data correspond to different numbers of angles, shapes, forms, or features that represented in the region-of-interest.
16. A graphics system comprising:
one or more hardware processors configured to:
select a region-of-interest in a mesh model comprising a first plurality of polygons that are connected to one another to represent the region-of-interest with a first amount of visual data;
compare the first amount of visual data against a second amount of visual data contained in a representation of the region-of-interest formed by a plurality of disconnected points from a point cloud;
determine that the first amount of visual data differs from the second amount of visual data by more than a threshold amount; and
replace the first plurality of polygons in the region-of-interest with a second plurality of polygons that represent the region-of-interest in the mesh model with a third amount of visual data that is greater than the first amount of visual data.
17. The graphics system of claim 16 , wherein determining that the first amount of visual data differs from the second amount of visual data by more than the threshold amount comprises:
detecting fewer angles or features formed by the first plurality of polygons than formed by the plurality of disconnected points.
18. The graphics system of claim 16 , wherein determining that the first amount of visual data differs from the second amount of visual data by more than the threshold amount comprises:
computing positional differences between the first plurality of polygons and the plurality of disconnected points; and
determining that the positional differences are greater than the threshold amount.
19. The graphics system of claim 16 , wherein comparing the first amount of visual data against the second amount of visual data comprises:
comparing a first shape formed by the first plurality of polygons against a second shape formed by the plurality of disconnected points; and
determining that the first shape comprises fewer details, angles, or features than the second shape.
20. A non-transitory computer-readable medium storing program instructions that, when executed by one or more hardware processors of a graphics system, cause the graphics system to perform operations comprising:
selecting a region-of-interest in a mesh model comprising a first plurality of polygons that are connected to one another to represent the region-of-interest with a first amount of visual data;
comparing the first amount of visual data against a second amount of visual data contained in a representation of the region-of-interest formed by a plurality of disconnected points from a point cloud;
determining that the first amount of visual data differs from the second amount of visual data by more than a threshold amount; and
replacing the first plurality of polygons in the region-of-interest with a second plurality of polygons that represent the region-of-interest in the mesh model with a third amount of visual data that is greater than the first amount of visual data.