Surface modeling systems and methods
View Patent ↗A method of generating a surface model of a physical environment includes obtaining an image of the physical environment. The method also includes generating a planar polygon mesh from at least the image. The method further includes extracting a boundary polygon of the planar polygon mesh. Moreover, the method includes generating a convex hull for the boundary polygon of the surface mesh. In addition, the method includes generating a minimal area oriented boundary polygon from the convex hull. The method may also include generating a maximal area oriented internal polygon inside of the boundary polygon of the planar polygon mesh.
1. A method of generating a surface model of a physical environment, the method being performed by an augmented reality (AR)/mixed reality (MR) system and comprising:
an image processor of the AR/MR system obtaining an image of the physical environment;
the image processor of the AR/MR system generating a planar polygon mesh from at least the image;
the image processor of the AR/MR system extracting a boundary polygon from the planar polygon mesh;
the image processor of the AR/MR system generating a convex hull for the boundary polygon; and
the image processor of the AR/MR system generating a minimal area oriented boundary polygon (MAOBP) from the convex hull, the minimal area oriented boundary polygon (MAOBP) comprising a polygon that contains the convex hull and the boundary polygon.
2. The method of claim 1 , wherein obtaining the image of the physical environment comprises: the image processor of the AR/MR system obtaining a 3-D point cloud corresponding to the physical environment using an imaging device; and obtaining pose information for the imaging device,
the method further comprising the image processor of the AR/MR system computing a truncated signed distance function for the 3-D point cloud using the pose information,
wherein generating the planar polygon mesh comprises the image processor of the AR/MR system tessellating the truncated signed distance function.
3. The method of claim 2 , further comprising the image processor of the AR/MR system combining two smaller planar polygon meshes into one larger planar polygon mesh.
4. The method of claim 1 , further comprising the image processor of the AR/MR system obtaining a gravity vector, wherein the generated planar polygon mesh is at least one of substantially parallel and orthogonal to the gravity vector.
5. The method of claim 1 , wherein generating the convex hull comprises the image processor of the AR/MR system using a Graham-Scan algorithm.
6. The method of claim 1 , wherein generating the minimal area oriented boundary polygon (MAOBP) comprises the image processor of the AR/MR system using a rotating calipers algorithm.
7. The method of claim 1 , wherein the minimal area oriented boundary polygon (MAOBP) contains the boundary polygon of the planar polygon mesh.
8. The method of claim 1 , further comprising the image processor of the AR/MR system generating a maximal area oriented internal polygon (MAOIP) that is entirely contained inside of the boundary polygon of the planar polygon mesh.
9. The method of claim 8 , wherein generating the maximal area oriented internal polygon (MAOIP) comprises the image processor of the AR/MR system performing a search in a search area defined by the boundary polygon.
10. The method of claim 9 , wherein generating the maximal area oriented internal polygon (MAOIP) comprises the image processor of the AR/MR system forming a grid in the search area.
11. The method of claim 10 , further comprising the image processor of the AR/MR system adjusting a resolution of the grid based on a size of the search area.
12. The method of claim 8 , further comprising the image processor of the AR/MR system receiving a selection of a point inside of the boundary polygon of the planar polygon mesh wherein generating the maximal area oriented internal polygon (MAOIP) comprises the image processor of the AR/MR system performing a search in a search area defined using the in selected point.
13. The method of claim 12 , wherein generating the maximal area oriented internal polygon (MAOIP) comprises the image processor of the AR/MR system forming a grid in the search area.
14. The method of claim 12 , further comprising the image processor of the AR/MR system adjusting a resolution of the grid based on a size of the search area.
15. The method of claim 8 , wherein the minimal area oriented boundary polygon (MAOBP) and the maximal area oriented internal polygon (MAOIP) have a same shape.
16. The method of claim 1 , wherein the planar polygon mesh is generated by the image processor of the AR/MR system based on a marching cubes algorithm.
17. The method of claim 1 , wherein the minimal area oriented boundary polygon (MAOBP) is at least one of a rectangle, a triangle, and a circle.
18. The method of claim 1 , further comprising the image processor of the AR/MR system determining a fit between the minimal area oriented boundary polygon (MAOBP) and the boundary polygon.
19. The method of claim 18 , wherein determining the fit comprises the image processor of the AR/MR system calculating a difference between a first area of the minimal area oriented boundary polygon (MAOBP) and a second area of the boundary polygon.
20. The method of claim 1 , further comprising the image processor of the AR/MR system storing data representing the minimal area oriented boundary polygon (MAOBP), wherein the minimal area oriented boundary polygon (MAOBP) is a rectangle.
21. The method of claim 20 , wherein the data comprises four sets of coordinates corresponding to the rectangle.
22. The method of claim 21 , wherein the data further comprises a length of the rectangle, a width of the rectangle, and a center of the rectangle.
23. The method of claim 21 , wherein each of the four sets of coordinates is a pair of coordinates.
24. The method of claim 1 , wherein generating the planar polygon mesh comprises the image processor of the AR/MR system capturing static portions of a series of images of the physical environment.
25. A system for generating a surface model of a physical environment, comprising:
an imaging device configured to obtain an image of the physical environment; and
an image processor operatively coupled to the imaging device, the image processor being configured to:
receive the image acquired by the imaging device;
generate a planar polygon mesh from at least the image;
extract a boundary polygon from the planar polygon mesh;
generate a convex hull for the boundary polygon; and
generate a minimal area oriented boundary polygon (MAOBP) from the convex hull, the MAOBP comprising a polygon that contains the convex hull and the boundary polygon.
26. The system of claim 25 , further comprising
a pose sensor operatively coupled to the imaging device and the image processor,
wherein the image processor is also configured to:
compute a truncated signed distance function for a 3-D point cloud using the pose information received from the pose sensor,
generate the planar polygon mesh by tessellating the truncated signed distance function, and
extract the boundary polygon from at least the image by extracting the boundary polygon of the planar polygon mesh,
wherein the imaging device is configured to obtain the image of the physical environment by:
obtaining a 3-D point cloud corresponding to the physical environment using the imaging device; and
obtaining pose information for the imaging device using the pose sensor.
27. The system of claim 25 , wherein the image processor is also configured to:
generate a maximal area oriented internal polygon (MAOIP) that is entirely contained inside of the boundary polygon of the planar polygon mesh.
28. The system of claim 27 , wherein the image processor is configured to generate the maximal area oriented internal polygon (MAOIP) by:
performing a search in a search area defined by the boundary polygon;
forming a grid in the search area; and
adjusting a resolution of the grid based on a size of the search area.
29. The system of claim 27 , wherein the image processor is also configured to:
receive a selection of a point inside of the boundary polygon of the planar polygon mesh, and
generate a maximal area oriented internal polygon (MAOIP) that is entirely contained inside of the boundary polygon of the planar polygon mesh by:
performing a search in a search area defined using the in selected point; and
forming a grid in the search area,
wherein the image processor is also configured to adjust a resolution of the grid based on a size of the search area.
30. The system of claim 25 , wherein the minimal area oriented boundary polygon (MAOBP) is at least one of a rectangle, a triangle, and a circle.
31. The system of claim 25 , wherein the image processor is also configured to store data representing the minimal area oriented boundary polygon (MAOBP),
wherein the minimal area oriented boundary polygon (MAOBP) is a rectangle, and
wherein the data comprises four sets of coordinates corresponding to the rectangle.
32. The system of claim 31 , wherein the data further comprises a length of the rectangle, a width of the rectangle, and a center of the rectangle.
33. The system of claim 31 , wherein each of the four sets of coordinates is a pair of coordinates.
34. The system of claim 25 , wherein the image processor is configured to generate the planar polygon mesh from at least the image by at least capturing static portions of a series of images of the physical environment.
35. The method of claim 1 , wherein the minimal area oriented boundary polygon (MAOBP) comprises a smallest polygon that contains the convex hull and the boundary polygon.
36. The method of claim 35 , wherein the minimal area oriented boundary polygon (MAOBP) comprises a smallest rectangle that contains the convex hull and the boundary polygon.
37. The method of claim 35 , further comprising the image processor of the AR/MR system
generating a maximal area oriented internal polygon (MAOIP) that is contained inside of the boundary polygon of the planar polygon mesh.
38. The method of claim 37 , wherein the minimal area oriented boundary polygon (MAOBP) overestimates an extent of a surface being modeled, and the maximal area oriented internal polygon (MAOIP) underestimates the extent of the surface being modeled.
39. The system of claim 25 , wherein the minimal area oriented boundary polygon (MAOBP) comprises a smallest polygon that contains the convex hull and the boundary polygon.
40. The system of claim 39 , wherein the minimal area oriented boundary polygon (MAOBP) comprises a smallest rectangle that contains the convex hull and the boundary polygon.
41. The system of claim 39 , wherein the image processor is configured
generate a maximal area oriented internal polygon (MAOIP) that is contained inside of the boundary polygon of the planar polygon mesh.
42. The method of claim 41 , wherein the minimal area oriented boundary polygon (MAOBP) overestimates an extent of a surface being modeled, and the maximal area oriented internal polygon (MAOIP) underestimates the extent of the surface being modeled.