Systems and methods for processing 2D/3D data for structures of interest in a scene and wireframes generated therefrom
The inventions herein relate generally to improvements in the generation of wireframe renderings derived from 2D and/or 3D data that includes at least one structure of interest in a scene. Such wireframe renderings and similar formats can be used in, among other things, 2D/3D CAD drawings, designs, drafts, models, building information models, augmented reality or virtual reality, and the like. Measurements, dimensions, geometric information, and semantic information generated according to the inventive methods can be accurate in relation to the actual structures. The wireframe renderings can be generated from a combination of a plurality of 2D images and point clouds, processing of point clouds to generate virtual/synthetic views to be used with the point clouds, or from 2D image data that has been processed in a machine learning process to generate 3D data. In some aspects, the wireframe renderings are accurate in relation to the actual structure of interest, automatically generated, or both.
1. A method of generating wireframe rendering of at least one structure of interest in a scene comprising:
a. providing, automatically by a computer, 2D and 3D data for at least one structure of interest in a scene;
b. processing, automatically by the computer, the 2D and 3D data to generate 3D information comprising an edge or skeletal representation associated with the at least one structure of interest, wherein the 2D and 3D data comprising the at least one structure of interest in the scene is generated by:
i. providing a plurality of 2D images including the at least one structure of interest and at least one point cloud including the at least one structure of interest, wherein the plurality of 2D images are associated with the at least one point cloud; or
ii. processing a plurality of point clouds to extract at least some 2D data associated with the at least one structure of interest, thereby providing point cloud data associated with 2D data derived from the plurality of point clouds; and
c. extracting, automatically by a computer, at least one geometric primitive from the 3D information comprising the edge or skeletal representation, each of the at least one geometric primitive being an object surface of the at least one structure of interest, wherein each extracted geometric primitive, independently, is selected from a list of geometric primitives in Table 1:
TABLE 1
No.
Type
Canonical Expression
1
One real plane
ax + by + cz + d = 0
2
Ellipsoid
x 2 /a + y 2 /b + z 2 /c = 1
3
Elliptic cylinder
x 2 /a + y 2 /b = 1
4
Hyperbolic cylinder
x 2 /a − y 2 /b = 1
5
Parabolic cylinder
x 2 + 2y = 0
6
Quadric cone
x 2 /a + y 2 /b − z 2 /c = 0
7
Hyperboloid of one sheet
x 2 /a + y 2 /b − z 2 /c = 1
8
Hyperboloid of two sheets
x 2 /a + y 2 /b − z 2 /c = −1
9
Hyperbolic paraboloid
x 2 /a − y 2 /b + 2z = 0
10
Elliptic paraboloid
x 2 /a − y 2 /b + 2z = 0
11
Intersecting planes
x 2 /a − y 2 /b = 0
12
Parallel planes
x 2 = 1
and
d. processing, automatically by the computer, output from step c to provide a wireframe rendering of at least part of the at least one structure of interest in the scene, the processing comprising:
determining edges of the object surfaces;
determining intersections and spatial relationships between intersecting object surfaces;
resolving edges of the object surfaces based at least in part upon the intersections, the spatial relationships, and the plurality of 2D images; and
combining the resolved object surfaces to generate the wireframe rendering.
2. The method of claim 1 , wherein the at least one structure of interest comprises at least one structural element that comprises a non-planar surface.
3. The method of claim 1 , wherein:
a. The at least one structure of interest has an actual measurement value, actual dimensions, and actual geometric information;
b. one or more of measurement information, dimensions, and geometric information for the at least one structure of interest is or are derived from the wireframe rendering; and
c. each of the derived measurement information, dimensions, and geometric information is, independently, within 5% or less of the corresponding actual measurement value, actual dimensions, and actual geometric information, thereby providing a numerically accurate wireframe rendering of at least part of the at least one structure of interest in the scene.
4. The method of claim 1 , wherein semantic information is generated for the at least one structure of interest.
5. The method of claim 1 , wherein the at least one structure of interest comprises a building exterior or a building interior.
6. The method of claim 1 , wherein the at least one structure of interest comprises a roof, and the provided wireframe rendering is provided as part of a roofing report.
7. The method of claim 1 , wherein at least two geometric primitives are extracted, and all or part of a first geometric primitive boundary is within a second geometric primitive boundary.
8. The method of claim 1 , further comprising incorporating the wireframe rendering into a machine learning training set.
9. A method of generating wireframe rendering of at least one structure of interest in a scene comprising:
a. providing 2D and 3D data for at least one structure of interest in a scene;
b. processing the 2D and 3D data to generate 3D information comprising an edge or skeletal representation associated with the at least one structure of interest, wherein the 2D and 3D data comprising the at least one structure of interest in the scene is generated by:
i. providing a plurality of 2D images including the at least one structure of interest and at least one point cloud including the at least one structure of interest, wherein the plurality of 2D images are associated with the at least one point cloud; or
ii. processing a plurality of point clouds to extract at least some 2D data associated with the at least one structure of interest, thereby providing point cloud data associated with 2D data derived from the plurality of point clouds; and
c. extracting a plurality of geometric primitives from the 3D information comprising the edge or skeletal representation, each of the plurality of geometric primitives being an object surface of the at least one structure of interest, wherein each extracted geometric primitive, independently, is selected from a list of geometric primitives in Table 1:
TABLE 1
No.
Type
Canonical Expression
1
One real plane
ax + by + cz + d = 0
2
Ellipsoid
x 2 /a + y 2 /b + z 2 /c = 1
3
Elliptic cylinder
x 2 /a + y 2 /b = 1
4
Hyperbolic cylinder
x 2 /a − y 2 /b = 1
5
Parabolic cylinder
x 2 + 2y = 0
6
Quadric cone
x 2 /a + y 2 /b − z 2 /c = 0
7
Hyperboloid of one sheet
x 2 /a + y 2 /b − z 2 /c = 1
8
Hyperboloid of two sheets
x 2 /a + y 2 /b − z 2 /c = −1
9
Hyperbolic paraboloid
x 2 /a − y 2 /b + 2z = 0
10
Elliptic paraboloid
x 2 /a − y 2 /b + 2z = 0
11
Intersecting planes
x 2 /a − y 2 /b = 0
12
Parallel planes
x 2 = 1
and
d. processing output from step c to provide a wireframe rendering of at least part of the at least one structure of interest in the scene, the processing comprising:
determining edges of the object surfaces;
determining intersections and spatial relationships between intersecting object surfaces;
resolving edges of the object surfaces based at least in part upon the intersections, the spatial relationships, and the plurality of 2D images; and
combining the resolved object surfaces to generate the wireframe rendering,
wherein:
i. the at least one structure of interest has an actual measurement value, actual dimensions, and actual geometric information;
ii. one or more of measurement information, dimensions, and geometric information for the at least one structure of interest is or are derived from the wireframe rendering; and
iii. each of the derived measurement information, dimensions, and geometric information is, independently, within 5% or less of the corresponding actual measurement value, actual dimensions, and actual geometric information, thereby providing a numerically accurate wireframe rendering of at least part of the at least one structure of interest in the scene.
10. The method of claim 9 , wherein each of the steps is performed by a computer, thereby providing an automatically generated wireframe rendering.
11. The method of claim 9 , wherein the at least one structure of interest comprises at least one structural element that comprises a non-planar surface.
12. The method of claim 9 , wherein at least two geometric primitives are extracted, and all or part of a first geometric primitive boundary is within a second geometric primitive boundary.
13. The method of claim 9 , wherein semantic information is generated for the at least one structure of interest.
14. The method of claim 9 , wherein the at least one structure of interest comprises a building exterior or a building interior.
15. The method of claim 9 , wherein the at least one structure of interest comprises a roof, and the provided wireframe rendering is provided as part of a roofing report.
16. The method of claim 9 , further comprising incorporating the wireframe rendering into a machine learning training set.