Computer vision systems and methods for modeling three-dimensional structures using two-dimensional segments detected in digital aerial images

1. A system for modeling a three-dimensional structure utilizing two-dimensional segments comprising:

a memory; and

a processor in communication with the memory, the processor:

extracting a plurality of two-dimensional segments corresponding to the three-dimensional structure from a plurality of images indicative of different views of the three-dimensional structure;

determining a plurality of three-dimensional candidate segments based on the extracted plurality of two-dimensional segments;

adding the plurality of three-dimensional candidate segments to a three-dimensional segment cloud;

removing outlier segments from the three-dimensional segment cloud that are located outside of a three-dimensional bounding box that is based on an expected building footprint and a maximum height; and

transforming the three-dimensional segment cloud into a wireframe indicative of the three-dimensional structure by performing a wireframe extraction process on the three-dimensional segment cloud, the wireframe extraction process identifying clusters of three-dimensional candidate segments from the plurality of three-dimensional candidate segments in the three-dimensional cloud, processing each cluster into a three-dimensional line segment corresponding to a respective cluster, and optimizing an orientation of each three-dimensional line segment corresponding to the respective cluster.

2. The system of claim 1 , wherein the processor:

captures the plurality of images from different camera viewpoints;

determines a projection plane, camera parameter sets, and image parameters associated with each image of the plurality of images; and

identifies, based on the projection plane, the camera parameter sets, and the image parameters, two-dimensional segments sets in each image of the plurality of images corresponding to edges of the three-dimensional structure.

3. The system of claim 1 , wherein the processor determines the plurality of three-dimensional candidate segments based on the plurality of two-dimensional segments by determining a plurality of ground three-dimensional segments, determining a plurality of horizontal three-dimensional segments, and determining a plurality of oblique three-dimensional segments.

4. The system of claim 3 , wherein the processor determines the plurality of ground three-dimensional segments by:

pairing each extracted two-dimensional segment with proximate extracted two-dimensional segments;

selecting parallel two-dimensional segment pairs;

determining a three-dimensional segment from each selected parallel two-dimensional segment pair; and

selecting ground three-dimensional segments from the determined three-dimensional segments.

5. The system of claim 3 , wherein the processor determines the plurality of horizontal three-dimensional segments by:

pairing each extracted two-dimensional segment with parallel extracted two-dimensional segments;

determining a three-dimensional segment from each two-dimensional segment pair;

compiling the determined three-dimensional segments, based on characteristics thereof, into a plurality of clusters; and

selecting at least one cluster from the plurality of clusters and determining a horizontal three-dimensional segment indicative of the selected cluster.

6. The system of claim 3 , wherein the processor determines the plurality of oblique three-dimensional segments by:

pairing each extracted two-dimensional segment with non-parallel extracted two-dimensional segments;

determining a three-dimensional segment from each two-dimensional segment pair;

compiling the determined three-dimensional segments, based on characteristics thereof, into a plurality of clusters; and

selecting at least one cluster from the plurality of clusters and determining an oblique three-dimensional segment indicative of the selected cluster.

7. The system of claim 1 , wherein the processor determines the plurality of three-dimensional segments based on the extracted plurality of two-dimensional segments by determining a plurality of epipolar three-dimensional segments, the processor:

pairing extracted two-dimensional segments in other views with compatible epipolar lines, the epipolar lines reducing comparisons between extracted two-dimensional segments;

determining a three-dimensional segment from each two-dimensional segment pair;

selecting three-dimensional segment pairs having a consensus above a predetermined threshold value; and

excluding outlier three-dimensional segment pairs.

8. A method for modeling a three-dimensional structure utilizing two-dimensional segments comprising the steps of:

extracting a plurality of two-dimensional segments corresponding to the three-dimensional structure from a plurality of images indicative of different views of the three-dimensional structure;

determining a plurality of three-dimensional candidate segments based on the extracted plurality of two-dimensional segments;

adding the plurality of three-dimensional candidate segments to a three-dimensional segment cloud;

removing outlier segments from the three-dimensional segment cloud that are located outside of a three-dimensional bounding box that is based on an expected building footprint and a maximum height; and

transforming the three-dimensional segment cloud into a wireframe indicative of the three-dimensional structure by performing a wireframe extraction process on the three-dimensional segment cloud, the wireframe extraction process identifying clusters of three-dimensional candidate segments in the three-dimensional cloud, processing each cluster into a three-dimensional line segment corresponding to respective cluster, and optimizing an orientation of each three-dimensional line segment corresponding to the respective cluster.

9. The method of claim 8 , further comprising:

captures the plurality of images from different camera viewpoints;

determining a projection plane, camera parameter sets, and image parameters associated with each image of the plurality of images; and

identifying, based on the projection plane, the camera parameter sets, and the image parameters, the two-dimensional segments sets in each image of the plurality of images corresponding to edges of the three-dimensional structure.

10. The method of claim 8 , further comprising determining a plurality of ground three-dimensional segments, determining a plurality of horizontal three-dimensional segments, and determining a plurality of oblique three-dimensional segments.

11. The method of claim 10 , further comprising determining the plurality of ground three-dimensional segments by:

pairing each extracted two-dimensional segment with proximate extracted two-dimensional segments;

selecting parallel two-dimensional segment pairs;

determining a three-dimensional segment from each selected parallel two-dimensional segment pair; and

selecting ground three-dimensional segments from among the determined three-dimensional segments.

12. The method of claim 10 , further comprising determining the plurality of horizontal three-dimensional segments by:

pairing each extracted two-dimensional segment with parallel extracted two-dimensional segments;

determining a three-dimensional segment from each two-dimensional segment pair;

compiling the determined three-dimensional segments, based on characteristics thereof, into a plurality of clusters; and

selecting at least one cluster from the plurality of clusters and determining a horizontal three-dimensional segment indicative of the selected cluster.

13. The method of claim 10 , further comprising determining the plurality of oblique three-dimensional segments by:

pairing each extracted two-dimensional segment with non-parallel extracted two-dimensional segments;

determining a three-dimensional segment from each two-dimensional segment pair;

compiling the determined three-dimensional segments, based on characteristics thereof, into a plurality of clusters; and

selecting at least one cluster from the plurality of clusters and determining an oblique three-dimensional segment indicative of the selected cluster.

14. The method of claim 8 , further comprising determining a plurality of epipolar three-dimensional segments by:

pairing extracted two-dimensional segments in other views with compatible epipolar lines, the epipolar lines reducing comparisons between extracted two-dimensional segments;

determining a three-dimensional segment from each two-dimensional segment pair;

selecting three-dimensional segment pairs having a consensus above a predetermined threshold value; and

excluding outlier three-dimensional segment pairs.

15. A non-transitory computer readable medium having instructions stored thereon for modeling a three-dimensional structure utilizing two-dimensional segments which, when executed by a processor, causes the processor to carry out the steps of:

extracting a plurality of two-dimensional segments corresponding to the three-dimensional structure from a plurality of images indicative of different views of the three-dimensional structure;

determining a plurality of three-dimensional candidate segments based on the extracted plurality of two-dimensional segments;

adding the plurality of three-dimensional candidate segments to a three-dimensional segment cloud;

removing outlier segments from the three-dimensional segment cloud that are located outside of a three-dimensional bounding box that is based on an expected building footprint and a maximum height; and

transforming the three-dimensional segment cloud into a wireframe indicative of the three-dimensional structure by performing a wireframe extraction process on the three-dimensional segment cloud the wireframe extraction process identifying clusters of three-dimensional candidate segments in the three-dimensional cloud, processing each cluster into a three-dimensional line segment corresponding to respective cluster, and optimizing an orientation of each three-dimensional line segment corresponding to the respective cluster.

16. The non-transitory computer readable medium of claim 15 , the processor further carrying out the steps of:

capturing the plurality of images from different camera viewpoints;

determining a projection plane, camera parameter sets, and image parameters associated with each image of the plurality of images; and

identifying, based on the projection plane, the camera parameter sets and the image parameters, the two-dimensional segments sets in each image of the plurality of images corresponding to edges of the three-dimensional structure.

17. The non-transitory computer readable medium of claim 15 , the processor further carrying out the steps of determining a plurality of ground three-dimensional segments, determining a plurality of horizontal three-dimensional segments, and determining a plurality of oblique three-dimensional segments.

18. The non-transitory computer readable medium of claim 17 , the processor determining the plurality of ground three-dimensional segments by carrying out the steps of:

pairing each extracted two-dimensional segment with proximate extracted two-dimensional segments;

selecting parallel two-dimensional segment pairs;

determining a three-dimensional segment from each selected parallel two-dimensional segment pair; and

selecting ground three-dimensional segments from the determined three-dimensional segments.

19. The non-transitory computer readable medium of claim 17 , the processor determining the plurality of horizontal three-dimensional segments by carrying out the steps of:

pairing each extracted two-dimensional segment with parallel extracted two-dimensional segments;

determining a three-dimensional segment from each two-dimensional segment pair;

compiling the determined three-dimensional segments, based on characteristics thereof, into a plurality of clusters; and

selecting at least one cluster from the plurality of clusters and determining a horizontal three-dimensional segment indicative of the selected cluster.

20. The non-transitory computer readable medium of claim 17 , the processor determining the plurality of oblique three-dimensional segments by carrying out the steps of:

pairing each extracted two-dimensional segment with non-parallel extracted two-dimensional segments;

determining a three-dimensional segment from each two-dimensional segment pair;

compiling the determined three-dimensional segments, based on characteristics thereof, into a plurality of clusters; and

selecting at least one cluster from the plurality of clusters and determining an oblique three-dimensional segment indicative of the selected cluster.

21. The non-transitory computer readable medium of claim 15 , the processor determining a plurality of epipolar three-dimensional segments by carrying out the steps of:

pairing extracted two-dimensional segments in other views with compatible epipolar lines, the epipolar lines reducing comparisons between extracted two-dimensional segments;

determining a three-dimensional segment from each two-dimensional segment pair;

selecting three-dimensional segment pairs having a consensus above a predetermined threshold value; and

excluding outlier three-dimensional segment pairs.