Systems and methods for estimating feature locations based on image analysis and homography
Techniques are described for identifying correspondences between images to generate a fundamental matrix for the camera positions related to the images. The resultant fundamental matrix enables epipolar geometry to correlate common features among the images. Correspondences are identified by confirming feature matches across images by applying a homography to data representing features across images. Further techniques are described herein for generating a representation of a boundary of a feature of a structure based on a digital image. In one or more embodiments, generating a representation of a boundary of a particular feature in a digital image comprises determining a portion of the image that corresponds to the structure, and determining a portion of the image that corresponds to the particular feature. One more vanishing points are associated with the portion of the image corresponding to the particular feature. The one or more vanishing points are used to generate a set of bounding lines for the particular feature, based on which the boundary indicator for the feature is generated.
1 . A computer-implemented method comprising:
receiving a first image comprising data for a set of representations of co-planar rectangular features on a surface having a first orientation;
receiving a second image comprising data for the set of representations of co-planar rectangular features on the surface having a second orientation;
selecting, from the set of representations on the surface having the first orientation, a first representation, wherein the set of representations includes one or more first additional representations on the surface having the first orientation in addition to the first representation;
selecting, from the set of representations on the surface having the second orientation, a second representation, wherein the set of representations includes one or more second additional representations on the surface having the second orientation in addition to the second representation;
generating a homography aligning the first representation to the second representation;
estimating locations of the first additional representations on the surface having the second orientation based on the homography;
selecting an inlier pair formed from a first inlier and a second inlier based on the estimated locations, the inlier pair selected from the first additional representations and the second additional representations and the inlier pair being determined to match; and
generating an adjusted homography collectively aligning the first representation and the first inlier to the second representation and the second inlier.
2 . The method of claim 1 , further comprising generating a fundamental matrix between the first and second images based on the homography.
3 . The method of claim 2 , further comprising:
drawing an epipolar line in the second image based on the fundamental matrix with the first image; and
correlating at least one feature in the first image in the second image.
4 . The method of claim 3 , further comprising estimating at least one camera position based on the correlated features.
5 . The method of claim 4 , further comprising reconstructing a three dimensional model of the correlated features based on the estimated camera position.
6 . The method of claim 1 , further comprising validating the homography based on a comparison between the estimated locations of one or more of the first additional representations relative to the second additional representations, wherein the first additional representations and the second additional representations are associated with same co-planar rectangular features.
7 . The method of claim 6 , wherein validating further comprises detecting non-alignment for all estimated locations of the first additional representations and at least one of the second additional representations.
8 . The method of claim 6 , wherein validating further comprises detecting an error below a threshold for a position of the estimated locations of at least one of the first additional representations and at least one of the second additional representations.
9 . The method of claim 1 , further comprising generating a fundamental matrix between the first and second images based on the adjusted homography.
10 . The method of claim 6 , wherein validating further comprises detecting that a particular feature representation, of the first additional representations, is an inlier based on determining that the estimated location of the particular feature representation is within a threshold number of pixels from locations of at least one of the second additional representations.
11 . The method of claim 10 , further comprising:
determining that one or more other feature representations, other than the particular feature representation of the first additional representations, are mis-aligned; and
in response to determining that the one or more other feature representations of the first additional representations, are mis-aligned, adjusting the homography based on an estimated position of at least one of the one or more other feature representations.
12 . The method of claim 11 , further comprising generating a fundamental matrix between the first and second images based on the adjusted homography.
13 . A computer-readable storage medium comprising instructions which, when executed by a computer system, cause the computer system to:
receive a first image comprising data for a set of representations of co-planar rectangular features on a surface having a first orientation;
receive a second image comprising data for representations of the set of co-planar rectangular features on the surface having a second orientation;
select, from the set of representations on the surface having the first orientation, a first representation, wherein the set of representations includes one or more first additional representations on the surface having the first orientation in addition to the first representation;
select, from the set of representations on the surface having the second orientation, a second representation, wherein the set of representations includes one or more second additional representations on the surface having the second orientation in addition to the second representation;
generate a homography aligning the first representation to the second representation;
estimate locations of the first additional representations on the surface in the second orientation based on the homography;
select an inlier pair formed from a first inlier and a second inlier based on the estimated locations, the inlier pair selected from the first additional representations and the second additional representations and the inlier pair being determined to match; and
generate an adjusted homography collectively aligning the first representation and the first inlier to the second representation and the second inlier.
14 . The computer-readable storage medium of claim 13 , further comprising instructions that, when executed by the computer system, cause the computer system to generate a fundamental matrix between the first and second images based on the homography.
15 . The computer-readable storage medium of claim 14 , further comprising instructions that, when executed by the computer system, cause the computer system to:
draw an epipolar line in the second image based on the fundamental matrix with the first image; and
correlate at least one feature in the first image in the second image.
16 . The computer-readable storage medium of claim 15 , further comprising instructions that, when executed by the computer system, cause the computer system to estimate at least one camera position based on the correlated features.
17 . The computer-readable storage medium of claim 16 , further comprising instructions that, when executed by the computer system, cause the computer system to reconstruct a three dimensional model of the correlated features based on the estimated camera position.
18 . The computer-readable storage medium of claim 13 , further comprising instructions that, when executed by a computer system, cause the computer system to validate the homography based on a comparison between the estimated locations of one or more of the first additional representations relative to the second additional representations, wherein the first additional representations and the second additional representations are associated with same co-planar rectangular features.
19 . The computer-readable storage medium of claim 18 , wherein the instructions, when executed by the computer system, cause the computer system to validate the homography by detecting non-alignment for all estimated locations of the first additional representations and at least one of the second additional representations.
20 . The computer-readable storage medium of claim 18 , wherein the instructions, when executed by the computer system, cause the computer system to validate the homography by detecting an error below a threshold for a position of the estimated locations of at least one of the first additional representations and at least one of the second additional representations.
21 . The computer-readable storage medium of claim 13 , further comprising instructions that, when executed by the computer system, cause the computer system to generate a fundamental matrix between the first and second images based on the adjusted homography.
22 . The computer-readable storage medium of claim 18 , wherein the instructions, when executed by the computer system, cause the computer system to validate the homography by detecting that a particular feature representation, of the first additional representations, is an inlier based on determining that the estimated location of the particular feature representation is within a threshold number of pixels from locations of at least one of the second additional representations.
23 . The computer-readable storage medium of claim 22 , further comprising instructions that, when executed by the computer system, cause the computer system to:
determine that one or more other feature representations, other than the particular feature representation of the first additional representations, are mis-aligned; and
in response to determining that the one or more other feature representations of the first additional representations, are mis-aligned, adjust the homography based on an estimated position of at least one of the one or more other feature representations.
24 . The computer-readable storage medium of claim 23 , further comprising instructions that, when executed by the computer system, cause the computer system to generate a fundamental matrix between the first and second images based on the adjusted homography.
25 . The method of claim 1 , wherein the inlier pair was determined to match with less than a threshold error, wherein a remaining of the first additional representations was determined to match, based on the homography, with greater than the threshold error, and wherein the method further comprises:
estimating a location of the remaining of the first additional representations on the surface having the second orientation based on the adjusted homography; and
selecting a second inlier pair that includes the remaining of the first additional representations, wherein the remaining of the first additional representations was determined to match a remaining of the second additional representations with less than the threshold error,
and wherein the adjusted homography is further adjusted based on the second inlier pair.
26 . The computer-readable storage medium of claim 13 , wherein the inlier pair was determined to match with less than a threshold error, wherein a remaining of the first additional representations was determined to match, based on the homography, with greater than the threshold error, and wherein the computer-readable storage medium further comprises instructions that, when executed by the computer system, cause the computer system to:
estimate a location of the remaining of the first additional representations on the surface having the second orientation based on the adjusted homography; and
select a second inlier pair that includes the remaining of the first additional representations, wherein the remaining of the first additional representations was determined to match a remaining of the second additional representations with less than the threshold error,
and wherein the adjusted homography is further adjusted based on the second inlier pair.