Methods, storage media, and systems for selecting a pair of consistent real-world camera poses
Disclosed are methods, storage media, and systems for selecting a pair of consistent real-world camera poses for 3D reconstruction. The disclosed processes involve capturing multiple images of an object from various camera poses and analyzing the images and camera poses to select a consistent pair of camera poses. This selection is based on calculating perturbation errors and reprojection errors generated from 2D points or 2D line segments in the images. A weight is calculated for each camera pose pair based on these errors, and the pair with the largest weight, indicative of the highest consistency and stability, is selected.
1 . A method of selecting a pair of consistent real-world camera poses for three-dimensional reconstruction, the method comprising:
receiving a plurality of images and an associated plurality of camera poses;
for each image of the plurality of images, receiving a two-dimensional (2D) line segment, wherein the 2D line segments of the plurality of images correspond to one another;
for each pair of images of the plurality of images:
calculating a perturbation error, wherein the perturbation error quantifies a degree of a three-dimensional (3D) point distribution based on one or more offset pixels proximate to each of a first point of the 2D line segment in a first image of the pair of images and a corresponding second point of the 2D line segment in a second image of the pair of images;
generating a 3D line segment based on the 2D line segment in the first image of the pair of images and the 2D line segment in the second image of the pair of images;
calculating reprojection errors of the 3D line segment into each of the plurality of images; and
calculating a weight based on the perturbation error and the reprojection errors; and
selecting a pair of camera poses of pairs of camera poses associated with a largest weight.
2 . The method of claim 1 , wherein calculating the weight comprises calculating a product of an inverse of the perturbation error and an inverse of a sum of the reprojection errors.
3 . The method of claim 1 , further comprising calculating a weighted average measurement for a dimensional value of each 3D line segment.
4 . The method of claim 1 , wherein calculating the reprojection errors of the 3D line segment comprises, calculating a reprojection error for each image of the plurality of images based on a difference between a reprojection of a first end point of the 3D line segment and a corresponding first end point of the 2D line segment in the image and a difference between a reprojection of a second end point of the 3D line segment and a corresponding second end point of the 2D line segment in the image.
5 . The method of claim 1 , wherein calculating the weight comprises calculating a product of an inverse of the perturbation error, an inverse of a sum of the reprojection errors, and a distance between a first camera pose associated with the first image and a second camera pose associated with the second image.
6 . The method of claim 1 , further comprising generating a dimensional value of a generated 3D line segment generated based on a 2D line segment in a first image associated with a first camera pose of the selected pair of camera poses and a 2D line segment in a second image associated with a second camera pose of the selected pair of camera poses.
7 . The method of claim 6 , wherein the dimensional value is derived from an augmented reality dimensional value.
8 . The method of claim 6 , further comprising scaling a 3D building model based on selected pair of camera poses and according to the generated dimensional value.
9 . The method of claim 6 , wherein generating a dimensional value of the generated 3D line segment comprises selecting a 3D line segment associated with a greatest weight.
10 . One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform a method of selecting a pair of consistent real-world camera poses for three-dimensional reconstruction, the method comprising:
receiving a plurality of images and an associated plurality of camera poses;
for each image of the plurality of images, receiving a two-dimensional (2D) line segment, wherein the 2D line segments of the plurality of images correspond to one another;
for each pair of images of the plurality of images:
calculating a perturbation error, wherein the perturbation error quantifies a degree of a three-dimensional (3D) point distribution based on one or more offset pixels proximate to each of a first point of the 2D line segment in a first image of the pair of images and a corresponding second point of the 2D line segment in a second image of the pair of images;
generating a 3D line segment based on the 2D line segment in the first image of the pair of images and the 2D line segment in the second image of the pair of images;
calculating reprojection errors of the 3D line segment into each of the plurality of images; and
calculating a weight based on the perturbation error and the reprojection errors; and
selecting a pair of camera poses of pairs of camera poses associated with a largest weight.
11 . The media of claim 10 , wherein calculating the weight comprises calculating a product of an inverse of the perturbation error and an inverse of a sum of the reprojection errors.
12 . The media of claim 10 , further comprising instructions for calculating a weighted average measurement for a dimensional value of each 3D line segment.
13 . The media of claim 10 , wherein calculating the reprojection errors of the 3D line segment comprises instructions for, calculating a reprojection error for each image of the plurality of images based on a difference between a reprojection of a first end point of the 3D line segment and a corresponding first end point of the 2D line segment in the image and a difference between a reprojection of a second end point of the 3D line segment and a corresponding second end point of the 2D line segment in the image.
14 . The media of claim 10 , wherein calculating the weight comprises instructions for calculating a product of an inverse of the perturbation error, an inverse of a sum of the reprojection errors, and a distance between a first camera pose associated with the first image and a second camera pose associated with the second image.
15 . The media of claim 10 , further comprising instructions for generating a dimensional value of a generated 3D line segment based on a 2D line segment in a first image associated with a first camera pose of the selected pair of camera poses and a 2D line segment in a second image associated with a second camera pose of the selected pair of camera poses.
16 . The media of claim 15 , wherein the dimensional value is derived from an augmented reality dimensional value.
17 . The media of claim 15 , further comprising instructions for scaling a 3D building model based on the selected pair of camera poses and according to the generated dimensional value.
18 . The media of claim 15 , wherein generating a dimensional value of the generated 3D line segment comprises instructions for selecting a 3D line segment associated with a greatest weight.