Plane panorama location correction in three-dimensional mapping
Correction of the locations of panoramas and associated physical locations in a region are disclosed. The task of correcting the locations of the panoramas is divided into XY location correction followed by Z location correction. The panorama set is filtered to remove extraneous planes. In XY-plane correction, neighboring panoramas are identified using a neighbor graph, and the locations of sets of neighboring planes are constrained to be within a specified distance of one another, and a system of linear equations is constructed to represent the locations of the panoramas. A weighted absolute sum of the constraints is minimized across the entire system using a convex quadratic program solver to derive new locations for the planes. The location of ground planes is similarly corrected using a minimization of errors technique.
1. A method for correcting locations of panoramas and physical locations in a universal transverse mercator region, comprising:
identifying, using a processor, neighboring panoramas in an XY coordinate system, each of the neighboring panoramas having a number of planes defining structures and surfaces in the panorama;
identifying, using the processor, a subset of the number of planes, each of the subset having a horizontal normal within a threshold;
constructing, using the processor, a system of linear equations to represent the locations of the panoramas in the XY coordinate system, in which pairs of linear equations representing pairs of neighboring panoramas are required to be within specified threshold distances of one another, plus or minus an error term;
minimizing, using the processor, a weighted absolute sum of the error terms across the system of linear equations using a semidefinite quadratic program solver to define new XY locations for the neighboring panoramas; and
storing the new XY locations to the neighboring panoramas in a three dimensional map.
2. The method of claim 1 , further comprising:
identifying ground planes in the neighboring panoramas;
constructing a system of linear inequalities to represent the locations of ground planes in the neighboring panoramas along a Z axis of the XY coordinate system, in which pairs of linear equations representing the ground planes of the neighboring panoramas are required to be within specified threshold distances of one another, plus or minus a ground plane error term;
minimizing a weighted absolute sum of the ground plane error terms across the system of linear inequalities using a linear program solver to define new Z locations for the neighboring panoramas; and
applying the new Z locations to the neighboring panoramas.
3. The method of claim 2 , further comprising filtering and discarding planes in the panoramas that do not meet defined thresholds.
4. The method of claim 2 , wherein identifying the neighboring panoramas comprises constructing a neighbor graph.
5. A method comprising:
comparing, using a processor, separation distances between locations of panoramic images to a threshold;
filtering, using the processor, the panoramic images to remove panoramic images below a predetermined size;
identifying, using the processor, a first pair of neighboring panoramic images separated by less than the threshold;
identifying, using the processor, a second pair of neighboring panoramic images separated by less than the threshold;
constructing, using the processor, a first equation for the first pair of neighboring panoramic images and a second equation for the second pair of neighboring panoramic images;
calculating, using the processor and from at least the first equation and the second equation, at least one new location for at least one of the first pair of neighboring panoramic images or at least one of the second pair of neighboring panoramic images; and
storing the at least one new location in a three dimensional map.
6. The method of claim 5 , wherein calculating the at least one new location comprises:
minimizing a weighted absolute sum of error terms across the first equation and the second equation.
7. The method of claim 6 , wherein the quadratic program solver is semidefinite.
8. The method of claim 5 , further comprising:
adjusting at least one of the first pair of neighboring panoramic images or at least one of the second pair of neighboring panoramic images in a first direction and a second direction according to the new location.
9. The method of claim 5 , wherein pixels in the neighboring panoramic images are assigned to one or more planes representing physical structures or surfaces.
10. The method of claim 5 , further comprising:
filtering the panoramic images to remove panoramic images with at least one non-horizontal normal.
11. The method of claim 5 , further comprising:
generating a neighbor graph including the first pair of neighboring panoramic images and the second pair of neighboring panoramic images; and
removing at least one edge from the neighbor graph according to an angle between consecutive edges.
12. An apparatus comprising:
at least one processor; and
at least one memory including computer program code for one or more programs, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following:
identifying, from a set of panoramic images, a plurality of pairs of neighboring panoramic images separated by less than a threshold distance;
filtering the set of panoramic images to remove panoramic images with at least one non-horizontal normal;
generating a system of equations for the filtered plurality of pairs of neighboring panoramic images;
solving the system of equations for the plurality of pairs of neighboring panoramic images for a new location; and
storing the new location to a location value assigned to at least one of the neighboring panoramic images in a three dimensional map.
13. The apparatus of claim 12 , the computer program code configured to, with the at least one processor, cause the apparatus to further perform:
minimizing a weighted absolute sum of error terms across the first equation and the second equation using a quadratic program solver.
14. The apparatus of claim 12 , wherein pixels in the neighboring panoramic images are assigned to one or more planes representing physical structures or surfaces.
15. The apparatus of claim 12 , the computer program code configured to, with the at least one processor, cause the apparatus to further perform:
filtering the set of panoramic images to remove panoramic images below a predetermined size.
16. The apparatus of claim 12 , the computer program code configured to, with the at least one processor, cause the apparatus to further perform:
generating a neighbor graph including the set of panoramic images; and
removing at least one edge from the neighbor graph according to an angle between consecutive edges.
17. The apparatus of claim 12 , wherein one of the plurality of pairs of neighboring panoramic images includes images of different sides of a street.
18. The apparatus of claim 12 , wherein the system of equations for the plurality of pairs of neighboring panoramic images includes a first correction in a horizontal direction and a second correction in a vertical direction.