Image location through large object detection
Camera pose optimization, which includes determining the position and orientation of a camera in three-dimensional space at different times, is improved by detecting a higher-confidence reference object in the photographs captured by the camera and using the object to increase consistency and accuracy of pose data. Higher-confidence reference objects include objects that are stationary, fixed, easily recognized, and relatively large. In one embodiment, street level photographs of a geographic area are collected by a vehicle with a camera. The captured images are geo-coded using GPS data, which may be inaccurate. The vehicle drives in a loop and captures the same reference object multiple times from the substantially same position. The trajectory of the vehicle is then closed by aligning the points of multiple images where the trajectory crosses itself. This creates an additional constraint on the pose data, which in turn improves the data's consistency and accuracy.
1. A method for automatically correcting camera pose data, the method comprising:
obtaining, by one or more computing devices, a plurality of images of a geographic area and image pose data;
identifying, by the one or more computing devices, a reference object captured in at least one of the plurality of images;
determining, by the one or more computing devices, an estimated location of the reference object;
selecting, by the one or more computing devices, from the plurality of images, a plurality of groups of images in which the reference object is captured;
generating, by the one or more computing devices, corrected image pose data based at least in part on the plurality of groups of images and the estimated location of the reference object by aligning, by the one or more computing devices, one or more common points on a surface of the reference object within the plurality of groups of images in which the reference object is captured by aligning images from each of the plurality of groups captured from different positions from each other.
2. The method of claim 1 , wherein identifying the reference object includes, identifying, by the one or more computing devices, the reference object based on a size of the reference object.
3. The method of claim 1 , wherein identifying the reference object includes, identifying, by the one or more computing devices, the reference object based on the reference being stationary.
4. The method of claim 1 , wherein identifying the reference object includes, using a computer vision system.
5. The method of claim 1 , wherein selecting the plurality of groups of images in which the reference object is captured includes selecting, by the one or more computing devices, the plurality of groups of images in which reference object is captured at a same location.
6. The method of claim 5 , wherein the same location includes an intersection.
7. The method of claim 1 , wherein determining the estimated location of the reference object, includes receiving, by the one or computing devices, survey data, GPS data or local positioning service data.
8. The method of claim 1 , wherein determining the estimated location of the reference object, includes applying, by the one or computing devices, differential global positioning system techniques.
9. A system for automatically correcting camera pose data, the system comprising:
one or more computing devices;
a non-transitory computer-readable medium storing thereon a plurality of instructions that, when executed by the one or more computing devices, cause the system to:
obtain a plurality of images of a geographic area and image pose data,
obtain a reference object size threshold,
identify a reference object captured in at least several of the plurality of images,
determine an estimated location of the reference object,
select from the plurality of images, a plurality of groups of images in which the reference object is captured; and
generate corrected image pose data based at least in part on the estimated location of the reference object and those of the plurality of images that capture the reference object by aligning one or more common points on a surface of the reference object within the plurality of groups of images in which the reference object is captured by aligning images from each of the plurality of groups captured from different positions from each other.
10. The system of claim 9 , wherein to identify the reference object, the instructions cause the system to identify the reference object based on a size of the reference object.
11. The system of claim 9 , wherein to identify the reference object, the instructions cause the system to identify the reference object based on the reference being stationary.
12. The system of claim 9 wherein to identify the reference object, the instructions cause the system to identify the reference object using a computer vision system.
13. The system of claim 9 , wherein to select from the plurality of images, the instructions cause the system to select from the plurality of images, a plurality of groups of images in which the reference object is captured at a same location.
14. The system of claim 13 , wherein the same location includes an intersection.
15. The system of claim 9 , wherein to determine an estimated location of the reference object, the instructions cause the system to receive survey data, GPS data or local positioning service data.
16. The system of claim 9 , wherein to determine an estimated location of the reference object, the instructions cause the system to apply differential global positioning system techniques.
17. A non-transitory computer-readable medium storing thereon instructions for automatically correcting camera pose data, wherein the instructions, when executed by one or more computing devices, cause the one or more computing devices to:
obtain a plurality of images of a geographic area and image pose data:
receiving a first set of images collected along a first path, and
receiving a second set of images collected along a second path;
identify a reference object captured in at least several of the plurality of images;
determine an estimated location of the reference object;
select, from the plurality of images comprising the first and second sets of images, a group of images in which the reference object is captured;
generate corrected image pose data based at least in part on the group of images and the estimated location of the reference object by aligning one or more common points on a surface of the reference object within the group of images in which the reference object is captured and aligning images from the first set and the second set captured from first and second positions, respectively; and
generate a corrected estimated location of the reference object based at least in part on the image pose data and the estimated location of the reference object.
18. The computer-readable medium of claim 17 , wherein the instructions further cause the one or more computing devices to identify the reference object based on a size of the reference object.
19. The computer-readable medium of claim 17 , wherein the instructions further cause the one or more computing devices to identify the reference object based on the reference being stationary.
20. The computer-readable medium of claim 17 , wherein the instructions further cause the one or more computing devices to select from the plurality of images,
a plurality of groups of images in which the reference object is captured at a same location.