Single image pose estimation of image capture devices

1. A method for image based localization using an electronic computing device, the method comprising:

capturing a local image with an image capture device;

associating metadata with the local image;

causing the electronic computing device to receive the local image;

causing the electronic computing device to match the local image with at least one database image, wherein the at least one database image is three-dimensional (3D); and

calculating a pose of the image capture device based on a pose of the at least one database image and metadata associated with the local image, wherein

the metadata includes at least pitch and roll data corresponding with the image capture device at a time of image capture, wherein

the calculating the pose comprises:

extracting the pitch and roll data from the metadata;

extracting yaw data from the metadata;

matching features between the local image and the database image; and

estimating a homography matrix to solve for translation of the image capture device with respect to the database image, and wherein

the matching features comprises:

extracting high resolution scale invariant feature transform (SIFT) features from the local image and the database image;

performing a nearest neighbor search to establish any correspondence between the extracted SIFT features of the local image and the database image;

excluding any corresponding database image SIFT features that lack depth data; and

excluding ground plane SIFT features of the database image.

2. The method of claim 1 further comprising:

if the yaw data from metadata of the local image captured is noisy,

extracting a first plurality of vanishing points corresponding with the local image;

extracting a second plurality of vanishing points corresponding with the database image;

finding a best alignment pair between the vanishing points;

calculating a corrected yaw of the image capture device; and

calculating a normal direction of a vertical plane associated with each vanishing point in the local image that was matched to the vanishing point in the database image.

3. The method of claim 2 , wherein estimating the homography matrix comprises:

recovering a 3D translation vector, wherein the 3D translation vector includes direction and magnitude, between the local image and the matched database image; and

recovering a perpendicular distance between a center of the image capture device and the vertical plane in the local image for which the first plurality of vanishing points is extracted.

4. The method of claim 1 , wherein the image capture device is selected from the group consisting of: a mobile communication device, a digital camera, and a digital camcorder.

5. The method of claim 1 , wherein the image capture device includes a sensor selected from the group consisting of: a gyroscope, an accelerometer, a compass, and a magnetometer.

6. A method for image based localization using an electronic computing device, the method comprising:

causing the electronic computing device to receive a local image captured from an image capture device;

causing the electronic computing device to receive metadata associated with the local image;

causing the electronic computing device to match the local image with at least one database image, wherein the at least one database image is 3D; and

calculating a pose of the image capture device based on a pose of the at least one database image and metadata associated with the local image, wherein

the metadata includes at least pitch and roll data corresponding with the image capture device at a time of image capture, wherein

the calculating the pose comprises:

extracting the pitch and roll data from the metadata;

extracting yaw data from the metadata;

matching features between the local image and the database image; and

estimating a homography matrix to solve for translation of the image capture device with respect to the database image, and wherein

the matching features comprises:

extracting high resolution scale invariant feature transform (SIFT) features from the local image and the database image;

performing a nearest neighbor search to establish any correspondence between the extracted SIFT features of the local image and the database image;

excluding any corresponding database image SIFT features that lack depth data; and

excluding ground plane SIFT features of the database image.

7. The method of claim 6 further comprising:

if the yaw data from the metadata of the local image captured is noisy,

extracting a first plurality of vanishing points corresponding with the local image;

extracting a second plurality of vanishing points corresponding with the database image;

finding a best alignment pair between the vanishing points;

calculating a corrected yaw of the image capture device; and

calculating a normal direction of a vertical plane associated with each vanishing point in the local image that was matched to the vanishing point in the database image.

8. The method of claim 7 , wherein estimating the homography matrix comprises:

recovering a 3D translation vector, wherein the 3D translation vector includes direction and magnitude between the local image and the matched database image; and

recovering a perpendicular distance between a center of the image capture device and the vertical plane in the local image for which the first plurality of vanishing points is extracted.

9. The method of claim 6 , wherein the image capture device includes a sensor selected from the group consisting of: a gyroscope, an accelerometer, a compass, and a magnetometer.

10. A computing device program product for image based localization using a computing device, the computing device program product comprising:

a non-transitory computer readable medium;

first programmatic instructions for receiving a local image captured from an image capture device;

second programmatic instructions for receiving metadata associated with the local image;

third programmatic instructions for matching the local image with at least one database image, wherein the at least one database image is 3D; and

fourth programmatic instructions for calculating a pose of the image capture device based on a pose of the at least one database image and metadata associated with the local image, wherein

the metadata includes at least pitch and roll data corresponding with the image capture device at a time of image capture, wherein

the calculating the pose comprises:

fifth programmatic instructions for extracting the pitch and roll data from the metadata;

sixth programmatic instructions for extracting yaw data from the metadata;

seventh programmatic instructions for matching features between the local image and the database image; and

eighth programmatic instructions for estimating a homography matrix to solve for translation of the image capture device with respect to the database image, and wherein

the matching features comprises:

ninth programmatic instructions for extracting high resolution scale invariant feature transform (SIFT) features from the local image and the database image;

tenth programmatic instructions for performing a nearest neighbor search to establish any correspondence between the extracted SIFT features of the local image and the database image;

eleventh programmatic instructions for excluding any corresponding database image SIFT features that lack depth data; and

twelfth programmatic instructions for excluding ground plane SIFT features of the database image.

11. The computing device program product of claim 10 further comprising:

if the yaw data from the metadata of the local image captured is noisy,

thirteenth programmatic instructions for extracting a first plurality of vanishing points corresponding with the local image;

fourteenth programmatic instructions for extracting a second plurality of vanishing points corresponding with the database image;

fifteenth programmatic instructions for finding a best alignment pair between the vanishing points;

sixteenth programmatic instructions for calculating a corrected yaw of the image capture device; and

seventeenth programmatic instructions for calculating a normal direction of a vertical plane associated with each vanishing point in the local image that was matched to the vanishing point in the database image.