Method for determining a location from images acquired of an environment with an omni-directional camera

1. A method for determining a location and orientation in an environment, comprising, a processor for performing steps of the method, comprising the steps of:

acquiring a real omni-directional image of an unknown skyline in the environment from an unknown location and unknown orientation in the environment;

synthesizing a set of virtual omni-directional images from a 3D model of the environment, wherein each virtual omni-directional image is associated with a known skyline, a known location and a known orientation; and

comparing the real omni-directional image with each virtual omni-directional images to determine a best matching virtual omni-directional image with the associated known location and orientation that correspond to the unknown location and orientation.

2. The method of claim 1 , further comprising:

extracting features from pixels in the real and virtual images, and the comparing is based on the features.

3. The method of claim 1 , wherein the environment is urban.

4. The method of claim 1 , wherein the omni-directional camera is arranged in a vehicle, and the 3D model is acquired in real time using a communication network as the virtual images are generated.

5. The method of claim 1 , wherein the synthesizing is performed in real time from the 3D model acquired in real-time using a network.

6. The method of claim 2 , wherein the features include pixel intensities.

7. The method of claim 1 , further comprising:

segmenting the real image using the features into foreground and back ground pixels, and the background pixels correspond to a sky, and wherein the foreground pixels adjacent to the back ground pixels define a skyline.

8. The method of claim 1 , further comprising:

calibrating the omni-directional camera using a generic imaging model.

9. The method of claim 1 , wherein the synthesizing further comprises:

generating a set of perspective images for each virtual image from the 3D model; and

combining the perspective images to form the virtual omni directional image.

10. The method of claim 1 , wherein each virtual omni-directional image is a binary image.

11. The method of claim 2 , wherein the comparing uses a chamfer distance.

12. The method of claim 1 , wherein each virtual omni-directional image is synthesized at multiple resolutions.

13. The method of claim 2 , wherein the features include color, gradients, lines and vanishing points.

14. The method of claim 8 , wherein the generic calibration model can be applied to arbitrary combinations of cameras, mirrors and lenses.

15. The method of claim 1 , wherein the set of virtual omni-directional images are synthesized in real time.

16. The method of claim 1 , wherein the environment is an urban setting.

17. The method of claim 1 , wherein the environment is indoor.

18. The method of claim 1 , wherein the real omni-directional image is acquired using visible light.

19. The method of claim 7 , wherein the segmenting is performed using a graphcut.

20. The method of claim 8 , wherein the synthesizing uses a generic projection.

21. A system for determining a location and orientation in an environment, comprising:

an omni-directional camera configured to acquire a real omni-directional image of an unknown skyline in the environment from an unknown location and unknown orientation in the environment;

means for synthesizing a set of virtual omni-directional images from a 3D model of the environment, wherein each virtual omni-directional image is associated with a known skyline, a known location, and a known orientation; and

means for comparing the real omni-directional image with each virtual omni-directional images to determine a best matching virtual omni-directional image with the associated known location and known orientation that correspond to the unknown location and unknown orientation.