Camera-based inertial sensor alignment for PND

1. A mobile device using images for navigation, the mobile device comprising:

a camera, wherein the camera acquires a sequence of images;

a memory to hold three rotation matrices comprising:

a first rotation matrix, wherein the first rotation matrix is for converting between a body reference system of the mobile device and a platform reference system of a vehicle;

a second rotation matrix, wherein the second rotation matrix is for converting between the platform reference system of the vehicle and a local reference system; and

a third rotation matrix, wherein the third rotation matrix is for converting between the body reference system and the local reference system; and

a processor coupled to the camera and the memory, wherein the processor is configured to:

set an initial value of the first rotation matrix based on the sequence of images; and

set an initial value of the third rotation matrix based on the sequence of images.

2. The mobile device of claim 1 , further comprising:

an accelerometer to provide accelerometer measurements;

wherein the processor is further coupled to the accelerometer; and

wherein the processor is further configured to set an initial value of a gravitational vector based on the sequence of images, wherein the gravitational vector is used to null an effect of gravity in the accelerometer measurements.

3. The mobile device of claim 2 , the accelerometer comprises a three-dimensional accelerometer.

4. The mobile device of claim 1 , further comprising:

a gyrometer functionally coupled to the camera;

wherein the processor is further coupled to the gyrometer.

5. The mobile device of claim 4 , wherein the processor is further configured to update calibration of the gyrometer based on the sequence of images.

6. The mobile device of claim 1 , further comprising a position location module.

7. The mobile device of claim 6 , wherein the position location module comprises a global navigation satellite system receiver (GNSS receiver).

8. The mobile device of claim 7 , wherein the GNSS receiver comprises a global positioning satellite (GPS) receiver.

9. The mobile device of claim 1 , wherein the mobile device is mechanically coupled to the vehicle.

10. The mobile device of claim 1 , wherein the processor is further configured to perform dead reckoning based on the sequence of images.

11. The mobile device of claim 1 , further comprising:

an accelerometer to provide accelerometer measurements; and

a gyrometer functionally coupled to the camera;

wherein the processor is further coupled to the accelerometer and the gyrometer; and

wherein the processor is further configured to:

set an initial value of a gravitational vector based on the sequence of images, wherein the gravitational vector is used to null an effect of gravity in acceleration measurements; and

update calibration of the gyrometer based on the sequence of images.

12. A method in a mobile device of using images for navigation, the method comprising:

acquiring a sequence of images from a camera;

setting an initial value of a first rotation matrix based on the sequence of images, wherein the first rotation matrix is for converting between a body reference system of the mobile device and a platform reference system of a vehicle;

setting an initial value of a second rotation matrix based on the sequence of images, wherein the second rotation matrix is for converting between the platform reference system of the vehicle and a local reference system;

setting an initial value of a third rotation matrix based on the sequence of images, wherein the third rotation matrix is for converting between the body reference system and the local reference system;

writing the second rotation matrix to memory; and

converting measurements, between a body reference system and a local reference system, using the third rotation matrix.

13. The method of claim 12 , further comprising

setting an initial value of a gravitational vector based on the sequence of images;

wherein the gravitational vector is used to null an effect of gravity in accelerometer measurements.

14. The method of claim 12 , further comprising updating a calibration of a gyrometer based on the sequence of images.

15. The method of claim 12 , further comprising mechanically coupling the mobile device to a vehicle.

16. The method of claim 12 , wherein acquiring the sequence of images comprises resampling the sequence of images, having a first resolution, to a second resolution, wherein the second resolution is lower than the first resolution.

17. The method of claim 12 , wherein acquiring the sequence of images comprises extracting intensity values from the sequence of images.

18. The method of claim 12 , wherein setting the initial value of the second rotation matrix based on the sequence of images comprises determining a Focus of Expansion point (FOE point) derived from at least two of the sequence of images to set a direction of movement of the mobile device in a body reference system, and further comprising:

determining a gravitational vector in the body reference system; and

computing the first rotation matrix based on:

(1) the third rotation matrix; and

(2) an inverse of the second rotation matrix.

19. The method of claim 18 , wherein the FOE point indicates a height angle (α) and a width angle (β) relative to the body reference system.

20. The method of claim 18 , wherein the FOE point indicates a point in an X-Y image plane relative to the body reference system.

21. The method of claim 18 , wherein determining the FOE point comprises:

extracting a plurality of features from a first image of the sequence of images thereby defining a corresponding first plurality of points;

tracking the plurality of features on a second image of the sequence of images thereby defining a corresponding second plurality of points and further defining a corresponding plurality of lines between the corresponding first plurality of points and the corresponding second plurality of points; and

setting the FOE point at a center of the corresponding plurality of lines.

22. The method of claim 21 , wherein the center of the corresponding plurality of lines comprises a least-squares error point of the corresponding plurality of lines.

23. The method of claim 21 , wherein the center of the corresponding plurality of lines is based on Kalman filter estimate.

24. The method of claim 12 , wherein acquiring the sequence of images comprises correcting the sequence of images to remove a lens aberration.

25. A mobile device using images for navigation, the mobile device comprising:

means for acquiring a sequence of images from a camera;

means for setting an initial value of a first rotation matrix based on the sequence of images used to convert from a body reference system of the mobile device and a platform reference system of a vehicle;

means for setting an initial value of a second rotation matrix based on the sequence of images, wherein the second rotation matrix is for converting between the platform reference system of the vehicle and a local reference system;

means for setting an initial value of a third rotation matrix based on the sequence of images, wherein the third rotation matrix is for converting between the body reference system and the local reference system;

means for writing the third rotation matrix to memory; and

means for converting measurements, between the body reference system of the mobile device and a local reference system, using the third rotation matrix.

26. A non-transitory computer-readable medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for using images for navigation in a mobile device having a camera, memory and a processor, said method comprising operations of:

acquiring a sequence of images from a camera;

setting an initial value of a first rotation matrix based on the sequence of images, wherein the first rotation matrix is for converting between a body reference system of the mobile device and a platform reference system of a vehicle;

setting an initial value of a second rotation matrix based on the sequence of images, wherein the second rotation matrix is for converting between the platform reference system of the vehicle and a local reference system;

setting an initial value of a third rotation matrix based on the sequence of images, wherein the third rotation matrix is for converting between the body reference system and the local reference system;

writing the third rotation matrix to memory; and

converting measurements, between the body reference system of the mobile device and a local reference system, using the third rotation matrix.