Method and device to determine the camera position and angle

1. A method for determining an attitude angle of a camera, the camera being fixed to one and the same rigid object in a vehicle along with an Inertial Measurement Unit (IMU), the method comprising:

obtaining IMU attitude angles outputted from the IMU and images captured by the camera;

determining a target IMU attitude angle corresponding to each frame of image based on respective capturing time of the frames of images and respective outputting time of the IMU attitude angles; and

determining an attitude angle of the camera corresponding to each frame of image based on a predetermined conversion relationship between a camera coordinate system for the camera and an IMU coordinate system for the IMU and the target IMU attitude angle corresponding to each frame of image, wherein determining the target IMU attitude angle corresponding to each frame of image based on the respective capturing time of the frames of images and the respective outputting time of the IMU attitude angles comprises, for each frame of image:

selecting from the IMU attitude angles at least one IMU attitude angle whose outputting time matches the capturing time of a current frame of image; and

determining the target IMU attitude angle corresponding to the current frame of image based on the selected at least one IMU attitude angle.

2. The method of claim 1 , wherein said selecting from the IMU attitude angles the at least one IMU attitude angle whose outputting time matches the capturing time of the current frame of image comprises:

sorting the IMU attitude angles in an order according to their respective outputting time, to obtain a sequence of IMU attitude angles; and

moving a predetermined sliding window over the sequence of IMU attitude angles such that the sliding window is centered at the capturing time of the current frame of image, and determining at least one IMU attitude angle within the sliding window as the at least one IMU attitude angle whose outputting time matches the capturing time of the current frame of image.

3. The method of claim 2 , wherein said determining the target IMU attitude angle corresponding to the current frame of image based on the selected at least one IMU attitude angle comprises:

obtaining a movement curve of the camera or the rigid object;

obtaining an IMU attitude angle curve by fitting the movement curve based on the selected at least one IMU attitude angle; and

obtaining from the IMU attitude angle curve an IMU attitude angle corresponding to the capturing time of the current frame of image, as the target IMU attitude angle corresponding to the current frame of image.

4. The method of claim 2 , wherein said determining the target IMU attitude angle corresponding to the current frame of image based on the selected at least one IMU attitude angle comprises:

calculating an average of the selected at least one IMU attitude angle, as the target IMU attitude angle corresponding to the current frame of image.

5. The method of claim 2 , wherein said determining the target IMU attitude angle corresponding to the current frame of image based on the selected at least one IMU attitude angle comprises:

calculating an absolute difference between the outputting time of each of the selected at least one IMU attitude angle and the capturing time of the current frame of image; and

determining the IMU attitude angle having the smallest absolute difference as the target IMU attitude angle corresponding to the current frame of image.

6. The method of claim 1 , further comprising:

measuring a distance to an object by performing for each frame of image captured by the camera;

obtaining the attitude angle of the camera corresponding to a current frame of image;

adjusting an external parameter between a camera coordinate system and a world coordinate system based on the attitude angle of the camera;

determining respective image coordinates of position points on a lane line on a map in the current frame of image and respective distance information associated with the position points based on a predetermined conversion relationship between an image coordinate system and the world coordinate system and the external parameter between the camera coordinate system and the world coordinate system; and

determining a position point on the lane line on the map in the current frame of image that is closest to a target object, and determining a distance to the target object based on the distance information associated with the position point.

7. The method of claim 1 , further comprising:

determining a position of the vehicle, by:

obtaining the attitude angle of the camera corresponding to each frame of image captured by the camera;

adjusting an external parameter between a camera coordinate system and a vehicle coordinate system based on the attitude angle of the camera; and

obtaining an attitude angle of the vehicle in a world coordinate system based on a conversion relationship between the camera coordinate system and the vehicle coordinate system, a predetermined conversion relationship between the vehicle coordinate system and the world coordinate system, and the attitude angle of the camera.

8. A system, comprising:

an apparatus to determine an attitude angle of a camera, comprising a processor configured to:

obtain Inertial Measurement Unit (IMU) attitude angles outputted from an IMU and images captured by a camera; the camera and the IMU being fixed to one and the same rigid object in a vehicle;

determine a target IMU attitude angle corresponding to each frame of image based on respective capturing time of the frames of images and respective outputting time of the IMU attitude angles; and

determine an attitude angle of the camera corresponding to each frame of image based on a predetermined conversion relationship between a camera coordinate system for the camera and an IMU coordinate system for the IMU and the target IMU attitude angle corresponding to each frame of image, wherein the target IMU attitude angle corresponding to each frame of image is determined by the processor configured to, for each frame of image;

select from the IMU attitude angles at least one IMU attitude angle whose outputting time matches the capturing time of the current frame of image; and

determine the target IMU attitude angle corresponding to the current frame of image based on the selected at least one IMU attitude angle.

9. The system of claim 8 , wherein the processor being configured to select from the IMU attitude angles the at least one IMU attitude angle whose outputting time matches the capturing time of the current frame of image comprises the processor being configured to:

sort the IMU attitude angles in an order according to their respective outputting time, to obtain a sequence of IMU attitude angles; and

move a predetermined sliding window over the sequence of IMU attitude angles such that the sliding window is centered at the capturing time of the current frame of image, and determine at least one IMU attitude angle within the sliding window as the at least one IMU attitude angle whose outputting time matches the capturing time of the current frame of image.

10. The system of claim 9 , wherein the processor being configured to determine the target IMU attitude angle corresponding to the current frame of image based on the selected at least one IMU attitude angle comprises the processor being configured to:

obtain a movement curve of the camera or the rigid object;

obtain an IMU attitude angle curve by fitting the movement curve based on the selected at least one IMU attitude angle; and

obtain from the IMU attitude angle curve an IMU attitude angle corresponding to the capturing time of the current frame of image, as the target IMU attitude angle corresponding to the current frame of image.

11. The system of claim 9 , wherein the processor being configured to determine the target IMU attitude angle corresponding to the current frame of image based on the selected at least one IMU attitude angle comprises the processor being configured to:

calculate an average of the selected at least one IMU attitude angle as the target IMU attitude angle corresponding to the current frame of image.

12. The system of claim 9 , wherein the processor being configured to determine the target IMU attitude angle corresponding to the current frame of image based on the selected at least one IMU attitude angle comprises the processor being configured to:

calculate an absolute difference between the outputting time of each of the selected at least one IMU attitude angle and the capturing time of the current frame of image; and

determine the IMU attitude angle having the smallest absolute difference as the target IMU attitude angle corresponding to the current frame of image.

13. The system of claim 8 , further comprising:

a second apparatus to measure a distance to an object, connected to the apparatus to determine the attitude angle of the camera, the second apparatus comprising a processor configured to:

obtain, from the apparatus to determine the attitude angle of the camera, the attitude angle of the camera corresponding to a current frame of image captured by the camera;

adjust an external parameter between a camera coordinate system and a world coordinate system based on the attitude angle of the camera;

determine respective image coordinates of position points on a lane line on a map in the current frame of image and respective distance information associated with the position points based on a predetermined conversion relationship between an image coordinate system and the world coordinate system and the external parameter between the camera coordinate system and the world coordinate system; and

determine the position point on the lane line on the map in the current frame of image that is closest to a target object, and determine a distance to the target object based on the distance information associated with the position point.

14. The system of claim 8 , further comprising:

a second apparatus to determine a position of the vehicle,

connected to the apparatus to determine the attitude angle of the camera, the second apparatus comprising a processor configured to:

obtain, from the apparatus for determining the attitude angle of the camera, the attitude angle of the camera corresponding to each frame of image captured by the camera;

adjust an external parameter between a camera coordinate system and a vehicle coordinate system based on the attitude angle of the camera; and

obtain an attitude angle of the vehicle in a world coordinate system based on a conversion relationship between the camera coordinate system and the vehicle coordinate system and a predetermined conversion relationship between the vehicle coordinate system and the world coordinate system, and the attitude angle of the camera.

15. A non-transitory computer readable storage medium that includes instructions that upon execution by a processor, configures the processor to perform a method to determine an attitude angle of a camera, the camera being fixed to one and the same rigid object in a vehicle along with an Inertial Measurement Unit (IMU), the method comprising:

obtaining IMU attitude angles outputted from the IMU and images captured by the camera;

determining a target IMU attitude angle corresponding to each frame of image based on respective capturing time of the frames of images and respective outputting time of the IMU attitude angles; and

determining an attitude angle of the camera corresponding to each frame of image based on a predetermined conversion relationship between a camera coordinate system for the camera and an IMU coordinate system for the IMU and the target IMU attitude angle corresponding to each frame of image, wherein said determining the target IMU attitude angle corresponding to each frame of image based on the respective capturing time of the frames of images and the respective outputting time of the IMU attitude angles comprises, for each frame of image:

selecting from the IMU attitude angles at least one IMU attitude angle whose outputting time matches the capturing time of a current frame of image; and

determining the target IMU attitude angle corresponding to the current frame of image based on the selected at least one IMU attitude angle.

16. The non-transitory computer readable storage medium of claim 15 , wherein said selecting from the IMU attitude angles the at least one IMU attitude angle whose outputting time matches the capturing time of the current frame of image comprises:

sorting the IMU attitude angles in an order according to their respective outputting time, to obtain a sequence of IMU attitude angles; and

moving a predetermined sliding window over the sequence of IMU attitude angles such that the sliding window is centered at the capturing time of the current frame of image, and determining at least one IMU attitude angle within the sliding window as the at least one IMU attitude angle whose outputting time matches the capturing time of the current frame of image.