Camera calibration method based on vehicle localization
The present disclosure provides a camera calibration method based on vehicle localization, mainly considering that a vehicle is not of an ideal rigid structure, and shaking of a vehicle body during operation, uneven road surfaces, or different loads will cause variations in extrinsic parameters of a camera system, resulting in perceptual information errors. Therefore, through perceptual information from a fused image, an inertial measurement unit, a speedometer, and the like, combined with the road and marking information provided by a high-definition map, the relative motion between the vehicle body and a vehicle chassis is simultaneously described with a mass-spring-damper model, so as to complete the determination of six-degree-of-freedom positions of the vehicle body and the vehicle chassis, and finally the extrinsic parameters of cameras are immediately calibrated with the vehicle chassis as reference coordinates.
1. A camera calibration method based on vehicle localization, comprising:
obtaining a speed and acceleration of a three-dimensional displacement of a vehicle chassis and an angular speed and angular acceleration of a three-dimensional rotation angle of the vehicle chassis to estimate a current initial six-degree-of-freedom position of the chassis of a vehicle;
disposing an surround view cameras comprising a plurality of cameras on the vehicle, using the surround view cameras to obtain an around view image of an vicinity of a body of the vehicle, detecting an image object feature point set in the around view image and vanishing points of all the cameras of the surround view cameras, and matching with a semantic map feature point set of the vicinity of the vehicle, to calculate six-degree-of-freedom positions of all the cameras; connecting a mass-spring-damper model between the vehicle body and the vehicle chassis, and estimating, by the mass-spring-damper model, a three-dimensional relative displacement between the vehicle body and the vehicle chassis with the surround view cameras as a reference point;
iterating the six-degree-of-freedom position of the vehicle chassis according to a road surface normal vector and the rotation angle of the vehicle chassis, to complete determination of six-degree-of-freedom positions of the vehicle body and the vehicle chassis; and
calibrating the six-degree-of-freedom positions of the cameras with the vehicle chassis as reference coordinates.
2. The camera calibration method based on vehicle localization according to claim 1 , wherein the current initial six-degree-of-freedom position of the vehicle chassis is estimated by using speed and acceleration information of the vehicle calculated by a speedometer and an inertial measurement unit, and taking a position of the vehicle chassis at a previous sampling time point as a reference point to calculate a position of the vehicle chassis at a current sampling time point.
3. The camera calibration method based on vehicle localization according to claim 1 , wherein the road surface normal vector is generated by querying high-definition map information according to a position of the vehicle chassis.
4. The camera calibration method based on vehicle localization according to claim 1 , wherein the semantic map feature point set of the vicinity of the vehicle is generated by querying high-definition map information with a position of the vehicle chassis, and comprises a set of a road marking edge and a traffic sign edge.
5. The camera calibration method based on vehicle localization according to claim 1 , wherein the vanishing points of all the cameras of the surround view cameras and the image object feature point set are calculated according to a plurality of image features in the surround-view images.
6. The camera calibration method based on vehicle localization according to claim 1 , wherein as for the six-degree-of-freedom positions of all the cameras of the surround view cameras, pitch and yaw angles of each of the cameras are calculated according to the vanishing points of the plurality of cameras in the surround view cameras; the plurality of cameras are configured on the vehicle body and are in rigid connection with the vehicle body, and the pitch and yaw angles of the plurality of cameras are used for determining an initial value of a three-dimensional rotation angle of the vehicle body; and matching is conducted according to the semantic map feature point set of the vicinity of the vehicle and the image object feature point set detected by the cameras to calculate the six-degree-of-freedom positions of the cameras and the six-degree-of-freedom position of the vehicle body.
7. The camera calibration method based on vehicle localization according to claim 1 , wherein the six-degree-of-freedom position of the vehicle body and the six-degree-of-freedom position of the vehicle chassis are iterated by, according to an initial value of the six-degree-of-freedom position of the vehicle body, an initial value of the six-degree-of-freedom position of the vehicle chassis, an initial value of the speed and acceleration of the vehicle chassis, and the road surface normal vector obtained by querying a high-definition map according to a position of the vehicle chassis, using a mass-spring-damper model to describe relative motion of the vehicle body and the vehicle chassis, dynamic values of the mass-spring-damper model, comprising a position, a speed and an acceleration are continuously corrected, until an external force applied to the vehicle body and the vehicle chassis is approximate to a dynamic variation of the mass-spring-damper model and the six-degree-of-freedom position of the vehicle body and the six-degree-of-freedom position of the vehicle chassis are generated, and the six-degree-of-freedom positions of the cameras of the surround view cameras are calculated.