Three-dimensional towered checkerboard for multi-sensor calibration, and LiDAR and camera joint calibration method based on the checkerboard

1. A LiDAR and camera joint calibration method based on a three-dimensional towered checkerboard, wherein the method is based on the three-dimensional towered checkerboard, the three-dimensional towered checkerboard is a multilayer structure formed by stacking a plurality of cubes, and an orientation of each of the cubes are the same; planes, having a uniform orientation, of the cubes on each of layers are provided with two-dimensional checkerboards; an edge length of the two-dimensional checkerboards is less than an edge length of the cubes; and

the method comprises:

establishing a modeling coordinate system taking the three-dimensional towered checkerboard as a basis, and generating a simulated point cloud of the three-dimensional towered checkerboard;

denoising a three-dimensional point cloud obtained by LiDAR, and obtaining an actual point cloud of the three-dimensional towered checkerboard under a LiDAR coordinate system;

determining a transformation relationship between the LiDAR coordinate system and the modeling coordinate system according to the simulated point cloud and the actual point cloud;

generating a corner point set in sequence according to actual positions of corners of the two-dimensional checkerboards on the three-dimensional towered checkerboard in the modeling coordinate system;

transforming the corner point set into the LiDAR coordinate system according to the transformation relationship between the LiDAR coordinate system and the modeling coordinate system;

searching for the corners of the two-dimensional checkerboards by using a photo of the three-dimensional towered checkerboard captured by a camera, and obtaining the corner point set of the two-dimensional checkerboards on the photo; and

determining a transformation relationship between a camera coordinate system and the LiDAR coordinate system according to the corner point set under the LiDAR coordinate system and the corner point set under the two-dimensional checkerboard; the transformation relationship between the camera coordinate system and the LiDAR coordinate system being a joint calibration result.

2. The method according to claim 1 , wherein the establishing the modeling coordinate system taking the three-dimensional towered checkerboard as a basis, and generating the simulated point cloud of the three-dimensional towered checkerboard comprises:

by taking a bottom left protrusion of the three-dimensional towered checkerboard as an origin, along a distribution direction of the three-dimensional towered checkerboard, respectively establishing an x-axis, a y-axis and a z-axis, to establish the modeling coordinate system; and

generating the simulated point cloud of the three-dimensional towered checkerboard under the modeling coordinate system according to structure information of the three-dimensional towered checkerboard, with a point spacing of the simulated point cloud being less than 1 mm.

3. The method according to claim 1 , wherein the denoising the three-dimensional point cloud obtained by the LiDAR, and obtaining the actual point cloud of the three-dimensional towered checkerboard under the LiDAR coordinate system comprises:

removing points of the three-dimensional towered checkerboard on a ground and in an uncorrelated interval according to an interval where the three-dimensional towered checkerboard is located;

fitting a ground plane where the three-dimensional towered checkerboard is located, and removing all the points on the ground plane; the ground plane is a plane having the largest number of points in a space of the three-dimensional towered checkerboard and having a normal vector which goes approximately vertically upward; and

taking the three-dimensional point cloud after all the points on the ground plane and in the uncorrelated interval are removed as the actual point cloud of the three-dimensional towered checkerboard under the LiDAR coordinate system.

4. The method according to claim 1 , wherein the determining the transformation relationship between the LiDAR coordinate system and the modeling coordinate system according to the simulated point cloud and the actual point cloud comprises:

performing, by using an iterative closest point algorithm, three-dimensional point cloud registration on the simulated point cloud generated under the modeling coordinate system and the actual point cloud of the three-dimensional towered checkerboard under the LiDAR coordinate system, and determining the transformation relationship between the LiDAR coordinate system and the modeling coordinate system.

5. The method according to claim 1 , wherein the generating the corner point set in sequence according to the actual positions of the corners of the two-dimensional checkerboards on the three-dimensional towered checkerboard in the modeling coordinate system comprises:

numbering the two-dimensional checkerboards according to a predetermined sequence; and

generating the corner point set on the basis of serial numbers, in a sequence from smallest to largest, and according to the predetermined sequence.

6. The method according to claim 1 , wherein the transforming the corner point set into the LiDAR coordinate system according to the transformation relationship between the LiDAR coordinate system and the modeling coordinate system comprises:

a calculation method for the corner point set under the LiDAR coordinate system:

S L =F L−M *S M

in the equation, S L is the corner point set under the LiDAR coordinate system, F L−M is the transformation relationship between the LiDAR coordinate system and the modeling coordinate system, and S M is the corner point set generated under the modeling coordinate system.

7. The method according to claim 1 , wherein the searching for the corners of the two-dimensional checkerboards by using a photo of the three-dimensional towered checkerboard captured by a camera, and obtaining the corner point set of the two-dimensional checkerboards on the photo comprises:

sequentially obtaining target checkerboard areas having consistent lengths and widths as the two-dimensional checkerboards from the photo of the three-dimensional towered checkerboard;

every time a group of the target checkerboard areas is obtained, setting the target checkerboard areas black until the photo of the three-dimensional towered checkerboard does not contain the target checkerboard areas; and

arranging checkerboard corners in all the target checkerboard areas from left to right and from top to bottom, to obtain the corner point set of the two-dimensional checkerboards on the photo.

8. The method according to claim 1 , wherein the determining a transformation relationship between the camera coordinate system and the LiDAR coordinate system according to the corner point set under the LiDAR coordinate system and the corner point set under the two-dimensional checkerboard comprises:

performing, by using a perspective-n-point positioning method, camera pose measurement and matching on the corner point set under the LiDAR coordinate system and the corner point set obtained from the photo, and according to camera intrinsic and distortion parameters, determining the transformation relationship between the camera coordinate system and the LiDAR coordinate system.