Non-Repetitive Scanning Solid State LiDAR and Camera Extrinsic Calibration
A system performs pose calibration between a LiDAR sensor and a camera. The system may receive an image frame captured by the camera mounted on a device. The system may receive a point cloud captured by the LiDAR sensor mounted on the device, the LiDAR sensor having an overlapping field of view with the camera. The system may identify markers of a calibration target captured in the image frame by applying a feature identification model to the image frame. The system may identify the markers of the calibration target captured in the point cloud by: clustering points in the point cloud into one or more planes, selecting one of the planes based on sizes of the planes; and identifying holes in the selected plane as the markers of the calibration target. The system may determine a pose transformation between the camera and the LiDAR sensor based on information identifying the markers from the image frame and information identifying the markers from the point cloud.
1 . A computer-implemented method for performing extrinsic calibration between a camera and a light detection and ranging (LiDAR) sensor, the method comprising:
receiving an image frame captured by the camera mounted on a device;
receiving a point cloud captured by the LiDAR sensor mounted on the device, the LiDAR sensor having an overlapping field of view with the camera;
identifying markers of a calibration target captured in the image frame by applying a feature identification model to the image frame;
identifying the markers of the calibration target captured in the point cloud by:
clustering points in the point cloud into one or more planes;
selecting one of the planes based on sizes of the planes; and
identifying holes in the selected plane as the markers of the calibration target; and
determining a pose transformation between the camera and the LiDAR sensor based on information identifying the markers from the image frame and information identifying the markers from the point cloud.
2 . The computer-implemented method of claim 1 , wherein determining the pose transformation between the camera and the LiDAR sensor is further based on intrinsic parameters of the camera.
3 . The computer-implemented method of claim 1 , wherein the camera is part of a stereoscopic camera pair mounted on the device.
4 . The computer-implemented method of claim 1 , wherein the LiDAR sensor is a non-repetitive scanning solid state LiDAR sensor.
5 . The computer-implemented method of claim 1 , wherein identifying the markers of the calibration target captured in the point cloud further comprises:
projecting points clustered in the selected plane into a two-dimensional grid; and
identifying the holes in the projected points.
6 . The computer-implemented method of claim 1 , wherein identifying the markers of the calibration target captured in the point cloud further comprises:
projecting the markers into three-dimensional coordinate system of the point cloud;
determining three-dimensional coordinates for each marker in the three-dimensional coordinate system.
7 . The computer-implemented method of claim 1 , wherein clustering the points in the point cloud into one or more planes comprises:
performing a voxel growing approach to incrementally capture points into one cluster of points; and
identifying the one or more planes from the clusters of points.
8 . The computer-implemented method of claim 1 , wherein selecting one of the planes sized to match the calibration target comprises selecting the plane of largest size.
9 . The computer-implemented method of claim 1 , wherein identifying the holes in the selected plane as the markers of the calibration target comprises identifying the holes informed by a spatial configuration of the markers in the calibration target.
10 . The computer-implemented method of claim 1 , wherein determining the pose transformation comprises performing a Perspective-n-Point algorithm with the markers in the image frame and the markers in the point cloud to determine the pose transformation.
11 . A system for performing extrinsic calibration between a camera and a light detection and ranging (LiDAR) sensor, the system comprising:
a processor; and
a non-transitory computer-readable storage medium storing instructions that, when executed by the processor, cause the processor to perform operations comprising:
receiving an image frame captured by the camera mounted on a device;
receiving a point cloud captured by the LiDAR sensor mounted on the device, the LiDAR sensor having an overlapping field of view with the camera;
identifying markers of a calibration target captured in the image frame by applying a feature identification model to the image frame;
identifying the markers of the calibration target captured in the point cloud by:
clustering points in the point cloud into one or more planes;
selecting one of the planes based on sizes of the planes; and
identifying holes in the selected plane as the markers of the calibration target; and
determining a pose transformation between the camera and the LiDAR sensor based on information identifying the markers from the image frame and information identifying the markers from the point cloud.
12 . The system of claim 11 , wherein determining the pose transformation between the camera and the LiDAR sensor is further based on intrinsic parameters of the camera.
13 . The system of claim 11 , wherein the camera is part of a stereoscopic camera pair mounted on the device.
14 . The system of claim 11 , wherein the LiDAR sensor is a non-repetitive scanning solid state LiDAR sensor.
15 . The system of claim 11 , wherein identifying the markers of the calibration target captured in the point cloud further comprises:
projecting points clustered in the selected plane into a two-dimensional grid; and
identifying the holes in the projected points.
16 . The system of claim 11 , wherein identifying the markers of the calibration target captured in the point cloud further comprises:
projecting the markers into three-dimensional coordinate system of the point cloud;
determining three-dimensional coordinates for each marker in the three-dimensional coordinate system.
17 . The system of claim 11 , wherein clustering the points in the point cloud into one or more planes comprises:
performing a voxel growing approach to incrementally capture points into one cluster of points; and
identifying the one or more planes from the clusters of points.
18 . The system of claim 11 , wherein selecting one of the planes sized to match the calibration target comprises selecting the plane of largest size.
19 . The system of claim 11 , wherein identifying the holes in the selected plane as the markers of the calibration target comprises identifying the holes informed by a spatial configuration of the markers in the calibration target.
20 . The system of claim 11 , wherein determining the pose transformation comprises performing a Perspective-n-Point algorithm with the markers in the image frame and the markers in the point cloud to determine the pose transformation.