System and method for calibrating video and lidar subsystems
View Patent ↗A system uses range and Doppler velocity measurements from a lidar system and images from a video system to estimate a six degree-of-freedom trajectory of a target. The system calibrates the lidar subsystem with the video subsystem in two stages. In a first stage, the system calibrates the lidar subsystem so that lidar measurements provide 3D lidar coordinates. In a second stage, the system relates the 3D lidar coordinates with a video image obtained from the video subsystem.
1. A system for calibrating a lidar subsystem and a video subsystem, the system comprising:
a lidar subsystem configured to direct a first beam toward a reference grid target and generate a lidar measurement for each of a plurality of points on the reference grid target for the first beam, wherein the reference grid target comprises a plurality of highly reflective tapes along each edge of the reference grid target and a plurality of video targets interspersed throughout a surface of the reference grid target;
a video subsystem configured to provide a plurality of two-dimensional images of the reference grid target; and
a processor configured to:
receive, from the lidar subsystem, the lidar measurement for each of the plurality of points on the reference grid target,
receive, from the video subsystem, the plurality of images of the reference grid target,
determine a lidar measurement for each of the plurality of highly reflective tapes along each edge of the reference grid target from among the received lidar measurement for each of the plurality of points on the reference grid target;
determine 3D coordinates of each of the plurality of video targets based at least on the lidar measurement for each of the plurality of highly reflective tapes along each edge of the reference grid target; and
relate the 3D coordinates of each of the plurality of video targets with pixel coordinates of each of the plurality of video targets, wherein the pixel coordinates are obtained from an image of the reference grid target.
2. The system of claim 1 , wherein the lidar measurement for each of the plurality of points comprises 3D lidar coordinates corresponding to the lidar measurement of range and Doppler velocity at a particular azimuth, elevation, and a time at which the measurement was taken.
3. The system of claim 1 , wherein the lidar subsystem is configured to generate a first lidar measurement for each of the plurality of points on the reference grid target at a first range, wherein the video subsystem is configured to provide a first image of the reference grid target at the first range, and wherein the processor is further configured to:
determine a first lidar measurement for each of the plurality of highly reflective tapes along each edge of the reference grid target at the first range;
determine, at the first range, first 3D coordinates of each of the plurality of video targets based at least on the first lidar measurement for each of the plurality of highly reflective tapes; and
relates the first 3D coordinates of each of the plurality of video targets with pixel coordinates of the plurality of video targets, wherein the pixel coordinates are obtained from the first image.
4. The system of claim 3 , wherein the lidar subsystem is configured to generate a second lidar measurement for each of the plurality of points on the reference grid target at a second range, wherein the video subsystem is configured to provide a second image of the reference grid target at the second range, and wherein the processor is further configured to:
determine a second lidar measurement for each of the plurality of highly reflective tapes along each edge of the reference grid target at the second range;
determines, at the second range, second 3D coordinates of each of the plurality of video targets based at least on the second lidar measurement for each of the plurality of highly reflective tapes; and
relates the second 3D coordinates of each of the plurality of video targets with pixel coordinates of the plurality of video targets, wherein the pixel coordinates are obtained from the second image.