System and method for field calibrating video and lidar subsystems using facial features
View Patent ↗A system uses range and Doppler velocity measurements from a lidar subsystem and images from a video subsystem to estimate a six degree-of-freedom trajectory of a target. The video subsystem and the lidar subsystem may be aligned with one another by mapping the measurements of various facial features obtained by each of the subsystems to one another.
1. A system for calibrating a lidar subsystem, the system comprising:
a lidar subsystem configured to:
direct at least two beams toward a target, wherein the at least two beams comprises a first beam directed toward the target via a first scanning mirror and a second beam directed toward the target via a second scanning mirror,
generate a first set of three-dimensional measurements for a plurality of points on the target for the first beam, and
generate a second set of three-dimensional measurements for a plurality of points on the target for the second beam; and
a processor configured to:
receive, from the lidar subsystem, the first set of three-dimensional measurements and the second set of three-dimensional measurements,
compare the first set of three-dimensional measurements with the second set of three-dimensional measurements;
determine one or more offsets in locations of a plurality of facial features between the first set of three-dimensional measurements and the second set of three-dimensional measurements based on the comparison; and
determine one or more calibration adjustments to be made between the first scanning mirror and the second scanning mirror based on the one or more offsets.
2. The system of claim 1 , wherein the one or more offsets are determined at a plurality of scanning positions.
3. The system of claim 1 , wherein the processor is further configured to:
adjust one or more transformation parameters that transform a coordinate system of the first set of three-dimensional measurements to a coordinate system of the second set of three-dimensional measurements based on the one or more calibration adjustments.
4. A system for calibrating a lidar subsystem and a video subsystem, the system comprising:
a lidar subsystem configured to direct at least two beams toward a target and that generates a set of three-dimensional measurements for a plurality of points on the target for each of the at least two beams;
a video subsystem configured to provide at least one image of the target;
and
a processor configured to:
receive, from the lidar subsystem, the set of three-dimensional measurements,
receive, from the video subsystem, the at least one image of the target,
determine locations of a plurality of symmetrical facial features based on the at least one image of the target and a 3D symmetry place, wherein the 3D symmetry plane is determined based on the set of three-dimensional measurements,
determine a noseridge relative to the 3D symmetry plane,
determine a horizontal offset between the locations of the plurality of symmetrical facial features and the 3D symmetry plane,
correct the horizontal offset by a shift between the 3D horizontal offset and the noseridge,
determine a first location of lip line based on the at least one image of the target;
determine a second location of the lip line based on the set of three-dimensional measurements,
determine a vertical offset based on the first location and the second location, and
determine one or more calibration adjustments based on the corrected horizontal offset and the vertical offset.
5. The system of claim 4 , wherein the plurality of symmetrical facial features comprises eyes and nostrils.
6. A method for calibrating a lidar subsystem, the method comprising:
directing at least two beams of the lidar subsystem toward a target, wherein the at least two beams comprise a first beam directed toward the target by a first scanning mirror and a second beam directed toward the target by a second scanning mirror;
receiving, by a processor, a first set of three-dimensional measurements for a plurality of points on the target, wherein the first set of three-dimensional measurements is generated by the lidar subsystem from the first beam;
receiving, by the processor, a second set of three-dimensional measurements for a plurality of points on the target, wherein the second set of three-dimensional measurements is generated by the lidar subsystem from the second beam;
comparing, by the processor, the first set of three-dimensional measurements with the second set of three-dimensional measurements;
determining, by the processor, one or more offsets in locations of a plurality of facial features between the first set of three-dimensional measurements and the second set of three-dimensional measurements based on the comparing;
determining, by the processor, one or more calibration adjustments to be made between the first scanning mirror and the second scanning mirror based on the one or more offsets.