Auto-calibrating a target sensor using scene mapping information from a reference sensor
Systems and methods for performing auto-calibration for a target sensor from information gained by a reference sensor, thereby enabling cross-sensor self-calibration for a set of sensors. A method includes the step of obtaining a first mapping of a first observable scene from a first scene sensing device and a second mapping of a second observable scene from a second scene sensing device. The first and second scene sensing devices are positioned on a test object (e.g., vehicle) in a substantially fixed relationship with respect to each other. The first and second observable scenes have common reference objects and/or overlapping portions. The method also includes comparing the common reference object and/or overlapping portion of the first mapping with the common reference object and/or overlapping portion of the second mapping to determine a difference. Also, the method includes creating calibration parameters, based on the difference, for calibrating the second scene sensing device.
1 . A calibration system, comprising:
a first scene sensing device;
a second scene sensing device, wherein, the first scene sensing device and the second scene sensing device are positioned on a vehicle in a fixed relationship with respect to each other;
a processing device; and
a memory device configured to store a computer program having instructions that, when executed, enable the processing device to:
obtain, by the first scene sensing device, a first mapping of a first observable scene that includes a target vehicle that passes the vehicle,
obtain, by the second scene sensing device, a second mapping of a second observable scene that includes the target vehicle,
determine a difference between the first mapping and the second mapping based on a comparison of the target vehicle in the first mapping and in the second mapping,
based on the determined difference, create calibration parameters for calibrating the second scene sensing device, and
automatically calibrate the second scene sensing device using the created calibration parameters.
2 . The calibration system of claim 1 , wherein the target vehicle is in the first observable scene and in the second observable scene.
3 . The calibration system of claim 1 , wherein the first and second scene sensing devices are configured to observe scenes nearby the vehicle.
4 . The calibration system of claim 3 , wherein the second scene sensing device is calibrated while the vehicle is in operation.
5 . The calibration system of claim 3 , wherein the first scene sensing device is directed to view a portion of a road on which the vehicle is traveling, and wherein the instructions further enable the processing device to use characteristics of the road to perform a self-calibration procedure.
6 . The calibration system of claim 1 , wherein the first and second mappings are observed by the first and second scene sensing devices one of
at substantially the same time, and
at substantially different times.
7 . The calibration system of claim 1 , wherein the first scene sensing device is configured to create a feature mapping of the first observable scene having one or more observable objects, wherein the instructions further enable the processing device to localize the observable objects in three-dimensional (3D) space and predict the locality of the one or more observable objects in the 3D space to determine the difference.
8 . The calibration system of claim 1 , wherein creating the calibration parameters comprises producing roll and pitch calibration parameters.
9 . The calibration system of claim 1 , wherein the first scene sensing device is a pre-calibrated sensor used as a reference sensor and the second scene sensing device is a target sensor.
10 . The calibration system of claim 9 , further comprising one or more additional scene sensing devices, wherein the second scene sensing device is used as a reference sensor for the one or more additional scene sensing devices.
11 . The calibration system of claim 1 , wherein the second scene sensing device is calibrated with respect to an initial factory-calibrated position and orientation.
12 . The calibration system of claim 1 , wherein the first and second scene sensing devices comprise Advanced Driver-Assistance System (ADAS) and/or Autonomous Driving (AD) System sensors, and wherein the ADAS and/or AD sensors comprise one or more of perception sensors, cameras, image capture devices, lidar devices, and radar devices.
13 . The calibration system of claim 1 , wherein the first and second mappings comprise observed scene/material reflective property maps.
14 . A method, comprising:
obtaining, from a first scene sensing device, a first mapping of a first observable scene that includes a first vehicle that passes a second vehicle;
obtaining, from a second scene sensing device, a second mapping of a second observable scene that includes the first vehicle passing the second vehicle, wherein the first and second observable scenes include a reference object;
generating, based on the first observable scene obtained by the first mapping, a first video of the first vehicle;
generating, based on the second observable scene obtained by the second mapping, a second video of the first vehicle;
comparing the first vehicle in the first video and in the second video to determine a difference;
based on the determined difference, creating calibration parameters for calibrating the second scene sensing device;
automatically calibrating the second scene sensing device using the created calibration parameters; and
in response to determining that the second scene sensing device differs from an initial setting by more than a threshold, providing an alert signal indicating that the second scene sensing device has been displaced from the initial setting.
15 . The method of claim 14 , wherein the reference object is in the first observable scene and in the second observable scene.
16 . The method of claim 14 , wherein the second scene sensing device is automatically calibrated while the second vehicle is in operation.
17 . The method of claim 14 , wherein the first mapping and the second mapping are observed by the first and second scene sensing devices at one of
at substantially the same time, and
at substantially different times.
18 . The method of claim 14 , wherein the first scene sensing device is configured to create a feature mapping of the scene having one or more observable objects and the method further comprises:
enabling localization of the one or more observable objects in three-dimensional (3D) space; and
predicting, based on the localization, locality of the one or more observable objects in the 3D space to determine the difference.
19 . The method of claim 14 , wherein creating the calibration parameters comprises producing roll and pitch calibration parameters.
20 . The method of claim 14 , wherein the first and second scene sensing devices comprise one or more of perception sensors, cameras, image capture devices, lidar devices, and radar devices used in one or more Advanced Driver-Assistance Systems (ADAS) and/or Autonomous Driving (AD) Systems.
21 . A calibration system, comprising:
a first scene sensing device;
a second scene sensing device;
a rear-facing target sensor;
a processing device; and
a memory device configured to store a computer program having instructions that, when executed, enable the processing device to in response to vehicles detected by rear-facing target sensor:
obtain, by the first scene sensing device, a first mapping of a first observable scene that includes a first passing vehicle passing a reference vehicle on a first side of the reference vehicle,
obtain, by the second scene sensing device, a second mapping of a second observable scene that includes a second passing vehicle passing the reference vehicle on a second side of the reference vehicle, the second side opposite the first side,
determine a difference between the first mapping and the second mapping based on a comparison of the first passing vehicle in the first mapping and the second passing vehicle in the second mapping,
based on the determined difference, create roll and pitch calibration parameters for calibrating the second scene sensing device, and
automatically calibrate the second scene sensing device using the created roll and pitch calibration parameters.