Calibration of depth-sensing computer vision systems
Systems and methods utilize one or more 3D cameras (e.g., ToF cameras) in industrial safety applications. The 3D camera generates a depth map that may be used by external hardware and software to classify objects in a workcell and generate control signals for machinery. To facilitate sensor-specific calibration and coordination among sensors in a workcell, the sensors may store calibration data in a boot file that is loaded upon start-up. During initialization, the calibration data is loaded and, as the sensor operates, corrections are made to sensed data (e.g., pixel depth values) using the calibration data.
1. An image-processing system comprising:
at least one 3D sensor for generating an output array of pixelwise values indicative of distances to objects within a field of view thereof;
a nonvolatile memory for storing sensor-specific temperature-corrected calibration data for the at least one 3D sensor; and
at least one processor configured to:
initialize, upon power-up, the at least one sensor and load the calibration data from the nonvolatile memory;
cause the at least one 3D sensor to generate successive output arrays;
obtain temperature data substantially contemporaneously with generation of each successive output array, the temperature data being separate and distinct from the calibration data loaded from the nonvolatile memory;
process the successive output arrays into pixelwise arrays of depth values; and
correct the depth values of respective pixelwise arrays of depth values using the calibration data and the contemporaneously obtained temperature data obtained for a corresponding out array.
2. The system of claim 1 , wherein the at least one processor is further configured to recognize the objects within a field of view of the sensors.
3. The system of claim 2 , wherein the at least one processor is further configured to assess compliance with a safety metric based on distances among the recognized objects, the distances corresponding to the depth values associated with the objects.
4. The system of claim 1 , wherein the 3D sensors are time-of-flight (ToF) sensors.
5. The system of claim 1 , wherein the calibration data comprises coordinate transforms between first and second 3D sensors.
6. The system of claim 1 , further comprising at least one temperature sensor, the calibration data comprising, for each of first and second 3D sensors, a linear temperature correction factor for sensed depth.
7. The system of claim 6 , wherein the processor is configured to modify the output arrays in accordance with data from the temperature sensor and the linear temperature correction factor.
8. The system of claim 1 , wherein the calibration data comprises a focal distance, coordinates of a principal point, and radial and tangential distortion coefficients.
9. The system of claim 1 , wherein the calibration data comprises data characterizing dark noise.
10. The system of claim 1 , wherein the calibration data comprises a harmonic correction table.
11. A method of operating at least one 3D sensor configured to generate an output array of pixelwise values indicative of distances to objects within a field of view thereof, the method comprising the steps of:
storing, in a nonvolatile memory, sensor-specific temperature-corrected calibration data for the at least one 3D sensor;
initializing, upon power-up, the at least one 3D sensor and loading the calibration data from the nonvolatile memory;
generating, with the at least one 3D sensor, successive output arrays;
obtaining temperature data substantially contemporaneously with generation of each successive output sarray, the temperature data being separate and distinct from the calibration data loaded from the nonvolatile memory;
computationally processing the successive output arrays into pixelwise arrays of depth values; and
correcting the depth values of respective pixelwise arrays of depth values using the calibration data and the contemporaneously obtained temperature data obtained for a corresponding out array.
12. The method of claim 11 , further comprising the steps of computationally recognizing objects within a field of view of the sensors and computationally assessing compliance with a safety metric based on distances among the recognized objects, the distances corresponding to the depth values associated with the objects.
13. The method of claim 11 , wherein the 3D sensors are time-of-flight (ToF) sensors.
14. The method of claim 11 , wherein the calibration data comprises coordinate transforms between first and second 3D sensors.
15. The method of claim 11 , wherein the calibration data comprises, for each of the first and second 3D sensors, a linear temperature correction factor for sensed depth.
16. The method of claim 15 , wherein the output arrays are modified in accordance with data from the temperature sensor and the linear temperature correction factor.
17. The method of claim 11 , wherein the calibration data comprises a focal distance, coordinates of a principal point, and radial and tangential distortion coefficients.
18. The method of claim 11 , wherein the calibration data comprises data characterizing dark noise.
19. The method of claim 11 , wherein the calibration data comprises a harmonic correction table.