Systems and methods for object identification with a calibrated stereo-vision system
A long-baseline and long depth-range stereo vision system is provided that is suitable for use in non-rigid assemblies where relative motion between two or more cameras of the system does not degrade estimates of a depth map. The stereo vision system may include a processor that tracks camera parameters as a function of time to rectify images from the cameras even during fast and slow perturbations to camera positions. Factory calibration of the system is not needed, and manual calibration during regular operation is not needed, thus simplifying manufacturing of the system.
1. A stereo-vision system for monitoring an object, the system comprising:
a first camera sensor configured to obtain a plurality of frames of a first image of a scene and to generate first sensor signals corresponding to the frames of the first image;
a second camera sensor configured to obtain a plurality of frames of a second image of the scene and to generate second sensor signals corresponding to the frames of the second image; and
at least one computer processor configured to
calibrate the first and second camera sensors by:
determining calibration parameters for each of the first and second camera sensors, the calibration parameters including extrinsic parameters and intrinsic parameters, wherein determining the calibration parameters comprises:
using a first calibration engine to determine a first set of calibration parameters comprising a subset of the extrinsic parameters, and
using a second calibration engine to determine a second set of calibration parameters comprising the subset of extrinsic parameters and at least one additional calibration parameter from among the extrinsic parameters and the intrinsic parameters; and
updating camera parameters for the first and second camera sensors using the determined calibration parameters.
2. The system of claim 1 , wherein the first and second camera sensors are separated by a baseline in a range of 1 m to 200 m.
3. The system of claim 1 , wherein:
the first and second images are color images, and
the at least one computer processor is further configured to identify the object in the first and second images by identifying a color of at least one characteristic of the object.
4. The system of claim 1 , wherein the scene comprises a scene of a construction site.
5. The system of claim 1 , wherein
the first calibration engine is configured to determine the first set of calibration parameters every first number of frames of the first and second sensor signals, and
the second calibration engine is configured to determine the second set of calibration parameters every second number of frames of the first and second sensor signals, the second number being different from the first number.
6. The system of claim 1 , wherein the first camera sensor and the second camera sensor are untethered from each other.
7. The system of claim 1 , wherein the first camera sensor and the second camera sensor are mounted on a non-rigid structure that is able to flex from environmental conditions.
8. The system of claim 1 , wherein the first camera sensor and the second camera sensor are not attached to a common rigid mounting member.
9. The system of claim 1 , wherein using the first calibration engine to determine the first set of calibration parameters comprises using the first calibration engine to determine a roll calibration parameter and a pitch calibration parameter.
10. The system of claim 1 , wherein the at least one additional calibration parameter from among the extrinsic parameters and the intrinsic parameters comprises a translation vector.
11. The system of claim 1 , wherein using the second calibration engine to determine the second set of calibration parameters comprises using the second calibration engine to determine each of the extrinsic parameters and each of the intrinsic parameters.
12. A computer-implemented calibration method performed by at least one computer processor to provide a stereo-vision system for monitoring an object, the method comprising:
receiving from a first camera sensor first sensor signals corresponding to a plurality of frames of a first image of a scene;
receiving from a second camera sensor second sensor signals corresponding to a plurality of frames of a second image of the scene; and
calibrating the first and second camera sensors by:
determining calibration parameters for each of the first and second camera sensors, the calibration parameters including extrinsic parameters and intrinsic parameters, wherein determining the calibration parameters comprises:
using a first calibration engine to determine a first set of calibration parameters comprising a subset of the extrinsic parameters, and
using a second calibration engine to determine a second set of calibration parameters comprising the subset of extrinsic parameters and at least one additional calibration parameter from among the extrinsic parameters and the intrinsic parameters; and
updating camera parameters for the first and second camera sensors using the determined calibration parameters.
13. The method of claim 12 , wherein the first and second camera sensors are separated by a baseline in a range of 1 m to 200 m.
14. The method of claim 12 , wherein the first and second camera sensors are separated by a baseline in a range of 5 m to 50 m.
15. The method of claim 12 , further comprising:
identifying the object in the first and second images by identifying a color of at least one characteristic of the object,
wherein the first and second images are color images.
16. The method of claim 12 , wherein the scene comprises a scene of a construction site.
17. The method of claim 12 , further comprising:
tracking a series of estimates of the calibration parameters over time; and
filtering the series of estimates to determine the estimate.
18. The method of claim 12 , wherein
the first calibration engine is configured to determine the first set of calibration parameters every first number of frames of the first and second sensor signals, and
the second calibration engine is configured to determine the second set of calibration parameters every second number of frames of the first and second sensor signals, the second number being different from the first number.
19. A non-transitory computer-readable storage medium storing computer-readable code that, when executed by at least one computer processor, performs a method to provide a stereo-vision system for monitoring an object, wherein the method comprises:
receiving from a first camera sensor first sensor signals corresponding to a plurality of frames of a first image of a scene;
receiving from a second camera sensor second sensor signals corresponding to a plurality of frames of a second image of the scene;
calibrating the first and second camera sensors by:
determining calibration parameters for each of the first and second camera sensors, the calibration parameters comprising extrinsic parameters and intrinsic parameters, wherein determining the calibration parameters comprises:
using a first calibration engine to determine a first set of calibration parameters comprising a subset of the extrinsic parameters, and
using a second calibration engine to determine a second set of calibration parameters comprising the subset of extrinsic parameters and at least one additional calibration parameter from among the extrinsic parameters and the intrinsic parameters; and
updating camera parameters for the first and second camera sensors using the determined calibration parameters.