Methods, systems, and apparatus for multi-sensory stereo vision for robotics
Embodiments of the present invention include multi-sensory stereo vision sensors suitable for use in robotics, navigation, machine vision, manufacturing, and other applications. In some cases, a sensor includes a stereo camera that produces image data for use in generating disparity maps to determine the positions of objects in the scene and/or the position of the sensor itself. The stereo sensor may include image sensors that are fastened to a unitary frame to prevent undesired drift and a thermal pad that wicks heat away from a processor. The processor may provide an efficient means to directly compute stereo disparity maps onboard the sensor. A sensor can also include a laser rangefinder that provides range data suitable for calibrating the stereo camera and for improving its accuracy in certain environments. In some cases, the laser is coupled to a spindle, which in turn is driven by a geartrain through a slipping clutch.
1. A stereo vision system for an autonomous vehicle, the stereo vision system comprising:
a housing defining a cavity and at least one exterior surface;
a unitary frame within the cavity, the unitary frame defining at least one mounting surface;
a first imager, mounted to the at least one mounting surface, to acquire images of an environment from a first perspective;
a second imager, mounted to the at least one mounting surface, to acquire images of the environment from a second perspective different than the first perspective;
a collection of processing units, the collection of processing units including a processing unit disposed in the housing, wherein the collection of processing units is operable to:
obtain a three-dimensional point cloud of the environment from a LIDAR system;
generate a first disparity map based on images acquired by each of the first imager and the second imager;
calibrate each of the first imager and the second imager, using each of the three-dimensional point cloud and the first disparity map, by determining a similarity between data from the three-dimensional point cloud of the environment and the first disparity map;
generate a second disparity map based on images acquired by the calibrated first imager and the calibrated second imager; and
determine information about the environment using the images acquired by the calibrated first imager, the calibrated second imager, the second disparity map and the three-dimensional point cloud.
2. The stereo vision system of claim 1 , wherein the unitary frame consists of a single piece of material.
3. The stereo vision system of claim 1 , wherein the first imager and the second imager are disposed on a same plane.
4. The stereo vision system of claim 1 , wherein the first imager is disposed in a first plane and the second imager is disposed in a second plane parallel to the first plane.
5. The stereo vision system of claim 1 , wherein the first imager is disposed in a first plane and the second imager is disposed in a second plane that intersects the first plane.
6. The stereo vision system of claim 1 , further comprising:
at least one of at least one pin and at least one bolt fastening at least one of the first imager and the second imager to the at least one mounting surface.
7. The stereo vision system of claim 1 , further comprising:
a flexible cable coupling the second imager to the processing unit.
8. The stereo vision system of claim 1 , further comprising:
an internal wall disposed within the cavity and in thermal communication with the at least on exterior surface.
9. The stereo vision system of claim 8 , further comprising:
a thermally conductive material that is disposed within the cavity and in thermal communication with the internal wall and the processing unit, to conduct heat generated by the processing unit to the at least one exterior surface via the internal wall so as to dissipate the heat generated by the processing unit.
10. The stereo vision system of claim 1 , wherein the collection of processing units includes a field programmable gate array.
11. The stereo vision system of claim 1 , wherein the collection of processing units is further operable to:
determine a position of the autonomous vehicle.
12. The stereo vision system of claim 1 , wherein determining information about the environment includes determining a position of one or more moving objects within the environment.
13. The stereo vision system of claim 1 , further comprising:
a rectification processing block to rectify the images from the first imager and the second imager.
14. The stereo vision system of claim 1 , wherein the collection of processing units includes logic to separately trigger signals to the first imager and to the second imager so that the signals are received at the first imager and second imager at the same time.
15. The stereo vision system of claim 1 , wherein the LIDAR system includes:
a laser mounted on top of the housing to illuminate one or more objects in the environment with laser light; and
a sensor to detect laser light scattered and/or reflected by the one or more objects and to provide a range signal representative of a distance to the one or more objects based on the detected laser light, wherein the three-dimensional point cloud is based on the range signal representative of the distance to the one or more objects.