Stereo vision for sensing vehicles operating environment

1. A vehicle, comprising:

a chassis;

a drive system carrying the chassis; and

a vision system carried by said chassis, said vision system including:

a stereo visible light camera producing a colorized 3D data point cloud;

a stereo long wave infrared (LWIR) camera producing 3D LWIR data; and

a near infrared (NIR) camera, wherein both said LWIR camera and said NIR camera produce data that is fused with said 3D point cloud,

wherein said fusing includes detecting foliage by way of measuring and comparing NIR and red spectrum energy level ratios,

wherein the LWIR camera data is used to fill in information in areas of said 3D point cloud including areas where there is low light at night or dust which obscures a field of vision of said stereo visible light camera, and wherein sparse computations are used to determine sparse points based on edge features, and wherein a segmentation computation and a hole-filling computation are used to fill in regions, including depth regions, between sparse points to fuse with the 3D point cloud;

wherein said vision system is configured to fuse said 3D data, detected data, and filled-in data with said 3D point cloud to produce an enhanced 3D point cloud.

2. The vehicle of claim 1 , wherein said drive system is directed to at least one of steer, change velocity, start and stop the vehicle, dependent upon said enhanced 3D point cloud.

3. The vehicle of claim 1 , wherein said vision system is further configured to extract features from the 3D point cloud and the 3D data as part of producing said enhanced 3D point cloud.

4. The vehicle of claim 3 , wherein said vision system is further configured to match at least some of the extracted features from the 3D point cloud and the 3D data as part of producing said enhanced 3D point cloud.

5. The vehicle of claim 4 , wherein said vision system is further configured to perform a consistency validation of the 3D data that is matched.

6. A vision system for use by a vehicle having a drive system, the vision system comprising:

a stereo visible light camera producing a colorized 3D point cloud;

a stereo long wave infrared (LWIR) camera producing 3D LWIR data; and

a near infrared (NIR) camera, wherein both said stereo LWIR camera and said NIR camera produce data that is fused with said 3D point cloud;

wherein said fusing includes detecting foliage by way of measuring and comparing NIR and red spectrum energy level ratios,

wherein LWIR data is used to fill in information in areas of said 3D point including areas where there is low light at night or dust which obscures a field of vision of said stereo visible light camera, and wherein sparse computations are used to determine sparse points based on edge features, and wherein a segmentation computation and a hole-filling computation are used to fill in regions, including depth regions, between sparse points to fuse with the 3D point cloud,

wherein said vision system is configured to fuse said 3D LWIR data, detected data, and filled-in data with said 3D point cloud to produce an enhanced 3D point cloud.

7. The vision system of claim 6 , wherein the drive system is directed to at least one of steer, change velocity, start and stop the vehicle, dependent upon said enhanced 3D point cloud.

8. The vision system of claim 6 , wherein said vision system is further configured to extract features from the 3D point cloud and the 3D data as part of producing said enhanced 3D point cloud.

9. The vision system of claim 8 , wherein said vision system is further configured to match at least some of the extracted features from the 3D point cloud and the 3D data as part of producing said enhanced 3D point cloud.

10. The vision system of claim 9 , wherein said vision system is further configured to perform a consistency validation of the 3D data that is matched.

11. A method of directing a vehicle using a vision system, the method comprising the steps of:

producing a colorized 3D point cloud with data from a stereo visible light camera;

fusing data from a stereo long wave infrared (LWIR) camera with said 3D point cloud;

fusing data from a near infrared (NIR) camera with said 3D point cloud;

detecting foliage via fusing by way of measuring and comparing NIR and red spectrum energy level ratios,

filling in information via the LWIR camera data in areas of said 3D point cloud including areas where there is low light at night or dust which obscures a field of vision of said stereo visible light camera;

filling in regions, including depth regions via a segmentation computation and a hole-filling computation between sparse points to fuse with the 3D point cloud; and

producing from the fused, detected, and filled-in 3D data an enhanced 3D point cloud that is used to direct tasks of the vehicle.

12. The method of claim 11 , further comprising the step of directing a drive system of the vehicle to at least one of steer, change velocity, start and stop the vehicle, dependent upon said enhanced 3D point cloud.

13. The method of claim 11 , wherein said vision system is further configured to extract features from the 3D point cloud and the 3D data as part of producing said enhanced 3D point cloud.

14. The method of claim 13 , wherein said vision system is further configured to match at least some of the extracted features from the 3D point cloud and the 3D data as part of producing said enhanced 3D point cloud.

15. The method of claim 14 , wherein said vision system is further configured to perform a consistency validation of the 3D data that is matched.