Method and system for multiple stereo based depth estimation and collision warning/avoidance utilizing the same
The present teaching relates to method, system, medium, and implementation of determining depth information in autonomous driving. Stereo images are first obtained from multiple stereo pairs selected from at least two stereo pairs. The at least two stereo pairs have stereo cameras installed with the same baseline and in the same vertical plane. Left images from the multiple stereo pairs are fused to generate a fused left image and right images from the multiple stereo pairs are fused to generate a fused right image. Disparity is then estimated based on the fused left and right images and depth information can be computed based on the stereo images and the disparity.
1. A method implemented on a computer having at least one processor, a storage, and a communication platform for determining depth information in autonomous driving, comprising:
obtaining stereo images from multiple stereo pairs selected from at least two stereo pairs, wherein the at least two stereo pairs have stereo cameras installed with a same baseline and in a same vertical plane, wherein one of the at least two stereo pairs is configured to acquire stereo images of a scene with a wider field of view and remaining of the at least two stereo pairs is configured to acquire stereo images of the scene with a narrow field of view, wherein the obtaining further comprises:
receiving at least one image from the stereo pair capturing the scene with the wider field of view,
detecting one or more relevant objects from the at least one image, and
selecting the multiple stereo pairs from the at least two stereo pairs based on the one or more relevant objects, wherein each of the multiple stereo pairs with a narrow field of view is configured to capture a region in the at least one image where the object is observed;
fusing left images from the multiple stereo pairs to generate a fused left image;
fusing right images from the multiple stereo pairs to generate a fused right image;
estimating disparity based on the fused left and right images; and
computing depth information of a scene captured by the stereo images based on the stereo images and the disparity.
2. The method of claim 1 , wherein the stereo images acquired by the stereo pair with a wider field of view capture the scene and in a coarser resolution.
3. The method of claim 1 , wherein each of the remaining of the at least two stereo pairs is configured to acquire stereo images in a higher resolution, each of the stereo images with a narrow field of view capturing a part of the scene.
4. The method of claim 1 , further comprising at least one of:
determining an avoidance strategy based on the depth information; and
generating a collision warning signal in accordance with the avoidance strategy.
5. A machine readable and non-transitory medium having information recorded thereon for determining depth information in autonomous driving, wherein the information, when read by the machine, causes the machine to perform:
obtaining stereo images from multiple stereo pairs selected from at least two stereo pairs, wherein the at least two stereo pairs have stereo cameras installed with a same baseline and in a same vertical plane, wherein one of the at least two stereo pairs is configured to acquire stereo images of a scene with a wider field of view and remaining of the at least two stereo pairs is configured to acquire stereo images of the scene with a narrow field of view, wherein the obtaining further comprises:
receiving at least one image from the stereo pair capturing the scene with the wider field of view,
detecting one or more relevant objects from the at least one image, and
selecting the multiple stereo pairs from the at least two stereo pairs based on the one or more relevant objects, wherein each of the multiple stereo pairs with a narrow field of view is configured to capture a region in the at least one image where the object is observed;
fusing left images from the multiple stereo pairs to generate a fused left image;
fusing right images from the multiple stereo pairs to generate a fused right image;
estimating disparity based on the fused left and right images; and
computing depth information of a scene captured by the stereo images based on the stereo images and the disparity.
6. The medium of claim 5 , wherein the stereo images acquired by the stereo pair with a wider field of view capture the scene and in a coarser resolution.
7. The medium of claim 5 , wherein each of the remaining of the at least two stereo pairs is configured to acquire stereo images in a higher resolution, each of the stereo images with a narrow field of view capturing a part of the scene.
8. The medium of claim 5 , wherein, the information, when read by the machine, further causes the machine to perform:
determining an avoidance strategy based on the depth information; and
generating a collision warning signal in accordance with the avoidance strategy.
9. A system for determining depth information in autonomous driving, comprising:
multiple stereo pairs configured for obtaining stereo images, wherein the multiple stereo pairs are selected from at least two stereo pairs, which have stereo cameras installed with a same baseline and in a same vertical plane, wherein one of the at least two stereo pairs is configured to acquire stereo images of a scene with a wider field of view and remaining of the at least two stereo pairs is configured to acquire stereo images of the scene with a narrow field of view:
an object recognition unit configured for:
receiving at least one image from the stereo pair capturing the scene with the wider field of view,
detecting one or more relevant objects from the at least one image, and
selecting the multiple stereo pairs from the at least two stereo pairs based on the one or more relevant objects, wherein each of the multiple stereo pairs with a narrow field of view is configured to capture a region in the at least one image where the object is observed;
a left image fusion unit configured for fusing left images from the multiple stereo pairs to generate a fused left image;
a right image fusion unit configured for fusing right images from the multiple stereo pairs to generate a fused right image;
a stereo based disparity estimator configured estimating disparity based on the fused left and right images; and
a multi-stereo based depth estimator configured for computing depth information of a scene captured by the stereo images based on the stereo images and the disparity.
10. The system of claim 9 , wherein the stereo images acquired by the stereo pair with a wider field of view capture the scene and in a coarser resolution.
11. The system of claim 9 , wherein each of the remaining of the at least two stereo pairs is configured to acquire stereo images in a higher resolution, each of the stereo images with a narrow field of view capturing a part of the scene.
12. The system of claim 9 , further comprising an adaptive image acquisition unit configured for determining the multiple stereo pairs for receiving left and right stereo images from each of the multiple stereo pairs.
13. The system of claim 9 , wherein the depth information determined based on the stereo images from the at least some stereo pairs is used for at least one of:
determining an avoidance strategy based on the depth information; and
generating a collision warning signal in accordance with the avoidance strategy.