Driving assistant system, electronic device, and operation method thereof
Systems and method for automated driving assistance are described. A driving assistant system may include a first sensor unit configured to provide an input image including another vehicle and a background; an image processor configured to generate a plurality of pyramid images by down-sampling the input image, acquire a depthmap including depth values from the first sensor unit to the other vehicle and the background, generate a plurality of pieces of mask data with different average depths of the depth values based on the depthmap, and output a plurality of masked images representing different average distances from the first sensor unit based on the plurality of pieces of mask data and the plurality of pyramid images; a feature extractor configured to output feature data of each of the plurality of masked images; and a detector configured to detect the other vehicle included in the input image based on the feature data.
1. A driving assistant system comprising:
a first sensor unit configured to provide an input image including another vehicle and a background;
an image processor configured to generate a plurality of pyramid images by down-sampling the input image, identify a depthmap including depth values to the other vehicle and the background, generate a plurality of pieces of mask data having different average depths of the depth values based on the depthmap, and output a plurality of masked images representing different average distances based on the plurality of pieces of mask data and the plurality of pyramid images;
a feature extractor configured to output feature data of each of the plurality of masked images; and
a detector configured to detect the other vehicle included in the input image based on the feature data.
2. The driving assistant system of claim 1 , wherein
the first sensor unit comprises a stereo camera,
the input image comprises a left-eye image and a right-eye image, and
the image processor is further configured to acquire the depthmap based on a comparison of the left-eye image and the right-eye image.
3. The driving assistant system of claim 2 , wherein
the image processor is further configured to generate a region of interest (RoI) based on the input image and mask a region remaining by excluding the Ro from the plurality of pyramid images.
4. The driving assistant system of claim 1 , wherein
the plurality of pyramid images comprises a first pyramid image and a second pyramid image, and
the image processor is further configured to generate the first pyramid image by down-sampling the input image and to generate the second pyramid image by down-sampling the first pyramid image.
5. The driving assistant system of claim 4 , wherein
the plurality of pieces of mask data comprises first mask data and second mask data having a smaller average depth than the first mask data,
the plurality of masked images comprises a first masked image and a second masked image, and
the image processor is further configured to resize the first mask data and the second mask data so as to correspond to the first pyramid image and the second pyramid image, output the first masked image in which the first mask data is applied to the first pyramid image, and output the second masked image in which the second mask data is applied to the second pyramid image.
6. The driving assistant system of claim 1 , wherein
the image processor is further configured to mask a region remaining by excluding at least one of the other vehicle, a road and a pedestrian from the input image and generate the plurality of pyramid images based on a masked image.
7. The driving assistant system of claim 1 , wherein
the plurality of masked images comprises a first masked image and a second masked image,
the feature extractor is further configured to output first feature data generated based on the first masked image and second feature data generated based on the second masked image, and
the detector is further configured to receive concatenated data generated by concatenating the first feature data and the second feature data.
8. The driving assistant system of claim 7 , further comprising:
a down-sampler configured to down-sample the second feature data and output down-sampled feature data; and
a buffer configured to output the concatenated data by concatenating the first feature data and the down-sampled feature data.
9. The driving assistant system of claim 1 , further comprising:
a second sensor unit comprising a first distance sensor and a second distance sensor having a shorter sensing distance than the first distance sensor,
the detector further comprising a first sub-detector and a second sub-detector;
a first sensor fusing unit configured to fuse first detection data output from the first sub-detector and first sensing data output from the first distance sensor and output first fusing data; and
a second sensor fusing unit configured to fuse second detection data output from the second sub-detector and second sensing data output from the second distance sensor and output second fusing data,
wherein the first detection data is generated based on an image including an object of a farther distance than that of the second detection data.
10. The driving assistant system of claim 9 , further comprising
a third distance sensor having a shorter sensing distance than the second distance sensor,
wherein the third distance sensor outputs third sensing data, and
the second sensor fusing unit is further configured to fuse the second detection data, the second sensing data, and the third sensing data.
11. The driving assistant system of claim 9 , further comprising a merging unit configured to acquire three-dimensional information of the other vehicle based on the first fusing data and the second fusing data.