Apparatus for assisting driving of vehicle and method thereof

1 . An apparatus for assisting driving of a vehicle, the apparatus comprising:

a camera mounted on the vehicle and configured to have a front field of view of the vehicle and acquire image data;

a lidar mounted on the vehicle and configured to have a front-side field of view of the vehicle and acquire lidar data; and

a controller configured to process at least one of the image data or the lidar data,

wherein the controller comprises a calibration matrix configured to correct the image data and the lidar data and

wherein the controller is configured to:

obtain a feature map of the image data and a feature map of the lidar data using machine learning;

correct the calibration matrix based on a difference between the feature map of the image data and the feature map of the lidar data; and

correct the image data and the lidar data using the corrected calibration matrix.

2 . The apparatus of claim 1 , wherein the controller is configured to convert the lidar data into two-dimensional (2D) lidar data and convert the 2D lidar data to correspond to pixel coordinates of the image data.

3 . The apparatus of claim 2 , wherein the controller is configured to synchronize a size of the image data and a size of the 2D lidar data.

4 . The apparatus of claim 3 , wherein the controller is configured to obtain a feature map of the synchronized image data using a convolutional neural network algorithm, and obtain a feature map of the synchronized 2D lidar data using the convolutional neural network algorithm.

5 . The apparatus of claim 4 , wherein the controller is configured to obtain the feature map of the synchronized image data based on a convolution between the synchronized image data and a filter, and obtain the feature map of the synchronized 2D lidar data based on a convolution between the synchronized 2D lidar data and a filter.

6 . The apparatus of claim 4 , wherein the controller is configured to obtain a feature matrix where the feature map of the synchronized image data and the feature map of the synchronized 2D lidar data are integrated.

7 . The apparatus of claim 6 , wherein the controller is configured to obtain the feature matrix based on a difference between the feature map of the synchronized image data and the feature map of the synchronized 2D lidar data.

8 . The apparatus of claim 6 , wherein the controller is configured to obtain a correction parameter for correcting the calibration matrix from the feature matrix using the convolutional neural network algorithm.

9 . The apparatus of claim 8 , wherein the controller is configured to correct the image data and the lidar data using the calibration matrix corrected by the correction parameter.

10 . A method of assisting driving of a vehicle, the method comprising:

acquiring image data by a camera mounted on the vehicle and configured to have a front field of view of the vehicle;

acquiring lidar data by a lidar mounted on the vehicle and configured to have a front-side field of view of the vehicle;

obtaining, by a processor mounted on the vehicle, a feature map of the image data and a feature map of the lidar data using machine learning;

correcting a calibration matrix for correcting the image data and the lidar data, based on a difference between the feature map of the image data and the feature map of the lidar data; and

correcting the image data and the lidar data using the corrected calibration matrix.

11 . The method of claim 10 , wherein the obtaining of the feature map of the image data and the feature map of the lidar data comprises:

converting the lidar data into 2D lidar data; and

converting the 2D lidar data to correspond to pixel coordinates of the image data.

12 . The method of claim 11 , wherein the obtaining of the feature map of the image data and the feature map of the lidar data comprises synchronizing a size of the image data and a size of the 2D lidar data.

13 . The method of claim 12 , wherein the obtaining of the feature map of the image data and the feature map of the lidar data comprises:

obtaining a feature map of the synchronized image data using a convolutional neural network algorithm; and

obtaining a feature map of the synchronized 2D lidar data using the convolutional neural network algorithm.

14 . The method of claim 13 , wherein the obtaining of the feature map of the image data and the feature map of the lidar data comprises:

obtaining the feature map of the synchronized image data based on a convolution between the synchronized image data and a filter; and

obtaining the feature map of the synchronized 2D lidar data based on a convolution between the synchronized 2D lidar data and a filter.

15 . The method of claim 13 , wherein the obtaining of the feature map of the image data and the feature map of the lidar data comprises obtaining a feature matrix where the feature map of the synchronized image data and the feature map of the synchronized 2D lidar data are integrated.

16 . The method of claim 15 , wherein the obtaining of the feature map of the image data and the feature map of the lidar data comprises obtaining the feature matrix based on a difference between the feature map of the synchronized image data and the feature map of the synchronized 2D lidar data.

17 . The method of claim 15 , wherein the obtaining of the feature map of the image data and the feature map of the lidar data comprises obtaining a correction parameter for correcting the calibration matrix from the feature matrix using the convolutional neural network algorithm.

18 . The method of claim 17 , wherein the obtaining of the feature map of the image data and the feature map of the lidar data comprises correcting the image data and the lidar data using the calibration matrix corrected by the correction parameter.

19 . A non-transitory computer-readable recording medium storing a program for implementing the method of claim 10 .