Convolutional neural network based on channel-specific convolution for high-speed object recognition of 3D LiDAR

1 . A convolutional neural network structure based on channel-specific convolution for high-speed object recognition of a 3D LiDAR, the 3D LiDAR providing a point cloud of LiDAR data in a plurality of channels, the convolutional neural network structure comprising:

a first convolutional network for extracting features in the channels of the LiDAR data, wherein the LiDAR data is represented by L=[C 1 , C 2 . . . C i ] T , where C i =[p i,1 p i,2 . . . p i,j ] and p i,j ={x i,j , y i,j , z i,j , I i,j }, wherein L denotes the entire LiDAR data, i denotes a channel, C i denotes a set of points belonging to channel i, j denotes an index in each channel, and p i,j denotes one point, and each p i,j contains a location (x, y, z) value and a reflectivity (1) value, wherein the first convolutional network includes twelve convolutional layers and four pooling layers for each channel (i) of the plurality of channels, each convolutional layer of the first convolutional network includes a convolution mask having a size of 1×3, convolution is performed on the location (x, y, z) and reflectivity (I) values of the LiDAR data for each channel, three convolutions layers of the twelve convolutional layers and one pooling layer of the four pooling layers are stacked in each set, and a total of four sets of convolution is performed, the first convolutional network configured to,

receive the point cloud of LiDAR data in the plurality of channels from the 3D LiDAR, each channel of the plurality of channels including a set of points, each point of the set of points containing a portion of the LiDAR data, and

separately extract, from the point cloud of LiDAR, first feature data in each individual channel of the plurality of channels, wherein the first feature data in each individual channel (i) of the plurality of channels is represented by F j :

a second convolutional network different than the first convolutional network and connected to the first convolutional network, wherein the second convolutional network includes three convolutional layers and each convolutional layer of the second convolutional network includes a convolution mask having a size of 3×3, the second convolutional network configured to,

receive a combined feature set (F) corresponding to a combination of the first feature data ( F j ) extracted in each channel of the plurality of channels,

extract second feature data from two or more channels of the plurality of channels based on the combined feature set, and

generate a feature map (F″) based on the extracted second feature data from the two or more channels of the plurality of channels from the second convolutional network and the received combined feature set corresponding to the combination of the first feature data extracted in each channel of the plurality of channels from the first convolutional network; and

a detection network connected to the second convolutional network, wherein the detection network includes four convolution layers, a class layer and a box layer for displaying a location of the object, and each convolutional layer of the detection network includes a convolution mask having a size of 3×3, the detection network configured to,

receive the feature map generated from the second convolutional network, and

search location and class of an object by using the feature map generated from the second convolutional network,

wherein the class layer is configured to output a class and score of an object by using a result data coming out through the four convolution layers and the box layer is configured to display a location of the object, a size of a box, and an angle of the object by using the result data coming out through the four convolution layers.