Active perception system for double-axle steering cab-less mining vehicle

1 . An active perception system for a double-axle steering cab-less mining vehicle, the system comprising: at least one processor, a memory storing instructions, when executed by the at least one processor, causing the at least one processor to perform operations comprising:

building a three-dimensional (3D) perception map and extracting a region of interest (ROI) by marking an unstructured road boundary;

detecting single-frame data by Euclidean clustering-based millimeter wave radar detection, depth learning algorithm-based Lidar detection and depth learning algorithm-based camera detection, comprising:

receiving point cloud information of a ROI and speed, inclination and steering angle of the vehicle, and obtaining data output by a millimeter wave radar, a Lidar and a camera;

determining a rotation angle of each sensor according to the steering angle information of the vehicle combined with the point cloud information of the ROI and controlling the rotation angle of each sensor, and determining the rotation angle of each sensor according to an output value of a slope sensor when a vehicle body is tilted or a center of gravity shifts and compensating a visual field blind area caused by a pitch angle of the vehicle;

performing Euclidean clustering on a point cloud on a road after ground information is removed from the ROI,

wherein performing Euclidean clustering on the point cloud on the road comprises:

randomly selecting target points, and

when a distance between the target points is less than a threshold, putting the target points into a cluster until there are no points with a distance less than the threshold, and outputting a clustering result;

cutting an original point cloud into grids from a perspective of a top view, pillarizing a 3D point cloud, representing a point cloud in each Pillar by a 9-dimensional vector, inputting the point cloud into a PointNet feature extraction network to generate a two-dimensional (2D) pseudo-image, inputting the pseudo-image into a 2D convolutional neural network (CNN) to further extract point cloud features, performing BoundingBox regression through a single-shot detector (SSD) head and outputting a position and orientation of the obstacle; and

enhancing an image using Mosaic data, changing a size and position of a truth value target, inputting the enhanced image into the network, unifying the size of the image, extracting deep semantic information and shallow spatial information of the image through a residual network, and detecting a category and position of the target;

predicting a risk index of each detection target by calculating an information entropy of each sensor and acquiring indicators comprising a mark, speed and distance of the target, and determining a state of an obstacle after the risk index feeds back on the information entropy;

receiving perceived environmental data;

determining a new speed, inclination and steering angle of the vehicle through a corresponding planning algorithm for vehicle control, and feeding back the new speed, inclination and steering angle of the vehicle to control a rotation direction of each sensor adaptively, so as to increase an effective perception and detection range; and

actively adjusting the vehicle to a forward advancing mode and a backward advancing mode according to a driving mode and comprehensive information of each sensor.

2 . The active perception system according to claim 1 , wherein the double-axle steering autonomous driving mining vehicle has the forward advancing mode and the backward advancing mode: in the forward advancing mode, a sensor at the front of a vehicle body is configured as a main sensor, and the forward advancing mode is started when the vehicle moves from a loading area to an unloading area under a full load; and in the backward advancing mode, a sensor at the rear of the vehicle body is configured as a main sensor, and the backward advancing mode is started when the vehicle moves from the unloading area to the loading area under a no-load state.

3 . The active perception system according to claim 1 , comprising a forward sensor component and a backward sensor component with the same configuration, wherein each of the sensor components comprises at least a slope sensor, a global navigation satellite system/inertial measurement unit (GNSS/IMU), a Lidar, a camera, a millimeter wave radar, and a rotation angle receiving module and rotation angle control module of each sensor.

4 . The active perception system according to claim 1 , wherein

two sets of Lidars, millimeter wave radars and cameras are arranged at the front and rear of a double-axle steering autonomous driving mining vehicle body, and a GNSS/IMU and a slope sensor are arranged on the vehicle body, and wherein, building the three-dimensional (3D) perception map and extracting the region of interest (ROI) by marking the unstructured road boundary comprises:

calibrating the Lidar and the GNSS/IMU to obtain the positioning relationship between a vehicle body coordinate system and a world coordinate system;

establishing a local world map with a global positioning system (GPS) of a first waypoint as an origin,

determining a conversion matrix by Lidar and vehicle GNSS/IMU calibration information, converting GPS information of a point cloud at every moment into the world coordinate system, and performing front-end iterative closest point (ICP) matching and back-end extended Kalman filter (EKF) optimization processing on the point cloud to form a local point cloud map; and

extracting the unstructured road boundary based on a point cloud normal vector, distinguishing a vertical ground normal vector and an inclined retaining wall normal vector, and extracting a driving boundary through orientation features of the normal vector, so as to generate a drivable area.

5 . The active perception system according to claim 1 , wherein predicting the risk index of each detection target by calculating the information entropy of each sensor and acquiring the indicators comprising the mark, speed and distance of the target, and determining the state of an obstacle after the risk index feeds back on the information entropy comprises:

generating a detection target list according to single-frame target detection information and fusing target features output by each sensor by a Bayesian method to obtain a consistent interpretation of the target,

calculating a likelihood probability P (E|H i ) of an observed feature E of the sensor when a given hypothesis Ha is true, comparing an observed value with a predetermined threshold th i , a number of expected targets, feature matching and quality of feature matching to determine a value of a likelihood function which represents a detection confidence of the target a j ; and

taking a target speed, a vehicle body speed, a tag value, and a position of the target relative to a vehicle body as evaluation criteria, establishing an expert system, comprising a knowledge base, a global database and an inference machine, for repeatedly inferring a detection risk level of the target, and considering that the detection risk level is high and actively updating the detection confidence of the target a j if a risk score of the target output by the expert system is greater than a risk level threshold.