Method for controlling motion parameters of pepper harvester based on combination of point clouds and images
A method for controlling motion parameters of a pepper harvester based on combination of point clouds and images, including: (1) performing space-time calibration on a radar and a camera, constructing a data collection platform, and pre-processing image and point cloud data; (2) acquiring a real-time number of pepper in front of a harvester and an average height from canopies of pepper plants to soil through a point cloud, acquiring a cutting width and a pepper fruit proportion in front of the harvester through an image, and acquiring a comprehensive lowermost location of pepper fruits by combination of the point cloud and the image; (3) acquiring real-time motion parameters of the pepper harvester; and (4) controlling a real-time rotation speed, operation speed, and height of a drum using a fuzzy PID control method based on the comprehensive lowermost location of the pepper fruits and a predicted value of a feeding quantity.
1 . A method for controlling motion parameters of a pepper harvester based on combination of point clouds and images, comprising:
acquiring laser point cloud data and image data of pepper plants in a to-be-harvested region from a lidar and a camera fixedly mounted on a cab top of a harvester;
implementing time synchronization between the camera and the lidar using a multi-source sensor time synchronization method based on frequency self-matching and then implementing space synchronization using a combined calibration method on a basis of the time synchronization so as to synchronize the laser point cloud data and the image data in both time and space thereafter, performing pre-processing operations comprising denoising and enhancing on the laser point cloud data and the image data collected by the camera, reducing an area of a non-operation region, reducing noise, increasing a running speed of a system, performing coordinate system transformation on the laser point cloud data collected by the lidar so as to obtain same region of interest (ROI) of processed laser point cloud data and processed image data, and finally obtaining a conversion relationship between an image pixel coordinate system and a lidar coordinate system, a conversion relationship between the lidar coordinate system and a vehicle coordinate system, and a conversion relationship between the vehicle coordinate system and a geodetic coordinate system;
acquiring a real-time number n of the pepper plants in the to-be-harvested region in front of the pepper harvester and an average height L from canopies of the pepper plants to soil through the processed laser point cloud data;
acquiring a real-time cutting width w and a pepper fruit proportion p of the to-be-harvested region in front of the pepper harvester through the processed image data;
acquiring a comprehensive lowermost location of pepper fruits in the to-be-harvested region by combination of the point clouds and images;
measuring a real-time rotation speed N of a picking drum using a Hall sensor, symmetrically and fixedly mounting sensor probes on left and right ends of a drum shaft, fixing steel magnets on spring-tooth mounting plates on left and right sides, measuring a real-time operation speed V 0 of the pepper harvester by a navigation speed measurement system, mounting the navigation speed measurement system on the cab top of the pepper harvester, and mounting drum height sensors on both sides of a drum supporting shaft for acquiring a real-time height of the drum, the drum height sensors being ultrasonic sensors;
predicting a pepper plant density q through an improved back propagation (BP) neural network model, taking the real-time number n of the pepper plants, a height H of a cutting table, the average height L from the canopies of the pepper plants to the soil, and the pepper fruit proportion p as inputs of an improved BP neural network and a total mass m under a fixed area as an output of the improved BP neural network, calculating the pepper plant density q through a formula according to the total mass m, and further calculating a predicted value of a feeding quantity at a next time,
a formula for calculating the pepper plant density being:
q
=
m
1000
·
s
′
.
wherein m is the total mass under the fixed area predicted by the improved BP neural network, and s t is an actual area processed;
a formula for calculating the feeding quantity being:
Q
=
wvq
wherein Q is the predicted value of the feeding quantity, w is the real-time cutting width, v is an operation speed of the pepper harvester, and q is the pepper plant density; and
controlling a real-time rotation speed of the drum and the operation speed of the pepper harvester according to the predicted value of the feeding quantity and controlling the height of the cutting table according to the comprehensive lowermost location of the pepper fruits, based on fuzzy proportional-integral-derivative (PID) control.
2 . The method for controlling motion parameters of a pepper harvester based on combination of point clouds and images according to claim 1 , wherein the lidar, the camera, and the navigation speed measurement system are a solid-state lidar, a drive-free universal serial bus (USB) high-definition camera, and a Beidou Navigation Satellite System/Global Position System dual-system navigation speed measurement system, the navigation speed measurement system being a real-time speed acquisition method based on extended Kalman filtering so as to improve a measurement accuracy and noise suppression capability of speed measurement.
3 . The method for controlling motion parameters of a pepper harvester based on combination of point clouds and images according to claim 1 , wherein the implementing the time synchronization and the space synchronization between the lidar and the camera further comprises: implementing the time synchronization using the multi-source sensor time synchronization method based on the frequency self-matching and then implementing the space synchronization using the combined calibration method on the basis of the time synchronization, wherein the vehicle coordinate system is to be defined during calibration, and laser point cloud coordinates are obtained through image pixel coordinates and corresponding geodetic coordinates are obtained through the laser point cloud coordinates finally.
4 . The method for controlling motion parameters of a pepper harvester based on combination of point clouds and images according to claim 1 , wherein the acquiring of the real-time number n of the pepper plants in the to-be-harvested region in front of the pepper harvester and the average height L from the canopies of the pepper plants in the to-be-harvested region to the soil further comprises: acquiring the real-time number n of the pepper plants in the to-be-harvested region and the average height L from the canopies of the pepper plants in the to-be-harvested region to the soil by performing a Euclidean distance clustering segmentation algorithm and a Density-Based Spatial Clustering of Applications with Noise (DBSCAN)-based adaptive point cloud clustering on the processed laser point cloud data, respectively, for predicting the pepper plant density q by the improved BP neural network model subsequently.
5 . The method for controlling motion parameters of a pepper harvester based on combination of point clouds and images according to claim 1 , wherein the acquiring of the real-time cutting width w of the pepper harvester further comprises: collecting and processing pictures of the pepper plants in front of the cutting table, marking the cutting table of the pepper harvester in the pictures thereafter, performing spacing template matching once to mark a row of plants nearest to the cutting table, taking a length of the cutting table as a total cutting width to further determine a number of plants within the cutting table in front of the cutting table, obtaining a distance between two farthest plants at this moment, and then obtaining the real-time cutting width w.
6 . The method for controlling motion parameters of a pepper harvester based on combination of point clouds and images according to claim 1 , wherein the acquiring of the pepper fruit proportion p of the pepper plants in the to-be-harvested region further comprises: processing the images collected by the camera to obtain a binarized image of the pepper fruits and a binarized image of a whole pepper plant, and then acquiring a number of pixel points of the binarized image of the pepper fruits and a number of pixel points of the binarized image of the whole pepper plant by an iterator access, a ratio of the number of pixel points of the binarized image of the pepper fruits to the number of pixel points of the binarized image of the whole pepper plant being the pepper fruit proportion p, the image processing comprising the following steps:
performing histogram equalization processing again on a pre-processed picture to further increase a contrast of the picture and a clarity of the image;
filtering processing: removing image noise and better preserving image edges using bilateral filtering based on comprehensive consideration;
image segmentation: performing image segmentation using an Otsu's Method;
morphological processing: eliminating small particle noise in the image using erosion and dilation; and
obtaining the binarized image of the pepper fruits and the binarized image of the whole pepper plant, respectively.
7 . The method for controlling motion parameters of a pepper harvester based on combination of point clouds and images according to claim 1 , wherein the acquiring of the comprehensive lowermost location of the pepper fruits in the to-be-harvested region further comprises: after the pixel points of the binarized image of the whole pepper plant are scanned and detected to obtain a pepper pixel coordinate set of the pepper fruits located at a lowermost layer in a pepper plant image, obtaining a corresponding lidar coordinate point set through a conversion relationship between image pixel coordinates and lidar coordinates, obtaining a corresponding geodetic coordinate point set through a conversion relationship between the lidar coordinates and geodetic coordinates, and finally calculating an average value of Z coordinate values of points in a geodetic coordinate set as the comprehensive lowermost location of the pepper fruits.
8 . The method for controlling motion parameters of a pepper harvester based on combination of point clouds and images according to claim 1 , wherein the controlling of the operation speed further comprises: setting a reasonable feeding quantity range λ 1 <Q<λ 2 in advance according to a magnitude of a torque to be applied to the drum shaft, appropriately reducing the operation speed when the predicted value of the feeding quantity exceeds the reasonable feeding quantity of the pepper harvester, appropriately increasing the operation speed when the predicted value of the feeding quantity is lower than the reasonable feeding quantity of the pepper harvester, and reducing a failure rate of the pepper harvester while ensuring an operation efficiency of the pepper harvester, a formula for calculating a target operation speed being:
V
=
V
0
+
Q
rated
-
Q
2.5
·
q
wherein V is the target operation speed, V 0 is the real-time operation speed of the pepper harvester, Q rated is a rated feeding quantity of the pepper harvester, Q is the predicted feeding quantity, and q is the pepper plant density.
9 . The method for controlling motion parameters of a pepper harvester based on combination of point clouds and images according to claim 1 , wherein the controlling the real-time rotation speed of the drum comprises: setting a reasonable feeding quantity range λ 1 <Q <λ 2 in advance according to a magnitude of a torque to be applied to the drum shaft, not changing the real-time rotation speed of the drum when the predicted value is within the reasonable feeding quantity range, appropriately increasing the real-time rotation speed of the drum when the predicted feeding quantity is lower than a threshold, and reducing the real-time rotation speed of the drum conversely, an adjustment range of the real-time rotation speed of the drum being consistent with a change range of the predicted feeding quantity.
10 . The method for controlling motion parameters of a pepper harvester based on combination of point clouds and images according to claim 1 , wherein the controlling the height of the cutting table comprises: presetting a reasonable adjustment range of the height of the cutting table according to a measured average height from the canopies of the pepper plants to the soil, and resizing the cutting table according to the comprehensive lowermost location of the pepper fruits.