Material handling equipment, controller, and method for determining extrinsic parameter

1 . A material handling equipment, comprising a controller, the controller being configured to execute program instructions, to implement the following actions:

obtaining a relative pose between a carrier and the material handling equipment as a first pose arranging a carrier and the material handling equipment at a preset relative pose and obtaining a first pose between the carrier and the material handling equipment based on the preset relative pose;

obtaining a second pose by using a sensor, the second pose being a pose of the carrier or a pose of a partial structure of the carrier; and

determining, based on the first pose and the second pose, an extrinsic parameter of the sensor relative to the material handling equipment; and

controlling the material handling equipment to execute a handling task according to the extrinsic parameter.

2 . The material handling equipment according to claim 1 , wherein arranging a carrier and the material handling equipment at a preset relative pose and obtaining a first pose between the carrier and the material handling equipment based on the preset relative pose the obtaining a relative pose between a carrier and the material handling equipment as a first pose comprises:

aligning a center of the carrier with a center of the material handling equipment, to enable two of three-dimensional relative coordinate values of the carrier and the material handling equipment to be zero, and enable a relative roll angle, a relative pitch angle, and a relative yaw angle of the carrier and the material handling equipment to be preset values;

obtaining another relative coordinate value of the three-dimensional relative coordinates; and

calculating the first pose between the carrier and the material handling equipment based on the three-dimensional relative coordinate value, the relative roll angle, the relative pitch angle, and the relative yaw angle.

3 . The material handling equipment according to claim 1 , wherein the obtaining a second pose by using a sensor comprises:

obtaining a first point cloud by using the sensor, the first point cloud being a point cloud obtained by sensing at least a partial structure of the carrier;

extracting a second point cloud from the first point cloud; and

determining the second pose based on the second point cloud.

4 . The material handling equipment according to claim 3 , wherein the extracting a second point cloud from the first point cloud comprises:

extracting, from the first point cloud, a point cloud conforming to a preset feature, as the second point cloud, the preset feature being a feature corresponding to at least the partial structure of the carrier.

5 . The material handling equipment according to claim 4 , wherein the carrier is a pallet, and the preset feature comprises one or more of the following features:

one or more parameters of piers of the pallet: a height of the piers, a width of the piers, a quantity of the piers, and a distance between center points of the piers;

one or more parameters of pier holes of the pallet: a height of the pier holes, a width of the pier holes, a quantity of the pier holes, a distance between center points of the pier holes, and a ratio of a region of the pier holes to a region of a pier surface;

one or more parameters of beams of the pallet: a height of upper beams of the pallet, a width of the upper beams of the pallet, a height of lower beams of the pallet, and a width of the lower beams of the pallet;

one or more features of the pallet: a height of the pallet, a width of the pallet, and a shape of the pallet;

an area of the piers of the pallet;

an area of the pier holes of the pallet; or

continuity of the upper beams of the pallet or the lower beams of the pallet.

6 . The material handling equipment according to claim 4 , wherein the extracting, from the first point cloud, a point cloud conforming to a preset feature, as the second point cloud comprises:

projecting the first point cloud into a two-dimensional image;

converting the two-dimensional image into a grayscale image and a binary image;

performing convolutional detection on the grayscale image, to obtain a convolution result;

determining, based on the convolution result, a relevant region related to the preset feature in the grayscale image;

corresponding the relevant region in the grayscale image to the binary image, to obtain a relevant region related to the preset feature in the binary image;

extracting an image feature from the relevant region in the binary image; and

determining a point cloud corresponding to the relevant region in the grayscale image as the second point cloud when the image feature conforms to the preset feature within a preset tolerance range.

7 . The material handling equipment according to claim 6 , wherein the determining, based on the convolution result, a relevant region related to the preset feature in the grayscale image comprises: performing non-maximum suppression NMS on the convolution result.

8 . The material handling equipment according to claim 3 , wherein the obtaining a second pose by using a sensor further comprises:

performing the following actions on the first point cloud when the material handling equipment is of a first type:

performing downsampling on the first point cloud, to obtain a third point cloud;

filtering out a point in the third point cloud, to obtain a fourth point cloud, an angle between a normal of the point and a normal of an adjacent point being greater than a first threshold; and

filtering out a point whose outlier value reaches a second threshold in the fourth point cloud.

9 . The material handling equipment according to claim 8 , wherein the obtaining a second pose by using a sensor further comprises:

performing the following actions on the first point cloud when the material handling equipment is not of the first type:

performing downsampling on the first point cloud, to obtain a third point cloud; and

identifying and removing a trailing point in the third point cloud.

10 . The material handling equipment according to claim 3 , wherein the extracting a second point cloud from the first point cloud further comprises: performing region-of-interest clipping on the first point cloud, and extracting the second point cloud from a point cloud obtained through clipping.

11 . The material handling equipment according to claim 3 , wherein the determining the second pose based on the second point cloud comprises:

performing plane fitting on the second point cloud, to obtain a first plane region;

determining a three-dimensional coordinate value, a roll angle, a pitch angle, and a yaw angle of at least the partial structure of the carrier based on the first plane region; and

calculating the second pose based on the three-dimensional coordinate value, the roll angle, the pitch angle, and the yaw angle.

12 . The material handling equipment according to claim 3 , wherein the material handling equipment comprises an automated guided forklift, the carrier is a pallet, the pallet comprises piers and pier holes, and the second pose is a pose of the piers; and the calculating the second pose based on the second point cloud comprises:

performing plane fitting on the second point cloud, to obtain a first plane region;

determining a first coordinate value, a second coordinate value, a roll angle, a pitch angle, and a yaw angle of the piers based on the first plane region;

performing ground extraction on the first point cloud, to obtain a plane region of a ground on which the pallet is located, and determining a height value of the ground based on the plane region of the ground; and

determining the second pose based on the first coordinate value, the second coordinate value, the roll angle, the pitch angle, and the yaw angle of the piers and the height value.

13 . The material handling equipment according to claim 3 , wherein the material handling equipment comprises an automated guided forklift, the carrier is a pallet, the pallet comprises piers and pier holes, and the second pose is a pose of the piers; and the calculating the second pose based on the second point cloud comprises:

performing plane fitting on the second point cloud, to obtain a first plane region;

determining a first coordinate value, a second coordinate value, a roll angle, a pitch angle, and a yaw angle of the piers based on the first plane region;

extracting a fifth point cloud from a plane obtained through fitting, the fifth point cloud being points whose distances to the first plane region are less than a first distance threshold in the second point cloud;

projecting the fifth point cloud on a plane perpendicular to a fork-out direction of a fork of the automated guided forklift, to obtain a two-dimensional image;

extracting an upper boundary of a pier hole in the two-dimensional image;

using a half of a height value of the upper boundary as a third coordinate value of the piers; and

determining the second pose based on the first coordinate value, the second coordinate value, the third coordinate value, the roll angle, the pitch angle, and the yaw angle of the piers.

14 . The material handling equipment according to claim 13 , wherein before the projecting the fifth point cloud on a plane perpendicular to a fork-out direction of a fork of the automated guided forklift, the actions further comprise:

calculating minimum/maximum coordinates of the fifth point cloud in three-dimensional space, to construct a voxel grid;

allocating the points in the fifth point cloud to the corresponding voxel grid;

aggregating the fifth point cloud within the same voxel grid, to calculate second representative points of the voxel grid; and

replacing all points in the voxel grid with the second representative points of the voxel grid.

15 . The material handling equipment according to claim 12 , wherein the performing plane fitting on the second point cloud comprises: performing plane fitting on the second point cloud based on a second distance threshold.

16 . The material handling equipment according to claim 12 , wherein the performing plane fitting on the second point cloud comprises:

performing plane fitting on the second point cloud based on a second distance threshold; and

performing least squares plane fitting on the second point cloud on which plane fitting is already performed.

17 . The material handling equipment according to claim 1 , wherein the determining, based on the first pose and the second pose, an extrinsic parameter of the sensor relative to the material handling equipment comprises:

calculating a first transformation matrix between the carrier and the sensor based on the second pose;

calculating a second transformation matrix between the carrier and the material handling equipment based on the first pose;

calculating a third transformation matrix between the sensor and the material handling equipment based on the first transformation matrix and the second transformation matrix; and

determining, based on the third transformation matrix, the extrinsic parameter of the sensor relative to the material handling equipment.

18 . A controller, configured to execute program instructions, to perform the following actions:

obtaining a relative pose between a carrier and a material handling equipment as a first pose arranging a carrier and the material handling equipment at a preset relative pose and obtaining a first pose between the carrier and the material handling equipment based on the preset relative pose;

obtaining a second pose by using a sensor, the second pose being a pose of the carrier or a pose of a partial structure of the carrier; and

determining, based on the first pose and the second pose, an extrinsic parameter of the sensor relative to the material handling equipment; and

controlling the material handling equipment to execute a handling task according to the extrinsic parameter.

19 . A method for determining an extrinsic parameter, the method comprising:

obtaining a relative pose between a carrier and a material handling equipment as a first pose arranging a carrier and the material handling equipment at a preset relative pose and obtaining a first pose between the carrier and the material handling equipment based on the preset relative pose;

obtaining a second pose by using a sensor, the second pose being a pose of the carrier or a pose of a partial structure of the carrier; and

determining, based on the first pose and the second pose, an extrinsic parameter of the sensor relative to the material handling equipment; and

controlling the material handling equipment to execute a handling task according to the extrinsic parameter.