Three dimensional grid map for cross member location and avoidance during an unloading operation
A three dimensional grid map is generated to locate a cross member on a receiving vehicle which is coupled to a following vehicle. The location of the cross member is tracked during an unloading operation so the leading vehicle avoids the cross member when unloading material into the receiving vehicle, and so the fill level is more accurately identified.
1 . A work machine system including a leading vehicle configured to unload material into a receiving vehicle during an unloading operation, the receiving vehicle being configured to be propelled by a following vehicle, the work machine system comprising:
a point cloud data sensor configured to detect point cloud data defining a plurality of points corresponding to the receiving vehicle;
a grid map generator configured to:
divide a material receiving volume, defined by the receiving vehicle, into a set of volumes; and
for each respective volume in the set of volumes,
identify a respective set of points, of the plurality of points, having a location that corresponds to the respective volume, and
assign a point value to the respective each volume in the set of volumes based on the respective set of points point cloud data; and
a cross member location system configured to identify a location of a fixture on the receiving vehicle based on the point values assigned to the volumes in the set of volumes; and
an unloading control system configured to control the unloading operation based on the location of the fixture.
2 . The work machine system of claim 1 and further comprising:
a coordinate system transform processor configured to convert a sensor coordinate system corresponding to the point cloud data sensor to a receiving vehicle coordinate system corresponding to the receiving vehicle.
3 . The work machine system of claim 1 wherein the grid map generator comprises:
a voxel configuration processor configured to identify the set of volumes as a set of voxels in the receiving vehicle coordinate system, each voxel corresponding to a separate volume in a separate location in the receiving vehicle, wherein the grid map generator is configured to assign a point value to each voxel based on the point cloud data.
4 . The work machine system of claim 3 wherein
the grid map generator comprises:
an occupancy threshold comparison system configured to compare the point value in each voxel to an occupancy threshold and, for each voxel having a point value that meets the occupancy threshold, identify the voxel as an occupied voxel; and
the cross member location system is configured to identify the location of the fixture on the receiving vehicle based on the occupied voxel.
5 . The work machine system of claim 4 and further comprising:
an occupied voxel list generator configured to generate an occupied voxel list indicative of occupied voxels and the locations of the occupied voxels.
6 . The work machine system of claim 5 wherein the unloading control system comprises:
a fill location identifier configured to identify a fill location in the receiving vehicle; and
a comparison system configured to compare the fill location to the locations of the occupied voxels in the occupied voxel list and generate a location comparison signal indicative of whether the fill location and the locations of any of the occupied voxels overlap.
7 . The work machine system of claim 6 and further comprising:
a fill location change system configured to, if the location comparison signal indicates that the locations of any of the occupied voxels overlap, generate a fill location change signal to change the fill location to a different fill location that does not overlap the locations in the occupied voxel list.
8 . The work machine system of claim 5 wherein the cross member location system is configured to store a map of the occupied voxels corresponding to the receiving vehicle in a data store, identified in the data store by a vehicle pair identifier identifying, as a following vehicle and receiving vehicle pair, at least one of the following vehicle and receiving vehicle.
9 . The work machine system of claim 1 and further comprising:
an operator prompt generator configured to generate an operator prompt on an operator interface on the leading vehicle, the operator prompt prompting an operator to position the receiving vehicle relative to the leading vehicle so the point cloud data sensor can detect point cloud data corresponding to the receiving vehicle.
10 . The work machine system of claim 9 wherein the point cloud data sensor comprises:
a stereo camera.
11 . The work machine system of claim 10 wherein the operator prompt generator is configured to generate the prompt to prompt the operator to move at least one of the receiving vehicle or the leading vehicle so the stereo camera captures images of the receiving vehicle from a plurality of different perspectives.
12 . The work machine system of claim 1 , wherein the unloading control system is configured to control the unloading operation by controlling at least one of the leading vehicle, the following vehicle, or the receiving vehicle.
13 . A computer implemented method of controlling an unloading operation for unloading material from a leading vehicle into a receiving vehicle that is propelled by a following vehicle, the method comprising:
detecting point cloud data defining a plurality of points corresponding to the receiving vehicle;
dividing a material receiving volume defined by the receiving vehicle into a set of volumes;
for each respective volume in the set of volumes,
identifying a respective set of points, of the plurality of points, having a location that corresponds to the respective volume, and
assigning a point value to the respective each volume in the set of volumes based on the respective set of points point cloud data; and
identifying a location of a fixture on the receiving vehicle based on the point values assigned to the volumes in the set of volumes; and
controlling the unloading operation based on the location of the fixture.
14 . The computer implemented method of claim 13 , wherein the point cloud data is detected by a point cloud data sensor, and further comprising:
converting a sensor coordinate system corresponding to the point cloud data sensor to a receiving vehicle coordinate system corresponding to the receiving vehicle.
15 . The computer implemented method of claim 13 wherein dividing the receiving vehicle into a set of volumes comprises:
identifying the set of volumes as a set of voxels in the receiving vehicle coordinate system, each voxel corresponding to a separate volume in a separate location in the receiving vehicle, and wherein assigning a point value to each volume comprises assigning a point value to each voxel based on the point cloud data.
16 . The computer implemented method of claim 15 wherein identifying a location of a fixture comprises:
comparing the point value in each voxel to an occupancy threshold; and
for each voxel having a point value that meets the occupancy threshold, identifying the voxel as an occupied voxel; and
identifying the location of the fixture on the receiving vehicle based on the occupied voxel.
17 . The computer implemented method of claim 16 wherein identifying a location of a fixture comprises:
generating an occupied voxel list indicative of occupied voxels and the locations of the occupied voxels; and further comprising:
identifying a fill location in the receiving vehicle;
comparing the fill location to the locations of the occupied voxels in the occupied voxel list;
generating a location comparison signal indicative of whether the fill location and the locations of any of the occupied voxels overlap; and
if the location comparison signal indicates that the locations of any of the occupied voxels overlap, generating a fill location change signal to change the fill location to a different fill location that does not overlap the locations in the occupied voxel list.
18 . The computer implemented method of claim 13 wherein detecting the point cloud data comprises:
prompting an operator to move at least one of the receiving vehicle or the leading vehicle; and
capturing images of the receiving vehicle, with a stereo camera, from a plurality of different perspectives.
19 . The computer implemented method of claim 13 , wherein controlling the unloading operation based on the location of the fixture comprises controlling at least one of the leading vehicle, the following vehicle, or the receiving vehicle.
20 . A control system comprising:
a calibration system comprising:
a point cloud data sensor configured to detect point cloud data defining a plurality of points corresponding to a receiving vehicle;
a grid map generator configured to:
divide a material receiving volume, defined by the receiving vehicle, into a set of volumes; and
for each respective volume in the set of volumes,
identify a respective set of points, of the plurality of points, having a location that corresponds to the respective volume, and
assign a point value to the respective each volume in the set of volumes based on the respective set of points point cloud data; and
a cross member location system configured to identify a location of a fixture on the receiving vehicle based on the point values assigned to the volumes in the set of volumes; and
an unloading control system configured to control an unloading operation based on the location of the fixture on the receiving vehicle.