Estimating and visualizing terrain parameters for excavation
Autonomous systems enable a machine, such as an excavator, to dig in a specified area with little to no human intervention. However, conventional autonomous systems require costly exteroceptive systems to monitor the terrain surface during the autonomous dig operation. Accordingly, embodiments are disclosed for estimating a terrain surface using only proprioceptive sensors, such as the linkage sensors on the work implement of an excavator. The terrain surface may be estimated by fitting a surface to one or more touchpoints collected using the work implement. The estimated terrain surface may be updated during a dig operation using a model of material flow and/or by collecting additional touchpoints. Embodiments enable a remote operator to visualize this terrain surface in a camera view by projecting the estimated terrain surface onto an image plane with a representation that depicts one or more terrain parameters.
1 . A method comprising:
using at least one hardware processor to:
generate a plurality of touchpoints from an output of one or more linkage sensors on a machine, wherein each of the touchpoints comprises a three-dimensional coordinate, and wherein the output represents a characteristic of a work implement of the machine resulting from touching a ground surface with the work implement while at least a portion of the machine remains stationary relative to the ground surface and prior to performing a dig operation with the work implement;
determine a terrain surface of the ground surface from the touchpoints;
generate a two-dimensional visual representation of the terrain surface by projecting the terrain surface onto an image plane; and
output to a display the two-dimensional visual representation of the terrain surface overlaid on a camera view, wherein the two-dimensional visual representation is observable during the dig operation with the work implement.
2 . The method of claim 1 , further comprising using the at least one hardware processor to receive an angle of repose, wherein determining the terrain surface comprises defining the terrain surface as a mound having the angle of repose, and wherein one touchpoint is at a center of the mound.
3 . The method of claim 1 , wherein determining the terrain surface comprises defining a boundary of the terrain surface to include the plurality of touchpoints.
4 . The method of claim 3 , wherein the boundary is defined to include the touchpoints at vertices of the boundary.
5 . The method of claim 1 , wherein determining the terrain surface comprises computing a best-fit surface from the plurality of touchpoints.
6 . The method of claim 1 , wherein determining the terrain surface comprises constraining the terrain surface to a region of maximum mechanical advantage associated with the machine.
7 . The method of claim 1 , further comprising using the at least one hardware processor to, during the dig operation by the machine, over one or more iterations:
update the terrain surface based on the output of the one or more linkage sensors and a model of material flow; and
update the two-dimensional visual representation based on the update to the terrain surface.
8 . The method of claim 7 , wherein updating the terrain surface comprises, for at least one position on the terrain surface:
determining that material has been removed from the position based on the output of the one or more linkage sensors;
in response to determining that material has been removed from the position, estimating an amount of material that has been removed from the position; and
decreasing an elevation of one or more points at the position based on the estimated amount of material that has been removed.
9 . The method of claim 8 , wherein the amount of material that has been removed is estimated based on the model of material flow.
10 . The method of claim 8 , wherein the amount of material that has been removed is estimated based on one or more dimensions of a bucket of the work implement.
11 . The method of claim 1 , wherein the two-dimensional visual representation comprises a mesh with lines contoured to elevations of the terrain surface.
12 . The method of claim 11 , wherein the two-dimensional visual representation comprises gradations of color, and wherein the gradations of color represent relative values of a terrain parameter across the terrain surface.
13 . The method of claim 11 , wherein the two-dimensional visual representation comprises gradations of intensity, and wherein the gradations of intensity represent relative values of a terrain parameter across the terrain surface.
14 . The method of claim 11 , wherein the two-dimensional visual representation comprises gradations of color and gradations of intensity, wherein the gradations of color represent relative values of a first terrain parameter across the terrain surface, wherein the gradations of intensity represent relative values of a second terrain parameter across the terrain surface, and wherein the second terrain parameter is different from the first terrain parameter.
15 . The method of claim 1 , wherein at least one of the one or more linkage sensors measures a position of a component of the work implement relative to another component of the machine.
16 . The method of claim 15 , wherein the work implement comprises a boom, a stick, and a bucket, and wherein the one or more linkage sensors measure a position of each of the boom, the stick, and the bucket, relative to another component of the machine.
17 . The method of claim 1 , further comprising using the at least one hardware processor to perform the dig operation autonomously by automatically:
generating one or more additional touchpoints;
redetermining the terrain surface based on the one or more additional touchpoints; and
updating the two-dimensional visual representation of the terrain surface based on the redetermined terrain surface.
18 . A system comprising:
at least one hardware processor; and
software that is configured to, when executed by the at least one hardware processor,
generate touchpoints from an output of one or more linkage sensors on a machine including a ground engaging member configured to drive the machine across a ground surface, wherein each of the touchpoints comprises a three-dimensional coordinate, and wherein the output represents a characteristic of a work implement of the machine resulting from touching a ground surface without movement of the machine across the ground surface by the ground engaging member before use of the work implement to perform a material removal operation,
determine a terrain surface of the ground surface from the touchpoints,
generate a two-dimensional visual representation of the terrain surface by projecting the terrain surface onto an image plane, and
display on a display a camera view with the two-dimensional visual representation of the terrain surface overlaid thereon.
19 . The system of claim 18 , wherein the at least one hardware processor is configured to define a boundary of the terrain surface which includes the touchpoints at vertices of the boundary.
20 . A non-transitory computer-readable medium having instructions stored thereon, wherein the instructions, when executed by a processor, cause the processor to:
generate touchpoints from an output of one or more linkage sensors on a machine, wherein each of the touchpoints comprises a three-dimensional coordinate, and wherein the output represents a characteristic of a work implement of the machine resulting from touching a ground surface with the work implement when at least a portion of the machine is at rest relative to the ground surface over which the machine is configured to move, wherein the generating occurs prior to performing digging of the ground surface with the work implement;
determine a terrain surface of the surface from the one or more touchpoints;
generate a two-dimensional visual representation of the terrain surface by projecting the terrain surface onto an image plane; and
output to a display a camera view and the two-dimensional visual representation of the terrain surface overlaid onto the camera view prior to the digging of the ground surface with the work implement.