Inline blade wear estimation based on processed soil surface

1 . A method for deriving a wear state of a soil interaction component of a soil processing implement of a construction vehicle, wherein the method comprises:

providing a geometry model regarding an assumed shape of the soil interaction component,

engaging a soil by using the soil interaction component and tracking a motion of the soil interaction component for deriving tracking data of the soil interaction component,

using the tracking data and the geometry model of the soil interaction component to derive an expected 3D surface model caused by the engaging,

after the engaging, providing visual 3D perception data of a soil area affected by the engaging with the soil interaction component, such that the visual 3D perception data and the expected 3D surface model can be referenced to one another,

comparing the visual 3D perception data with the expected 3D surface model and, based thereof, determining a deviation of an effective shape of the soil interaction component from the assumed shape, and

providing implement adjustment commands to the construction vehicle for adjusting the soil processing implement on the basis of the determined deviation, wherein the implement adjustment commands are provided to adjust the soil processing implement such that an anticipated wear with adjustment of the soil processing implement reduces the deviation of the effective shape of the soil interaction component from the assumed shape.

2 . The method according to claim 1 , wherein the method further comprises accessing an initial 3D surface model of the soil before the engaging, wherein a pose of the soil interaction component is controlled automatically on the basis of the initial 3D surface model of the soil and the geometry model of the soil interaction component.

3 . The method according to claim 2 , wherein:

the visual 3D perception data and/or the initial 3D surface model of the soil before the engaging comprises data regarding mechanical properties, a resistance, of the soil, and

data relating to an estimated remaining work hours of the soil interaction component is provided on the basis of the mechanical properties of the soil and on the basis of a time evolution of a corrected geometry model of the soil interaction component,

wherein the method further comprises:

providing an automatized adjustment of a pose of the soil processing implement for optimizing the estimated remaining work hours, on the basis of the mechanical properties of the soil, and

updating the data relating to the estimated remaining work hours.

4 . The method according to claim 2 , wherein the comparing of the visual 3D perception data with the expected 3D surface model further comprises:

identifying a deviating area between the expected 3D surface model and the visual 3D perception data with a difference in a surface height outside a tolerance range at a surface area,

determining a longitudinal extent of the deviating area, wherein a longitudinal direction is a direction of movement of the soil processing implement,

categorizing the deviating area as insignificant when the longitudinal extent of the deviating area is below an area threshold value, and

excluding the insignificant deviating area from the comparison.

5 . The method according to claim 1 , further comprising deriving a corrected geometry model of the soil interaction component on the basis of the determined deviation of the effective shape of the soil interaction component from the assumed shape, to provide or update a wear pattern database, wherein the wear pattern database comprises an evolution of the effective shape of the soil interaction component as a function of the tracking data.

6 . The method according to claim 5 , further comprising:

providing the implement adjustment commands for the soil processing implement on the basis of the corrected geometry model of the soil interaction component, wherein the implement adjustment commands cause:

engaging the soil by the soil interaction component on the basis of the corrected geometry model of the soil interaction component, and

updating the expected 3D surface model on the basis of the corrected geometry model of the soil interaction component,

and further comprising:

providing an error message to an operator of the construction vehicle on the basis of the corrected geometry model indicating that processing the soil cannot be performed within a tolerance range, and/or

providing an error message to the operator on the basis of the corrected geometry model indicating that the soil processing implement needs to be replaced, and/or

providing data to the operator relating to an estimated remaining work hours of the soil interaction component on the basis of a time evolution of the corrected geometry model of the soil interaction component, and on the basis of the wear pattern database.

7 . The method according to claim 6 , wherein the method further comprises:

accessing a load limit of the soil processing implement, the soil interaction component, representing an allowable load of the soil processing implement during a soil processing task with respect to the corrected geometry model and/or the estimated remaining work hours, and

providing an automatized adjustment of a pose of the soil processing implement for optimizing the estimated remaining work hours on the basis of the load limit.

8 . The method according to claim 6 , wherein the comparing of the visual 3D perception data with the expected 3D surface model further comprises:

identifying a deviating area between the expected 3D surface model and the visual 3D perception data with a difference in a surface height outside a tolerance range at a surface area,

determining a longitudinal extent of the deviating area, wherein a longitudinal direction is a direction of movement of the soil processing implement,

categorizing the deviating area as insignificant when the longitudinal extent of the deviating area is below an area threshold value, and

excluding the insignificant deviating area from the comparison.

9 . The method according to claim 5 , wherein the comparing of the visual 3D perception data with the expected 3D surface model further comprises:

identifying a deviating area between the expected 3D surface model and the visual 3D perception data with a difference in a surface height outside a tolerance range at a surface area,

determining a longitudinal extent of the deviating area, wherein a longitudinal direction is a direction of movement of the soil processing implement,

categorizing the deviating area as insignificant when the longitudinal extent of the deviating area is below an area threshold value, and

excluding the insignificant deviating area from the comparison.

10 . The method according to claim 1 , wherein the comparing of the visual 3D perception data with the expected 3D surface model further comprises:

identifying a deviating area between the expected 3D surface model and the visual 3D perception data with a difference in a surface height outside a tolerance range at a surface area,

determining a longitudinal extent of the deviating area, wherein a longitudinal direction is a direction of movement of the soil processing implement,

categorizing the deviating area as insignificant when the longitudinal extent of the deviating area is below an area threshold value, and

excluding the insignificant deviating area from the comparison.

11 . The method according to claim 1 , wherein the method further comprises:

performing a backward movement of the construction vehicle, wherein the backward movement represents a movement within a processed soil surface,

identifying traces of the construction vehicle, track or wheel marks, in the visual 3D perception data, and

excluding domains from the visual 3D perception data comprising the traces of the construction vehicle.

12 . A computer program product stored in a non-transitory computer readable medium and configured for automatic execution of computation of the method according to claim 11 , when run on a computing unit of a system.

13 . A system for deriving the wear state of the soil interaction component of the soil processing implement of the construction vehicle, wherein the system is configured to carry out the method of claim 1 , for which it comprises a computing unit, an implement tracking sensor, a positioning sensor, and a visual perception sensor, wherein:

the visual perception sensor is configured to:

be arranged on the construction vehicle such that its pose is referenceable to the construction vehicle, and

provide visual perception sensor data to the computing unit enabling a derivation of the visual 3D perception data of the soil area affected by the engaging with the soil interaction component,

the computing unit is configured to:

retrieve the assumed shape of the soil interaction component,

derive the tracking data on the basis of sensor readings of the implement tracking sensor and the positioning sensor,

derive the expected 3D surface model caused by the engaging using the tracking data and the geometry model of the soil interaction component,

derive the visual 3D perception data of the soil area affected by the engaging with the soil interaction component such that the visual 3D perception data and the expected 3D surface model can be referenced to one another, and

compare the visual 3D perception data with the expected 3D surface model and, based thereof, determine the deviation of the effective shape of the soil interaction component from the assumed shape.

14 . The system according to claim 13 , wherein the computing unit is configured to receive a 3D design model of a desired soil surface and to reference the 3D design model to the expected 3D surface model and the visual 3D perception data.

15 . The system according to claim 13 , wherein

the construction vehicle is a motor grader, a crawler or a cold planer, and

the soil processing implement is configured for soil cutting or leveling, and

the visual perception sensor comprises a component mounted on the construction vehicle so that its field of view comprises an area of a processed soil surface directly behind the soil processing implement, wherein the visual perception sensor further comprises a forward looking component providing a view of unengaged soil.

16 . The system according to claim 13 , wherein:

the visual perception sensor is configured to provide:

first visual data on the soil interaction component representing a wear state before starting soil processing, and

second visual data on the soil interaction component representing a wear state after finishing the soil processing,

the computing unit is configured to provide a comparison between a measured shape of the soil interaction component on the basis of the first and the second visual data and a corrected geometry model of the soil interaction component, wherein the corrected geometry model is derived on the basis of the determined deviation of the effective shape of the soil interaction component from the assumed shape.

17 . A computer program product stored in a non-transitory computer readable medium and configured for an automatic execution of the method according to claim 1 , when run on a computing unit of a system.

18 . A computer program product comprising program code, wherein the program code is stored on a non-transitory machine-readable medium, wherein the program code comprises a program code segment for deriving a wear state of a soil interaction component of a soil processing implement of a construction vehicle, wherein the computer program product comprises computer-executable instructions for performing:

retrieving input data comprising:

a geometry model regarding an assumed shape of the soil interaction component,

tracking data of the soil interaction component, and

visual perception sensor data of a soil area affected by engaging with the soil interaction component,

deriving on the basis of the input data:

visual 3D perception data of the soil area affected by the engaging with the soil interaction component, and

an expected 3D surface model caused by the engaging, wherein the visual 3D perception data and the expected 3D surface model are referenced to one another,

determining a deviation of an effective shape of the soil interaction component from the assumed shape on the basis of comparing the visual 3D perception data with the expected 3D surface model, and

providing adjustment commands to the construction vehicle for adjusting the soil processing implement on the basis of the determined deviation, wherein the adjustment commands are provided to adjust the soil processing implement such that an anticipated wear with adjustment of the soil processing implement reduces the deviation of the effective shape of the soil interaction component from the assumed shape.