System and method for visually representing subterranean objects of interest in a field of view display for a work area
A data processing system and method are provided for preventing avoidable engagement with subterranean objects during operations by work machines comprising ground-engaging work implements. Electronically mapped position data are determined for subterranean objects of interest (SOI) in an area to be worked, comprising at least latitudinal and longitudinal dimensions for each SOI. The position data may be detected and mapped using subterranean monitoring sensors mounted on work implements. Image data are captured corresponding to a field of view directed from a point associated with the work machine. Respective positions are determined for any SOI within the field of view, relative to the point. A display is automatically rendered corresponding to the image data, further visually populated to include virtual tokens for each SOI within the field of view and corresponding to the respective position data. The display may comprise augmented reality or mixed reality content rendered on a wearable unit.
1 . A computer-implemented method of visually representing subterranean objects of interest during operations by a work machine comprising a work implement having a ground engaging tool on a first end thereof, wherein the work implement is coupled at a second end to a frame of the work machine and configured to move independently of the frame, the method comprising:
determining electronically mapped position data of one or more subterranean objects of interest in an area to be worked, wherein the mapped position data comprise at least latitudinal and longitudinal dimensions for each respective subterranean object of interest in a global coordinate system;
capturing image data corresponding to a field of view directed from a point associated with the work machine;
determining respective positions for any of the one or more subterranean objects of interest within the field of view, relative to the point associated with the work machine; and
automatically rendering a display corresponding to the captured image data, further visually populated to include virtual tokens for each of the any subterranean objects of interest within the field of view and corresponding to the respective position data thereof.
2 . The computer-implemented method of claim 1 , wherein the steps of determining respective positions for any of the one or more subterranean objects of interest within the field of view, relative to the point associated with the work machine, comprises fusing the mapped position data and position data corresponding to an image data source pose into a common reference coordinate system.
3 . The computer-implemented method of claim 2 , wherein the image data source is a camera mounted on the work machine, and moveable in orientation to define the image data source pose and the field of view.
4 . The computer-implemented method of claim 2 , wherein the image data source is a camera mounted on a wearable unit, the wearable unit comprising the display and an inertial sensor unit configured to generate output signals representing the image data source pose based on movements of a wearer thereof.
5 . The computer-implemented method of claim 1 , wherein the virtual tokens are visually rendered to correspond with ground surface locations within the field of view having the latitudinal and longitudinal dimensions for each respective subterranean object of interest, and each virtual token is further associated with visually rendered indicia representing a mapped depth of the respective subterranean object of interest.
6 . The computer-implemented method of claim 1 , comprising automatically controlling movement of the ground engaging tool with respect to a target grade profile, further accounting for and avoiding engagement with the one or more subterranean objects of interest in the area to be worked.
7 . The computer-implemented method of claim 1 , comprising automatically rendering the display further visually populated to include a virtual path for the ground engaging tool and accounting for and avoiding engagement with the one or more subterranean objects of interest in the area to be worked.
8 . The computer-implemented method of claim 1 , comprising:
detecting at least one of the one or more subterranean objects of interest via output signals from a subterranean monitoring sensor associated with the work implement; and
automatically mapping one or more parameters associated with the at least one of the one or more subterranean objects to respective locations in an electronic worksite map.
9 . The computer-implemented method of claim 8 , wherein the electronic worksite map comprises previously mapped position data for the at least one of the one or more subterranean objects of interest, the method further comprising verifying a work plan for the work machine generated based on the previously mapped position data, by comparison with the detected at least one of the one or more subterranean objects of interest.
10 . The computer-implemented method of claim 8 , wherein the electronic worksite map comprises previously mapped position data within a portion of the area to be worked, the method comprising generating a prompt to traverse a remaining portion of the area to be worked and to detect and map any further subterranean objects of interest.
11 . The computer-implemented method of claim 1 , comprising:
detecting at least one of the one or more subterranean objects of interest via output signals from a subterranean monitoring sensor associated with a work implement of a first work machine;
automatically mapping one or more parameters associated with the at least one of the one or more subterranean objects to respective locations in an electronic worksite map;
wherein upon updating the electronic worksite map by automatically mapping the one or more parameters associated with the at least first type of detected object to respective locations therein the electronic worksite map, the updated electronic worksite map is uploaded to a remote data storage by the first work machine, and retrievable by a second work machine performing an earth working operation in association with the area to be worked.
12 . A system for visually representing subterranean objects of interest during operations by a work machine comprising a work implement having a ground engaging tool on a first end thereof, wherein the work implement is coupled at a second end to a frame of the work machine and configured to move independently of the frame, the system comprising:
data storage comprising electronically mapped position data of one or more subterranean objects of interest in an area to be worked, wherein the mapped position data comprise at least latitudinal and longitudinal dimensions for each respective subterranean object of interest in a global coordinate system;
an image data source configured to capture image data corresponding to a field of view directed from a point associated with the work machine; and
one or more processors functionally linked to the data storage and to the image data source and configured to:
determine respective positions for any of the one or more subterranean objects of interest within the field of view, relative to the point associated with the work machine; and
automatically render a display corresponding to the captured image data, further visually populated to include virtual tokens for each of the any subterranean objects of interest within the field of view and corresponding to the respective position data thereof.
13 . The system of claim 12 , wherein the one or more processors are configured to determine respective positions for any of the one or more subterranean objects of interest within the field of view, relative to the point associated with the work machine, at least in part by fusing the mapped position data and position data corresponding to an image data source pose into a common reference coordinate system.
14 . The system of claim 13 , wherein the image data source is a camera mounted on the work machine, and moveable in orientation to define the image data source pose and the field of view.
15 . The system of claim 13 , wherein the image data source is a camera mounted on a wearable unit, the wearable unit comprising the display and an inertial sensor unit configured to generate output signals representing the image data source pose based on movements of a wearer thereof.
16 . The system of claim 12 , wherein the virtual tokens are visually rendered to correspond with ground surface locations within the field of view having the latitudinal and longitudinal dimensions for each respective subterranean object of interest, and each virtual token is further associated with visually rendered indicia representing a mapped depth of the respective subterranean object of interest.
17 . The system of claim 12 , wherein the one or more processors are configured to generate control signals for automatically controlling movement of the ground engaging tool with respect to a target grade profile, further accounting for and avoiding engagement with the one or more subterranean objects of interest in the area to be worked.
18 . The system of claim 12 , wherein the one or more processors are configured to:
detect at least one of the one or more subterranean objects of interest via output signals from a subterranean monitoring sensor associated with the work implement; and
automatically map one or more parameters associated with the at least one of the one or more subterranean objects to respective locations in an electronic worksite map.
19 . The system of claim 18 , wherein the electronic worksite map comprises previously mapped position data for the at least one of the one or more subterranean objects of interest, the one or more processors further configured to verify a work plan for the work machine generated based on the previously mapped position data, by comparison with the detected at least one of the one or more subterranean objects of interest.
20 . The system of claim 12 , wherein:
at least one of the one or more processors is associated with a first work machine, and configured to:
detect at least one of the one or more subterranean objects of interest via output signals from a subterranean monitoring sensor associated with a work implement of the first work machine;
automatically map one or more parameters associated with the at least one of the one or more subterranean objects to respective locations in an electronic worksite map; and
wherein upon updating the electronic worksite map by automatically mapping the one or more parameters associated with the at least first type of detected object to respective locations therein the electronic worksite map, the updated electronic worksite map is uploaded to the data storage;
wherein at least a further one of the one or more processors is associated with a second work machine and configured to retrieve the updated electronic worksite map for performing an earth working operation in association with the area to be worked.