System and method for responding to obstacles in worksites using autonomous machines

1 . A method, comprising:

receiving first information, by a first processor and via a network, determined by a sensor disposed at a worksite, the first information including a first timestamp, and an indication of an object disposed at a first location at the worksite, wherein the first location is in a path of a first autonomous machine on a work plan at the worksite;

receiving, by the first processor, an object classification of the object; providing, by the first processor and via the network, instructions to a second processor disposed at the worksite, wherein when executed by the second processor, the instructions cause the second processor to:

determine that the object requires remediation to enable the first autonomous machine to complete the work plan;

determine a remediation procedure comprising one or more actions to enable the first autonomous machine to complete the work plan;

determine a second machine included in the remediation procedure from a set of machines located at the worksite based on capabilities of the second machine to perform the remediation procedure;

determine an adjusted work plan based on the first location, the object classification, the remediation procedure, and the second machine, the adjusted work plan comprising instructions for the first autonomous machine to proceed with the work plan by avoiding a region comprising the first location of the object and a remediation region associated with the remediation procedure, wherein the remediation region comprises an area in which the second machine will operate to perform the remediation procedure;

provide the remediation procedure to the second machine;

receive confirmation of completion of the remediation procedure from the second machine; and

in response to receiving the confirmation of completion, cause causing the first autonomous machine to complete a task in the region as part of the adjusted work plan.

2 . The method of claim 1 , wherein the sensor is carried by the first autonomous machine traversing a work surface of the worksite, the sensor providing the first information and the second information.

3 . The method of claim 1 , wherein the object classification comprises stationary objects or non-stationary objects.

4 . The method of claim 1 , wherein the object classification comprises at least one of:

machinery;

personnel; or

materials.

5 . The method of claim 1 , wherein the first information comprises image data and receiving the object classification comprises inputting the first information and the second information into a machine learning model trained for object recognition, the machine learning model configured to output the object classification.

6 . The method of claim 1 , wherein the instructions further cause the second processor to convey, to the first processor, an instruction to proceed with an adjusted work plan, the adjusted work plan configured to enable the first autonomous machine to perform a subsequent portion of the work plan outside of the region comprising the first location and the remediation region until the confirmation of completion of the remediation procedure is received.

7 . The method of claim 6 , wherein the adjusted work plan is based at least in part on the remediation procedure.

8 . The method of claim 6 , wherein the adjusted work plan comprises causing the first autonomous machine to proceed with the work plan at a reduced pace or proceed to a section of the work plan not obstructed by the object.

9 . The method of claim 1 , wherein the instructions to provide the remediation procedure to the second machine comprises:

determining a second work plan for the second machine based on the object classification and tracking information for the object associated with the first location, wherein the second work plan comprises operations for the second machine within the remediation region and is distinct from the work plan of the first autonomous machine;

conveying the second work plan to the second machine; and

causing the second machine to proceed with the second work plan, wherein the second machine comprises a second autonomous machine.

10 . A system, comprising:

one or more processors;

a sensor disposed at a worksite and in communication with the one or more processors; and

a memory operably connected to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to:

receive sensor data from the sensor, the sensor data including an indication of an object disposed at a location at the worksite, wherein the location is in a path of a first autonomous machine on a work plan at the worksite;

receive an object classification for the object;

determine that the object requires remediation to enable the first autonomous machine to complete the work plan;

determine, based on the object classification and the sensor data, a remediation procedure comprising one or more actions to remove the object from the path and enable the first autonomous machine to complete the work plan;

determine a second machine included in the remediation procedure from a set of machines located at the worksite based on capabilities of the second machine to perform the remediation procedure;

determine an adjusted work plan based on the location, the object classification, the remediation procedure, and the second machine, the adjusted work plan comprising instructions for the first autonomous machine to proceed with the work plan by avoiding a region comprising the location of the object and a remediation region associated with the remediation procedure, wherein the remediation region comprises an area in which the second machine will operate to perform the remediation procedure;

cause the second machine to perform the remediation procedure;

receive confirmation of completion of the remediation procedure from the second machine; and

in response to receiving the confirmation of completion, cause the first autonomous machine to complete a task in the region as part of the adjusted work plan.

11 . The system of claim 10 , wherein determining the second machine comprises determining an availability of the second machine at the worksite.

12 . The system of claim 11 , wherein the instructions cause the one or more processors to further:

determine an adjusted work plan for the autonomous machine, wherein the adjusted work plan is based at least in part on avoiding a path of the second machine during the remediation procedure; and

cause the autonomous machine to perform the adjusted work plan.

13 . The system of claim 12 , wherein the adjusted work plan comprises causing the first autonomous machine to proceed with the work plan at a reduced pace or proceed to a section of the work plan not obstructed by the object.

14 . The system of claim 10 , wherein the sensor is coupled to an autonomous aerial vehicle configured to monitor the worksite.

15 . The system of claim 10 , wherein the sensor is coupled to the first autonomous machine.

16 . The system of claim 10 , wherein causing the second machine to perform the remediation procedure comprises:

determining a second work plan for the second machine based on the object classification and location tracking information for the object, wherein the second work plan comprises operations for the second machine within the remediation region and is distinct from the work plan of the first autonomous machine;

conveying the second work plan to the second machine; and

causing the second machine to proceed with the second work plan, wherein the second machine comprises a second autonomous machine.

17 . The system of claim 10 , wherein the object classification comprises stationary objects or non-stationary objects.

18 . One or more non-transitory computer-readable media having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving sensor data from a sensor positioned at a worksite, the sensor data including an indication of an object disposed at a location at the worksite, wherein the location is in a path of a first autonomous machine on a work plan at the worksite;

determining, based on the sensor data, tracking information for the object;

receiving an object classification for the object;

determining that the object requires remediation to enable the first autonomous machine to complete the work plan;

determining, based on the object classification and the sensor data, a remediation procedure comprising one or more actions to remove the object from the path and enable the first autonomous machine to complete the work plan;

determining a second machine included in the remediation procedure from a set of machines located at the worksite based on capabilities of the second machine to perform the remediation procedure;

determining an adjusted work plan based on the location, the object classification, the remediation procedure, and the second machine, the adjusted work plan comprising instructions for the first autonomous machine to proceed with the work plan by avoiding a region comprising the location of the object and a remediation region associated with the remediation procedure, wherein the remediation region comprises an area in which the second machine will operate to perform the remediation procedure;

causing the second machine to perform the remediation procedure;

receiving confirmation of completion of the remediation procedure from the second machine; and

in response to receiving the confirmation of completion, causing the first autonomous machine to complete a task in the region as part of the adjusted work plan.

19 . The one or more non-transitory computer-readable media of claim 18 , further comprising:

determining an adjusted work plan for the autonomous machine, wherein the adjusted work plan is based at least in part on avoiding a path of the second machine during the remediation procedure; and

causing the autonomous machine to perform the adjusted work plan.

20 . The one or more non-transitory computer-readable media of claim 18 , wherein causing the second machine to perform the remediation procedure comprises:

determining a second work plan for the second machine based on the object classification and location tracking information for the object, wherein the second work plan comprises operations for the second machine within the remediation region and is distinct from the work plan of the first autonomous machine;

conveying the second work plan to the second machine; and

causing the second machine to proceed with the second work plan, wherein the second machine comprises a second autonomous machine.