Implement management system for determining implement state
An implement management system detects implement wear and monitors implement states to modify operating modes of a vehicle. The system can determine implement wear using the pull of the implement on the vehicle, the force and angle of which is represented by an orientation vector. The system may measure a current orientation vector and determine an expected orientation vector using sensors and a model (e.g., a machine learned model). Additionally, the implement management system can determine an implement state based on images of the soil and the implement captured by a camera onboard the vehicle during operation. The system may apply different models to the images to determine a likely state of the implement. The difference between the expected and current orientation vectors or the determined implement state may be used to determine whether and how the vehicle's operating mode should be modified.
1. A method comprising:
selecting, from among a plurality of implement ground engagement state models respectively corresponding to a plurality of soil states, an implement ground engagement state model based on a determined one of the plurality of soil states;
applying the selected implement ground engagement state model to a set of images of an implement; and
performing an action based on a ground engagement state of the implement determined by the selected implement ground engagement state model.
2. The method of claim 1 , wherein the one of the plurality of soil states is determined by accessing soil state information obtained via sensors of a vehicle pulling the implement through soil.
3. The method of claim 2 , wherein the set of images of the implement are captured by an image sensor mounted on the vehicle.
4. The method of claim 2 , further comprising:
accessing implement state information obtained via height sensors of the vehicle;
determining a height of the implement based on the accessed implement state information; and
determining the ground engagement state of the implement based on the determined height and an output of the selected implement ground engagement state model.
5. The method of claim 4 , wherein the height sensors include one or more of potentiometers, altimeter, angle sensors, GPS, radar, or sonar.
6. The method of claim 4 , wherein determining the height of the implement comprises:
determining an expected elevation based on GPS and a predefined route of the vehicle;
determining an empirical elevation of the implement using an altimeter; and
calculating a difference between the expected elevation and the empirical elevation to determine the height.
7. The method of claim 2 , further comprising:
determining a location of the vehicle within a predefined route of the vehicle; and
in response to the determined location matching a predetermined location, determining the ground engagement state of the implement.
8. The method of claim 2 , wherein performing the action comprises, in response to a next vehicle maneuver requiring the ground engagement state of the implement to be raised and the determined ground engagement state of the implement being ground-engaged, pausing operation of the vehicle.
9. The method of claim 2 , wherein performing the action comprises, in response to a next vehicle maneuver requiring the ground engagement state of the implement to be raised and the determined ground engagement state of the implement being raised, authorizing the vehicle to perform the next vehicle maneuver.
10. The method of claim 9 , wherein the next vehicle maneuver is a tight turn.
11. The method of claim 2 , wherein performing the action comprises, in response to an operating mode requiring the ground engagement state of the implement to be ground-engaged and the determined ground engagement state of the implement being raised, instructing the vehicle to engage the implement with the soil.
12. The method of claim 2 , wherein performing the action comprises modifying an operating mode of the vehicle based on the determined ground engagement state of the implement.
13. The method of claim 2 , wherein the accessed soil state information comprises images of the soil and wherein selecting the implement ground engagement state model comprises:
training a machine learning model on training images of soil, each training image labeled with an empirical plasticity level of the soil;
applying the trained machine learning model to the images of the soil to determine a plasticity level of the soil based on the images of the soil; and
selecting the implement ground engagement state model based on the determined plasticity level of the soil.
14. The method of claim 2 , wherein the soil state information obtained via the sensors comprises a torque measurement of the vehicle, wherein a plurality of torque measurements respectively correspond to the plurality of soil states.
15. The method of claim 1 , wherein the implement comprises one or more of a shank, sweep, disk, blade, wheel, tine, or knife.
16. The method of claim 1 , wherein each of the plurality of soil states comprises one or more of a soil type, a soil moisture measurement, or a soil compaction measurement.
17. The method of claim 1 , wherein the ground engagement state of the implement is raised or ground-engaged.
18. The method of claim 1 , further comprising determining the ground engagement state of the implement using a collective decision based on an output of the selected implement ground engagement state model and further based on one or more of hydraulic information, hitch information, sensor information, or perception information.
19. A system comprising a hardware processor and a non-transitory computer-readable storage medium storing executable instructions that, when executed by the processor, are configured to cause the system to perform steps comprising:
selecting, from among a plurality of implement ground engagement state models respectively corresponding to a plurality of soil states, an implement ground engagement state model based on a determined one of the plurality of soil states;
applying the selected implement ground engagement state model to a set of images of an implement; and
performing an action based on a ground engagement state of the implement determined by the selected implement ground engagement state model.
20. A non-transitory computer readable storage medium storing executable instructions that, when executed by one or more processors, cause the one or more processors to perform steps comprising:
selecting, from among a plurality of implement ground engagement state models respectively corresponding to a plurality of soil states, an implement ground engagement state model based on a determined one of the plurality of soil states;
applying the selected implement ground engagement state model to a set of images of an implement; and
performing an action based on a ground engagement state of the implement determined by the selected implement ground engagement state model.