IP Library Granted Patent US 11,301,694
Granted Patent B2
US 11,301,694 · App. 16/433,532 · Granted Apr 12, 2022

Detecting plugging of ground-engaging tools of an agricultural implement from imagery of a field using a machine-learned classification model

Inventor: Trevor Stanhope (Palos Hills, IL)
Assignee: CNH Industrial America LLC
G06K9/00791A01B76/00A01B79/02G06K9/6268A01C5/064
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,301,694
App. No.
16/433,532
Granted
Apr 12, 2022
Kind
B2
Abstract

In one aspect, a computing system may be configured to perform operations including obtaining image data that depicts a portion of a field positioned aft of a ground-engaging tool of an agricultural implement relative to a direction of travel of the agricultural implement. The operations may also include inputting the image data into a machine-learned classification configured to receive imagery and process the imagery to output one or more visual appearance classifications for the imagery. Furthermore, the operations may include receiving a visual appearance classification of the image data as an output of the machine-learned classification model. Additionally, the operations may include determining when the ground-engaging tool is plugged based on the visual appearance classification of the image data.

Claims (41)

1. A computing system, comprising:

one or more processors; and

one or more non-transitory computer-readable media that collectively store:

a machine-learned classification model configured to receive imagery and process the imagery to output one or more visual appearance classifications for the imagery; and

instructions that, when executed by the one or more processors, configure the computing system to perform operations, the operations comprising:

obtaining image data that depicts a portion of a field positioned aft of a ground-engaging tool of an agricultural implement relative to a direction of travel of the agricultural implement;

inputting the image data into the machine-learned classification model;

receiving a visual appearance classification of the image data as an output of the machine-learned classification model; and

determining when the ground-engaging tool is plugged based on the visual appearance classification of the image data.

2. The computing system of claim 1 , wherein obtaining the image data comprises obtaining image data captured by an imaging device installed on the agricultural implement such that the imaging device has a field of view directed entirely behind the ground-engaging tool.

3. The computing system of claim 1 , wherein:

obtaining the image data comprises obtaining image data that depicts a portion of the field positioned aft of a plurality of ground-engaging tools of the agricultural implement relative to the direction of travel;

determining when the ground-engaging tool is plugged comprises determining when one of the plurality of ground-engaging tools is plugged based on the visual appearance classification of the image data; and

when it is determined that one of the plurality of ground-engaging tools is plugged, the operations further comprise identifying which ground-engaging tool of the plurality of ground-engaging tools is plugged based on the visual appearance classification of the image data.

4. The computing system of claim 1 , wherein the machine-learned classification model comprises a convolutional neural network.

5. The computing system of claim 1 , wherein the image data comprises a plurality of image frames.

6. The computing system of claim 5 , wherein the machine-learned classification model comprises a binary classifier configured to output for, each image frame, a binary visual appearance classification that classifies such image frame as a plugged visual appearance or a non-plugged visual appearance.

7. The computing system of claim 1 , wherein operations further comprise:

initiating display of an image frame of the image data to an operator of the agricultural implement after the image frame has been classified;

receiving an operator input associated with a correct visual appearance classification for the image frame; and

training the machine-learned classification model based on the received operator input.

8. The computing system of claim 1 , wherein, when it is determined that the ground-engaging tool is plugged, the operations further comprise initiating a control action associated with de-plugging the ground-engaging tool.

9. The computing system of claim 8 , wherein the control action comprises notifying an operator of the agricultural implement that the ground-engaging tool is plugged.

10. The computing system of claim 8 , wherein the control action comprises adjusting an operating parameter of the agricultural implement.

11. A computer-implemented method, comprising:

obtaining, with a computing system comprising one or more computing devices, image data that depicts a portion of a field positioned aft of a ground-engaging tool of an agricultural implement relative to a direction of travel of the agricultural implement;

inputting, with the computing system, the image data into a machine-learned classification model configured to configured to receive imagery and process the imagery to output one or more visual appearance classifications for the imagery;

receiving, with the computing system, a visual appearance classification of the image data as an output of the machine-learned classification model; and

determining, with the computing system, when the ground-engaging tool is plugged based on the visual appearance classification of the image data.

12. The computer-implemented method of claim 11 , wherein obtaining the image data comprises obtaining image data captured by an imaging device installed on the agricultural implement such that the imaging device has a field of view directed entirely behind the ground-engaging tool.

13. The computer-implemented method of claim 11 , wherein:

obtaining, with the computing system, the image data comprises obtaining image data that depicts a portion of the field positioned aft of a plurality of ground-engaging tools of the agricultural implement relative to the direction of travel;

determining, with the computing system, when the ground-engaging tool is plugged comprises determining when one of the plurality of ground-engaging tools is plugged based on the visual appearance classification of the image data; and

when it is determined that one of the plurality of ground-engaging tools is plugged, the method further comprises identifying, with the computing system, which ground-engaging tool of the plurality of ground-engaging tools is plugged based on the visual appearance classification of the image data.

14. The computer-implemented method of claim 11 , wherein the machine-learned classification model comprises a convolutional neural network.

15. The computer-implemented method of claim 11 , wherein the image data comprises a plurality of image frames.

16. The computer-implemented method of claim 11 , further comprising:

initiating, with the computing system, display of an image frame of the image data to an operator of the agricultural implement after the image frame has been classified;

receiving, with the computing system, an operator input associated with a correct visual appearance classification for the image frame; and

training, with the computing system, the machine-learned classification model based on the received operator input.

17. The computer-implemented method of claim 11 , wherein, when it is determined that the ground-engaging tool is plugged, the computer-implement method further comprises initiating, with the computing system, a control action associated with de-plugging the ground-engaging tool.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 22, 2022
From: CNH INDUSTRIAL AMERICA LLC
To: BLUE LEAF I.P., INC.
Reel/Frame 061987/0305 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 6, 2019
From: STANHOPE, TREVOR
To: CNH INDUSTRIAL AMERICA LLC
Reel/Frame 049395/0639 →
Continuity (1)
Related Publication 20200387720A1 · Dec 10, 2020
Cited By (1)
US 12,342,744