IP Library Granted Patent US 12,056,921
Granted Patent B2
US 12,056,921 · App. 17/188,258 · Granted Aug 6, 2024

Diagnostic system visualization and control for an agricultural spraying machine

Inventors: Michelle Kwak (Ankeny, IA); Devin L. Mangus (Ankeny, IA); Barbra Cavanaugh (Ankeny, IA); Peter D. Martinson (Grimes, IA); Andrea Maria Anna Agarwal (Ankeny, IA); Michael Carley (Ankeny, IA)
Assignee: Deere & Company
G06V20/188B05B1/20B05B12/122H04N23/90A01C23/007A01M7/0089
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,056,921
App. No.
17/188,258
Granted
Aug 6, 2024
Kind
B2
Abstract

An agricultural machine includes a product application system comprising a plurality of actuatable applicator mechanisms. Each actuatable applicator mechanism is configured to apply a substance on an agricultural surface. An imaging system includes an imaging capture component configured to capture of an image of plant matter on the agricultural surface, and an image processing module configured to determine a characteristic of the plant matter based on the image. A control system is configured to control one or more of the actuatable applicator mechanisms to apply the substance on the agricultural surface based on the determined characteristic, obtain diagnostic data indicative of operation of the imaging system, and generate a user interface display that displays a representation of the diagnostic data.

Claims (50)

1. An agricultural machine comprising:

a product application system comprising a plurality of actuatable applicator mechanisms, each actuatable applicator mechanism configured to apply a substance on an agricultural surface;

an imaging system comprising:

first camera configured to capture an image of a plant matter on the agricultural surface;

a first image processing module configured to determine a characteristic of the plant matter based on the image;

a second camera; and

a second image processing module configured to process image data from the second camera; and

a control system configured to:

control one or more of the actuatable applicator mechanisms to apply the substance on the agricultural surface based on the characteristic;

obtain diagnostic data indicative of operation of at least one of the first camera, the second camera, the first image processing module, or the second image processing module; and

generate a user interface display that displays a representation of the diagnostic data.

2. The agricultural machine of claim 1 , wherein the plurality of actuatable applicator mechanisms comprise a plurality of controllable spray nozzles, and wherein the diagnostic data represents one or more of capture or processing of the image.

3. The agricultural machine of claim 1 , wherein the diagnostic data indicates a fault condition.

4. The agricultural machine of claim 1 , wherein the diagnostic data indicates a particular spraying mode.

5. The agricultural machine of claim 1 , wherein the first image processing module is associated with a first actuatable applicator mechanism of the plurality of actuatable applicator mechanisms,

the second image processing module is associated with a second actuatable applicator mechanism of the plurality of actuatable applicator mechanisms,

the control system is configured to control the first actuatable applicator mechanism based on a signal from the first image processing module,

the control system is configured to control the second actuatable applicator mechanism based on a signal from the second image processing module, and

the diagnostic data is indicative of an operation of at least one image processing module of the first image processing module or the second image processing module.

6. The agricultural machine of claim 5 , wherein

the agricultural machine comprises a boom,

the first actuatable applicator mechanism and the second actuatable applicator mechanism are spaced apart along the boom, and

the user interface display includes a plurality of display elements that represent a relative location of each of the first actuatable applicator mechanism and the second actuatable applicator mechanism on the boom.

7. The agricultural machine of claim 6 , wherein the user interface display visually associates the first actuatable applicator mechanism with the first camera and the second actuatable applicator mechanism with the second camera.

8. The agricultural machine of claim 6 , wherein the user interface display visually associates the first actuatable applicator mechanism with the first image processing module and the second actuatable applicator mechanism with the second image processing module.

9. The agricultural machine of claim 1 , wherein the user interface display displays the image and includes a display element that visually represents a region of interest within the image.

10. A method performed by an agricultural machine having a product application system comprising a plurality of actuatable applicator mechanisms, each actuatable applicator mechanism configured to apply a substance on an agricultural surface, the method comprising:

obtaining, from a first camera, an image of a plant matter on the agricultural surface;

determining, by a first image processing module, a characteristic of the plant matter based on the image;

obtaining image data from a second camera;

processing the image data by a second image processing module:

controlling one or more actuatable applicator mechanisms, of the plurality of actuatable applicator mechanisms, to apply the substance on the agricultural surface based on the characteristic;

obtaining diagnostic data indicative of operation of at least one of the first camera, the second camera, the first image processing module, or the second image processing module; and

generating a user interface display that displays a representation of the diagnostic data.

11. The method of claim 10 , wherein the plurality of actuatable applicator mechanisms comprise a plurality of controllable spray nozzles.

12. The method of claim 10 , wherein the diagnostic data indicates at least one of a fault condition or a particular spraying mode.

13. The method of claim 10 , wherein

the first image processing module is associated with a first actuatable applicator mechanism of the plurality of actuatable applicator mechanisms, and the method comprises controlling the first actuatable applicator mechanism based on a signal from the first image processing module.

14. The method of claim 13 , wherein the agricultural machine comprises a boom and the first actuatable applicator mechanism and a second actuatable applicator mechanism are spaced apart along the boom, and the user interface display includes a plurality of display elements that represent a relative location of each of the first actuatable applicator mechanism and the second actuatable applicator mechanism on the boom.

15. The method of claim 14 , wherein the user interface display visually associates the first actuatable applicator mechanism with the first camera and the second actuatable applicator mechanism with the second camera.

16. The method of claim 14 , wherein the user interface display visually associates the first actuatable applicator mechanism with the first image processing module and the second actuatable applicator mechanism with the second image processing module.

17. The method of claim 10 , wherein the user interface display displays the image and includes a display element that visually represents an image region within the image.

18. A control system for an agricultural machine having a product application system comprising a plurality of actuatable applicator mechanisms, each actuatable applicator mechanism configured to apply a substance on an agricultural surface, the control system configured to:

obtain, from a first camera an image of plant matter on the agricultural surface;

determine, by a first image processing module, a characteristic of the plant matter based on the image;

obtain image data from a second camera;

process the image data by a second image processing module;

control one or more actuatable applicator mechanisms, of the plurality of actuatable applicator mechanisms, to apply the substance on the agricultural surface based on the characteristic;

obtain diagnostic data indicative of operation of at least one of the first camera, the second camera, the first image processing module, or the second image processing module; and

generate a user interface display that displays a representation of the diagnostic data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 1, 2021
From: KWAK, MICHELLE; MANGUS, DEVIN L.; CAVANAUGH, BARBRA; MARTINSON, PETER D.; AGARWAL, ANDREA MARIA ANNA; CARLEY, MICHAEL
To: DEERE & COMPANY
Reel/Frame 055451/0441 →
Continuity (2)
Provisional Application 63107673 · Oct 30, 2020
Related Publication 20220138464A1 · May 5, 2022