IP Library › Granted Patent US 11,782,453
Granted Patent B2
US 11,782,453 · App. 17/176,291 · Granted Oct 10, 2023

Image-based position assessment of agricultural machine components

Inventor: Timothy J. Kraus (Blakesburg, IA)
Assignee: DEERE & COMPANY
G05D1/0253G05D1/0212G06T7/97G06T2207/30188G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,782,453
App. No.
17/176,291
Granted
Oct 10, 2023
Kind
B2
Abstract

An image-based position assessment method is performed in conjunction with a mechanism onboard an agricultural machine and containing a traveling component. In embodiments, the method includes receiving, at an image processing system, camera images captured by a diagnostic camera mounted to the agricultural machine. A field of view of the diagnostic camera at least partially encompasses an intended motion path along which the traveling component is configured to travel. The image processing system analyzes the camera images to determine whether a recorded time-dependent component position of the traveling component deviates excessively from an expected time-dependent component position of the traveling component, as taken along the intended motion path. The image processing further generates, or causes generation of, a first notification on a display device indicating that a positional adjustment to the traveling component is recommended if the recorded time-dependent component position deviates excessively from the expected time-dependent component position.

Claims (49)

1. An image-based position assessment method performed in conjunction with a timed mechanism onboard an agricultural machine, the timed mechanism including a traveling component configured to travel an intended motion path that is non-linear when the timed mechanism is driven during operation of the agricultural machine, the image-based position assessment method comprising:

receiving, at an image processing system, one or more diagnostic camera images captured by a diagnostic camera mounted to the agricultural machine at a designated mount location, the diagnostic camera having a field of view at least partially encompassing the intended motion path;

applying, at the image processing system, permissible path boundaries to the diagnostic camera images for comparison to the traveling component of the timed mechanism as taken along the intended motion path;

analyzing, at the image processing system, the diagnostic camera images to determine whether a recorded time-dependent component position of the traveling component deviates beyond the permissible path boundaries from an expected time-dependent component position of the traveling component, as taken along the intended motion path; and

generating or causing generation of a first notification on a display device, via the image processing system, indicating that a positional adjustment to the traveling component is recommended if determining that the recorded time-dependent component position deviates beyond the permissible path boundaries from the expected time-dependent component position.

2. The image-based position assessment method of claim 1 , further comprising generating or causing generation of a second notification on the display device indicating that a positional adjustment to the traveling component is unneeded if determining that the recorded time-dependent component position does not deviate beyond the permissible path boundaries from the expected time-dependent component position.

3. The image-based position assessment method of claim 1 , wherein receiving comprises receiving the diagnostic camera images as a series of diagnostic camera images captured by the diagnostic camera over a duration of time sufficient to allow the traveling component to travel the intended motion path at least once.

4. The image-based position assessment method of claim 1 , wherein the timed mechanism includes a co-moving component configured to move in conjunction with the traveling component; and

wherein the image-based position assessment method further comprises:

further analyzing, at the image processing system, the diagnostic camera images to determine whether a recorded time-dependent component position of the co-moving component deviates excessively from an expected time-dependent component position of the co-moving component; and

further generating or causing the generation of a second notification on the display device indicating that a positional adjustment to the co-moving component is recommended if determining that the recorded time-dependent component position of the co-moving component deviates excessively from an expected time-dependent component position of the co-moving component.

5. The image-based position assessment method of claim 1 , wherein at least one timing mark is provided on the traveling component; and

wherein the image-based position assessment method further comprises determining whether the recorded time-dependent component position deviates excessively from the expected time-dependent component position based, at least in part, on a detected location of the timing mark during a timing cycle of the timed mechanism.

6. The image-based position assessment method of claim 1 , wherein analyzing comprises:

receiving, at the image processing system, identification data from which a mechanism type of the timed mechanism can be determined;

utilizing the identification data to recall, from a database accessible to the image processing system, motion path data defining the intended motion path; and

establishing the expected time-dependent component position based, at least in part, on the recalled motion path data defining the intended path.

7. The image-based position assessment method of claim 6 , further comprising:

determining a timepoint in a timing cycle of the timed mechanism at which the recorded time-dependent component position occurs; and

establishing the expected time-dependent component position as a function of the recalled motion path data and the timepoint at which the recorded time-dependent component position occurs.

8. The image-based position assessment method of claim 6 , wherein receiving comprises receiving, at the image processing system, user input data describing the timed mechanism or the agricultural machine.

9. The image-based position assessment method of claim 8 , wherein receiving comprises receiving, at the image processing system, user input data specifying a make and model of the agricultural machine.

10. The image-based position assessment method of claim 1 , wherein the image processing system comprises at least one server in communication with the display device over a network; and

wherein the image-based position assessment method further comprises:

receiving the diagnostic camera images at the server when transmitted to the server over the network; and

transmitting instructions from the server, over the network, and to the display device instructing the display device to generate the first notification if determining that the recorded time-dependent component position deviates excessively from the expected time-dependent component position.

11. The image-based position assessment method of claim 1 , wherein the agricultural machine comprises an agricultural vehicle having a cabin in which the display device is located.

12. The image-based position assessment method of claim 1 , wherein the agricultural machine comprises an agricultural implement towed by an agricultural vehicle, the agricultural vehicle having a cabin in which the display device is located.

13. The image-based position assessment method of claim 1 , wherein the display device comprises a personal display device carried by an operator of the agricultural machine or by a technician servicing the agricultural machine.

14. The image-based position assessment method of claim 1 , wherein the agricultural machine comprises a square baler;

wherein the timed mechanism comprises a knotter mechanism of the square baler; and

wherein the traveling component comprises a hay needle contained in the knotter mechanism.

15. The image-based position assessment method of claim 14 , wherein analyzing comprises utilizing the diagnostic camera images to assess whether the recorded time-dependent component position of the hay needle deviates excessively from the expected time-dependent component position based, at least in part, on a time-dependent positioning of a first timing mark provided on the hay needle and visible in the field of view of the diagnostic camera.

16. The image-based position assessment method of claim 14 , wherein analyzing further comprises utilizing the diagnostic camera images to assess whether the recorded time-dependent component position of the hay needle deviates excessively from the expected time-dependent component position further based on a time-dependent positioning of a second timing mark relative to the first timing mark, the second timing mark provided on a second component having an opening through which the hay needle passes when traveling the intended motion path.

17. An agricultural machine, comprising:

a frame;

an agricultural machine mechanism containing a traveling component movably coupled to the frame, the traveling component configured to travel an intended motion path that is non-linear when the agricultural machine mechanism is mechanically driven during operation of the agricultural machine;

a display device; and

a diagnostic camera mount joined to the frame and configured to attach a diagnostic camera to the frame during an image-based position assessment performed to assess whether the traveling component is properly positioned when traveling the intended motion path during operation of the agricultural machine mechanism, the diagnostic camera mount securing the diagnostic camera in a spatially-fixed orientation in which a field of view of the diagnostic camera encompasses at least a portion of the intended motion path;

wherein the image-based position assessment includes:

receiving, at an image processing system, one or more diagnostic camera images captured by the diagnostic camera mounted to the agricultural machine at the diagnostic camera mount and having a field of view at least partially encompassing the intended motion path;

applying, at the image processing system, permissible path boundaries to the diagnostic camera images for comparison to the traveling component of the timed mechanism as taken along the intended motion path;

analyzing, at the image processing system, the diagnostic camera images to determine whether a recorded time-dependent component position of the traveling component deviates beyond the permissible path boundaries from an expected time-dependent component position of the traveling component, as taken along the intended motion path; and

generating or causing generation of a first notification on the display device, via the image processing system, indicating that a positional adjustment to the traveling component is recommended if determining that the recorded time-dependent component position deviates beyond the permissible path boundaries from the expected time-dependent component position.

18. The agricultural machine of claim 17 , further comprising:

a second component contained in the timed mechanism and located adjacent the intended motion path; and

timing marks provided on the traveling component, the second component, or a combination thereof and visible in the field of view of the diagnostic camera when attached to the frame via the diagnostic camera mount.

19. The agricultural machine of claim 17 , wherein the diagnostic camera mount comprises a receptacle in which the diagnostic camera is received when attaching the diagnostic camera to the frame.

20. The agricultural machine of claim 17 , wherein the agricultural machine, the agricultural machine mechanism, and the traveling component comprise a square baler, a knotter mechanism, and a hay needle, respectively.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 16, 2021
From: KRAUS, TIMOTHY J.
To: DEERE & COMPANY
Reel/Frame 055268/0183 →
Continuity (1)
Related Publication 20220261007A1 · Aug 18, 2022