IP Library Granted Patent US 12,369,513
Granted Patent B2
US 12,369,513 · App. 17/669,030 · Granted Jul 29, 2025

Obtaining and augmenting agricultural data and generating an augmented display

Inventors: Noel W. Anderson (Fargo, ND); Cary S. Hubner (Geneseo, IL); Brandon M. McDonald (Johnston, IA); Nathan Ogden (Mahomet, IL); Brittany Sprabery (Woodinville, WA); Sean A. Mahrt (Le Claire, IA); Sarah L. Schinckel (Ankeny, IA); Mark D. Moran (Urbana, IL)
Assignee: Deere & Company
A01B79/005A01B69/008A01B79/02G06F3/0488G06T7/0014H04W4/185G06T2200/24G06T2207/10024G06T2207/30188G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,369,513
App. No.
17/669,030
Granted
Jul 29, 2025
Kind
B2
Abstract

A geographic position of an agricultural machine is captured. Agricultural data is received that corresponds to a geographic position. Georeferenced visual indicia are displayed that are indicative of the received agricultural data.

Claims (65)

1. A computer-implemented method of automatically controlling an agricultural system, comprising:

receiving predictive visual data indicative of an expected visualization of a geographic location on an agricultural worksite;

receiving an actual visual representation of the geographic location;

comparing the expected visualization of the geographic location to the actual visual representation of the geographic location to identify an anomaly;

identifying a severity level of the identified anomaly:

generating a geographically referenced anomaly indicator indicative of the severity level of the identified anomaly;

generating an enhanced display, the enhanced display showing the geographic location enhanced with the geographically referenced anomaly indicator; and

automatically controlling an agricultural machine in the agricultural system based on the geographically referenced anomaly indicator.

2. The computer-implemented method of claim 1 and further comprising:

detecting a user interaction with the enhanced display;

processing the detected user interaction to identify additional anomaly information corresponding to the identified anomaly; and

modifying the enhanced display with the additional anomaly information.

3. The computer-implemented method of claim 2 wherein processing the detected user interaction comprises:

performing diagnostic processing to identify, as the additional anomaly information, a diagnostic factor contributing to a possible cause of the anomaly; and

generating a diagnostic factor display element, wherein modifying the enhanced display comprises adding the diagnostic factor display element to the enhanced display.

4. The computer-implemented method of claim 2 wherein processing the detected user interaction comprises:

performing prognosis processing to identify remedial information indicative of a possible remedy for the anomaly; and

generating a possible remedy display element, wherein modifying the enhanced display comprises adding the possible remedy display element to the enhanced display.

5. The computer-implemented method of claim 4 wherein the identified anomaly corresponds to a crop characteristic anomaly, and where the severity level of the identified anomaly corresponds to a severity level of the crop characteristic anomaly.

6. The computer-implemented method of claim 1 wherein identifying the severity level of the identified anomaly comprises:

generating a metric value indicative of a measure of difference between the expected visualization and the actual visual representation of the geographic location;

comparing the metric value to a metric threshold value to obtain a metric difference value; and

identifying the severity level of the identified anomaly based on the metric difference value.

7. The computer-implemented method of claim 1 wherein receiving the predictive visual data comprises:

identifying a neighboring geographic location adjacent the geographic location; and

generating the predictive visual data based on a visualization of the neighboring geographic location.

8. The computer-implemented method of claim 1 wherein generating the geographically referenced anomaly indicator comprises:

generating a geometric shaped display element, a geometric shape of the geometric shaped display element being indicative of the geographically referenced anomaly indicator.

9. The computer-implemented method of claim 1 wherein generating the geographically referenced anomaly indicator comprises:

generating a display element having visual indicia, the visual indicia of the display element being indicative of the geographically referenced anomaly indicator.

10. The computer-implemented method of claim 1 wherein receiving predictive visual data indicative of the expected visualization of the geographic location on the agricultural worksite comprises:

obtaining a prior state of the geographic location and prior conditions that affect the prior state;

generating a predictive state of the geographic location based on the prior state and the prior conditions; and

generating the predictive visual data based on the predictive state of the geographic location.

11. The computer-implemented method of claim 1 wherein generating the enhanced display showing the geographic location includes simultaneously showing the geographically referenced anomaly indicator and a data display including geographically referenced planting characteristics.

12. An agricultural system, comprising:

an expected state identification system identifying predictive visual data indicative of an expected visualization of a geographic location on an agricultural worksite;

a comparison system receiving an actual visual representation of the geographic location, the comparison system configured to generate a metric value indicative of a measure of difference between the expected visualization of the geographic location and the actual visual representation of the geographic location to generate a comparison result;

an anomaly identifier identifying an anomaly based on the comparison result;

an anomaly output system generating a geographically referenced anomaly indicator indicative of the anomaly;

a display generation system generating an enhanced display, the enhanced display showing the geographic location enhanced with the geographically referenced anomaly indicator; and

a control system configured to automatically control an agricultural machine in the agricultural system based on the geographically referenced anomaly indicator shown in the enhanced display.

13. The agricultural system of claim 12 and further comprising:

a user interaction detector configured to detect a user interaction with the enhanced display;

a workflow processing system configured to process the detected user interaction to identify additional anomaly information corresponding to the identified anomaly; and

a display output system configured to modify the enhanced display with the additional anomaly information.

14. The agricultural system of claim 13 wherein the workflow processing system comprises:

a diagnostic system configured to perform diagnostic processing to identify, as the additional anomaly information, a diagnostic factor contributing to a possible cause of the anomaly and generate a diagnostic factor display element, the display generation system modifying the enhanced display by adding the diagnostic factor display element to the enhanced display.

15. The agricultural system of claim 14 wherein the workflow processing system comprises:

a prognosis system configured to perform prognosis processing to identify remedial information indicative of a possible remedy for the anomaly and generate a possible remedy display element, the display generation system modifying the enhanced display by adding the possible remedy display element to the enhanced display.

16. The agricultural system of claim 12 wherein the comparison system is configured to compare the metric value to a metric threshold value to obtain a metric difference value and wherein the anomaly identifier is configured to identify a severity level of the identified anomaly based on the comparison result.

17. The agricultural system of claim 12 wherein the display generation system is configured to generate, as the geographically referenced anomaly indicator, a geometric shaped display element, a geometric shape of the geometric shaped display element being indicative of the geographically referenced anomaly indicator.

18. The agricultural system of claim 12 wherein the display generation system is configured to generate, as the geographically referenced anomaly indicator a colored display element, a color of the colored display element being indicative of the geographically referenced anomaly indicator.

19. The agricultural system of claim 12 and further comprising:

a prior state identification system configured to identify a prior state of the geographic location;

a prior condition identification system configured to identify prior conditions that affect the prior state; and

an expected state identification system configured to generate a predictive state of the geographic location based on the prior state and prior conditions.

20. A computing system, comprising:

non-transitory computer readable medium containing computer instructions stored therein for causing a computer processor to:

obtain predictive visual data indicative of an expected visualization of a geographic location on an agricultural worksite;

obtain an actual visual representation of the geographic location;

compare the expected visualization of the geographic location to the actual visual representation of the geographic location to identify an anomaly;

generate a geographically referenced anomaly indicator indicative of the anomaly;

generate an enhanced display showing the geographic location enhanced with a relationship between the geographically referenced anomaly indicator and seed singulation data; and

automatically control an agricultural machine based on the geographically referenced anomaly indicator shown in the enhanced display.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 11, 2022
From: ANDERSON, NOEL W.; HUBNER, CARY S.; MCDONALD, BRANDON M.; OGDEN, NATHAN; SPRABERY, BRITTANY; MAHRT, SEAN A.; SCHINCKEL, SARAH L.; MORAN, MARK D.
To: DEERE & COMPANY
Reel/Frame 058986/0939 →
Continuity (2)
Provisional Application 63231870 · Aug 11, 2021
Related Publication 20230046844A1 · Feb 16, 2023
References Cited (87)
US 6553312B2 · Upadhyaya et al. · 2003 [cited by applicant]
US 7362738B2 · Taube et al. · 2008 [cited by applicant]
US 8150617B2 · Manber · 2012 [cited by examiner]
US 8321365B2 · Anderson · 2012 [cited by applicant]
US 9058633B2 · Lindores · 2015 [cited by examiner]
US 9282693B2 · Anderson · 2016 [cited by applicant]
US 9668420B2 · Andersan et al. · 2017 [cited by applicant]
US 9740208B2 · Sugumaran et al. · 2017 [cited by applicant]
US 9861040B2 · Bonefas · 2018 [cited by examiner]
US 10127725B2 · Kahler et al. · 2018 [cited by applicant]
US 10242499B2 · Bryson et al. · 2019 [cited by applicant]
US 10577103B2 · Cantrell et al. · 2020 [cited by applicant]
US 10638667B2 · Regan et al. · 2020 [cited by applicant]
US 10664518B2 · McKinnon et al. · 2020 [cited by applicant]
US 10666856B1 · Ruechker · 2020 [cited by applicant]
US 10679056B2 · Zemenchik · 2020 [cited by applicant]
US 10719709B2 · Zemenchik · 2020 [cited by applicant]
US 10740987B2 · Scott et al. · 2020 [cited by applicant]
US 10761544B2 · Anderson et al. · 2020 [cited by applicant]
US 10806074B2 · Zemenchik · 2020 [cited by applicant]
US 10820478B1 · Ferrari et al. · 2020 [cited by applicant]
US 10885675B1 · Shulman et al. · 2021 [cited by applicant]
US 10888399B2 · Kopelman et al. · 2021 [cited by applicant]
US 10896218B2 · Reyes Martinez et al. · 2021 [cited by applicant]
US 10930065B2 · Grant et al. · 2021 [cited by applicant]
US 10957036B2 · Stueve et al. · 2021 [cited by applicant]
US 20150187109A1 · Mentzer · 2015 [cited by examiner]
US 20170088147A1 · Tentinger et al. · 2017 [cited by applicant]
US 20170090196A1 · Hendron · 2017 [cited by applicant]
US 20170113610A1 · Payne · 2017 [cited by applicant]
US 20170131718A1 · Matsumura et al. · 2017 [cited by applicant]
US 20180224802A1 · Vasavada et al. · 2018 [cited by applicant]
US 20180271027A1 · Funabashi · 2018 [cited by applicant]
US 20190141883A1 · Zemenchik · 2019 [cited by applicant]
US 20190228224A1 · Guo et al. · 2019 [cited by applicant]
US 20190244428A1 · Greenberg et al. · 2019 [cited by applicant]
US 20200107490A1 · Zemenchik · 2020 [cited by applicant]
US 20200146203A1 · Deng · 2020 [cited by applicant]
US 20200150848A1 · Deng · 2020 [cited by applicant]
US 20200230633A1 · Serrat et al. · 2020 [cited by applicant]
US 20200281110A1 · McNichols et al. · 2020 [cited by applicant]
US 20200329632A1 · Yagyu et al. · 2020 [cited by applicant]
US 20200352082A1 · Maeder et al. · 2020 [cited by applicant]
US 20200352088A1 · Arnett et al. · 2020 [cited by applicant]
US 20210045362A1 · Gritzman et al. · 2021 [cited by applicant]
US 20210056338A1 · Padwick et al. · 2021 [cited by applicant]
US 20210084805A1 · Cozza et al. · 2021 [cited by applicant]
US 20210089027A1 · Dasika · 2021 [cited by applicant]
US 20210185885A1 · Sibley et al. · 2021 [cited by applicant]
US 20210243939A1 · Strnad et al. · 2021 [cited by applicant]
US 20220101554A1 · Fu et al. · 2022 [cited by applicant]
US 20220132828A1 · Kwak et al. · 2022 [cited by applicant]
US 20220256764A1 · Henry et al. · 2022 [cited by applicant]
US 20230000015A1 · Herrmann et al. · 2023 [cited by applicant]
US 20230028706A1 · Watson et al. · 2023 [cited by applicant]
US 20230112376A1 · Strnad et al. · 2023 [cited by applicant]
US 20230136009A1 · Faers et al. · 2023 [cited by applicant]
US 20230343090A1 · Khait et al. · 2023 [cited by applicant]
US 20230371492A1 · Houis · 2023 [cited by applicant]
US 20240295954A1 · Stoller et al. · 2024 [cited by applicant]
CN 107181323A · 2017 [cited by applicant]
CN 108257145 · 2018 [cited by applicant]
DE 102019114667A1 · 2020 [cited by applicant]
EP 3418956A4 · 2018 [cited by applicant]
JP 2871609 · 1999 [cited by applicant]
JP 6512463 · 2019 [cited by applicant]
WO WO2012142395A1 · 2012 [cited by applicant]
WO 2017024358A1 · 2017 [cited by applicant]
WO 2017061281A1 · 2017 [cited by applicant]
WO 2018048708A1 · 2018 [cited by applicant]
WO 2019000325 · 2019 [cited by applicant]
WO 20006236A1 · 2020 [cited by applicant]
WO WO2021081428A1 · 2021 [cited by applicant]
https://www.youtube.com/watch?v=Pu5pXadjtFc&t=3333s, 43:00-53:00 minutes—submitted as a transcript. Hyundai x CES | Hyundai Press Conference at CES 2022 61,669 views Streamed live on Jan. 4, 2022. [cited by applicant]
Hridja, Augmented Reality in Agriculture, Dec. 26, 2019, https://www.queppelin.com/augmented-reality-in-agriculture/, 11 pages. [cited by applicant]
Aimee Chanthadavong, AI and gamification being used to help Aussie farmers reduce spray drift impact. May 2, 2021, 5 pages. [cited by applicant]
Farmwise.io website, Apr. 28, 2021, 6 pages. [cited by applicant]
https://www.agritecture.com/blog/2018/11/27/the-hidden-potential-of-augmented-reality-in-farming, Nov. 27, 2018, 5 pages. [cited by applicant]
https://www.precisionplanting.com/products/product/smartdepth, Accessed on Feb. 2, 2022, 10 pages. [cited by applicant]
https://www.farmworks.com/files/manuals/FW_View_English.pdf, Accessed on Feb. 2, 2022. 52 pages. [cited by applicant]
https://www.ssequip.com/pdf/20-20-seed-sense-overview.pdf, Accessed on Feb. 2, 2022, 6 pages. [cited by applicant]
Suprem et al. “A review on application of technology systems, standards and interfaces for agriculture and food sector”, Computer Standards & Interfaces vol. 35, Issue 4, Jun. 2013, pp. 355-364. [cited by applicant]
U.S. Appl. No. 17/869,027 Non Final Office Action dated Aug. 10, 2023, 51 pages. [cited by applicant]
U.S. Appl. No. 17/669,027 Final Office Action dated Feb. 8, 2024, 99 pages. [cited by applicant]
Extended European Search Report and Written Opinion issued in European Patent Application No. 22182494.9, dated Dec. 2, 2022, in 07 pages. [cited by applicant]
Extended European Search Report and Written Opinion issued in European Patent Application No. 22182493.1, dated Nov. 24, 2022, in 07 pages. [cited by applicant]
Extended European Search Report and Written Opinion issued in European Patent Application No. 22182492.3, dated Dec. 1, 2022, in 07 pages. [cited by applicant]