IP Library Granted Patent US 12,315,249
Granted Patent B2
US 12,315,249 · App. 18/084,468 · Granted May 27, 2025

Remote sensing interpretation

Inventors: Hongxu Ma (San Jose, CA); Yujing Qian (Mountain View, CA); Yuchi Ma (Madison, WI); Yuanyuan Tian (Sunnyvale, CA)
Assignee: Deere & Company
G06V20/188G06N5/04G06N20/00G06Q50/02G06V20/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,315,249
App. No.
18/084,468
Granted
May 27, 2025
Kind
B2
Abstract

Implementations are described herein for obtaining a sequence of high-elevation images capturing a particular geographic area during a particular time period in a plurality of spectral bands; applying the sequence of high-elevation images as input to upstream machine learning model(s) to generate remote sensing embeddings indicating terrain feature(s) of the particular geographic area; applying agricultural data obtained from a local agricultural knowledge graph as input to additional upstream machine learning model(s) to generate agricultural knowledge embedding(s); inferring a natural-language description of a status of the particular geographic area based on generating: an aggregate representative embedding that semantically represents a plurality of agricultural conditions of the particular geographic area, and a natural-language description of one or more of the plurality of agricultural conditions of the particular area using a large language model; and causing a user device associated with an agricultural entity to present the natural-language description of the status.

Claims (53)

1. A method, comprising:

obtaining a sequence of high-elevation images capturing a particular geographic area during a particular time period, wherein the sequence of high-elevation images includes a plurality of images capturing the particular geographic area in a plurality of spectral bands;

applying the sequence of high-elevation images as input to one or more upstream machine learning models to generate a plurality of remote sensing embeddings indicating one or more terrain features of the particular geographic area remotely sensed based on processing the sequence of high-elevation images;

applying agricultural data obtained from a local agricultural knowledge graph associated with the particular geographic area as input to one or more additional upstream machine learning models to generate one or more agricultural knowledge embeddings;

inferring a natural-language description of a status of the particular geographic area, wherein the inferring includes:

applying the plurality of remote sensing embeddings and the one or more agricultural knowledge embeddings as input to one or more downstream machine learning models to generate, as output, an aggregate representative embedding that semantically represents a plurality of agricultural conditions of the particular geographic area; and

generating, using a large language model, the natural-language description of one or more of the plurality of agricultural conditions of the particular geographic area; and

causing a user device associated with an agricultural entity to present the natural-language description of the status of the particular geographic area to the agricultural entity.

2. The method of claim 1 , wherein the sequence of high-elevation images includes a plurality of timestamps each associated with a respective image of the plurality of images, and wherein applying the sequence of high-elevation images as input to the one or more upstream machine learning models to generate the plurality of remote sensing embeddings indicating the one or more terrain features of the particular geographic area remotely sensed based on processing the sequence of high-elevation images includes:

preprocessing the sequence of high-elevation images to generate:

a plurality of spectral embeddings each associated with a respective spectral band of the plurality of spectral bands; and

a plurality of temporal embeddings each associated with a respective timestamp of the plurality of timestamps; and

applying the plurality of spectral embeddings and the plurality of temporal embeddings as input to the one or more upstream machine learning models to generate the plurality of remote sensing embeddings.

3. The method of claim 2 , wherein applying the plurality of spectral embeddings and the plurality of temporal embeddings as input to the one or more upstream machine learning models includes applying the plurality of spectral embeddings and the plurality of temporal embeddings to at least one first machine learning model included in the one or more upstream machine learning models to generate, as output, synthetic image data, wherein the synthetic image data includes synthetic spectral band data or synthetic temporal data not included in the sequence of high-elevation images.

4. The method of claim 3 , wherein applying the plurality of spectral embeddings and the plurality of temporal embeddings as input to the one or more upstream machine learning models includes applying, as input to a remote sensing machine learning model included in the one or more upstream machine learning models, the synthetic image data and the sequence of high-elevation images to generate the plurality of remote sensing embeddings.

5. The method of claim 3 , wherein inferring the natural-language description of the status of the particular geographic area further includes applying, as additional input to the one or more downstream machine learning models, the synthetic image data to generate the one or more inferred embeddings.

6. The method of claim 2 , wherein the plurality of spectral embeddings indicate temporal data included in the sequence of high-elevation images for each particular spectral band corresponding to the plurality of images, and wherein the plurality of temporal embeddings indicate spectral data included in the sequence of high-elevation images for each particular point in time corresponding to the plurality of timestamps.

7. The method of claim 2 , wherein the agricultural data is obtained from the local agricultural knowledge graph based on identifying one or more nodes, or one or more edges connecting the one or more nodes, included in the local agricultural knowledge graph that are associated with at least one spectral embedding of the plurality of spectral embeddings or at least one temporal embedding of the plurality of temporal embeddings.

8. The method of claim 1 , wherein generating the natural-language description of the status of the particular geographic area is performed based on user input of the agricultural entity.

9. A system comprising one or more processors and non-transitory memory storing instructions that, in response to execution by the one or more processors, cause the one or more processors to:

obtain a sequence of high-elevation images capturing a particular geographic area during a particular time period, wherein the sequence of high-elevation images includes a plurality of images capturing the particular geographic area in a plurality of spectral bands;

apply the sequence of high-elevation images as input to one or more upstream machine learning models to generate a plurality of remote sensing embeddings indicating one or more terrain features of the particular geographic area remotely sensed based on processing the sequence of high-elevation images;

apply agricultural data obtained from a local agricultural knowledge graph associated with the particular geographic area as input to one or more additional upstream machine learning models to generate one or more agricultural knowledge embeddings;

infer a natural-language description of a status of the particular geographic area, wherein the instructions to infer include instructions to:

apply the plurality of remote sensing embeddings and the one or more agricultural knowledge embeddings as input to one or more downstream machine learning models to generate, as output, an aggregate representative embedding that semantically represents a plurality of agricultural conditions of the particular geographic area; and

generate, using a large language model, the natural-language description of one or more of the plurality of agricultural conditions of the particular geographic area; and

cause a user device associated with an agricultural entity to present the natural-language description of the status of the particular geographic area to the agricultural entity.

10. The system of claim 9 , wherein the sequence of high-elevation images includes a plurality of timestamps each associated with a respective image of the plurality of images, and wherein the instructions to apply the sequence of high-elevation images as input to the one or more upstream machine learning models to generate the plurality of remote sensing embeddings include instructions to:

preprocess the sequence of high-elevation images to generate:

a plurality of spectral embeddings each associated with a respective spectral band of the plurality of spectral bands; and

a plurality of temporal embeddings each associated with a respective timestamp of the plurality of timestamps; and

apply the plurality of spectral embeddings and the plurality of temporal embeddings as input to the one or more upstream machine learning models to generate the plurality of remote sensing embeddings.

11. The system of claim 10 , wherein the instructions to apply the plurality of spectral embeddings and the plurality of temporal embeddings as input to the one or more upstream machine learning models include instructions to apply the plurality of spectral embeddings and the plurality of temporal embeddings to at least one first machine learning model included in the one or more upstream machine learning models to generate, as output, synthetic image data, wherein the synthetic image data includes synthetic spectral band data or synthetic temporal data not included in the sequence of high-elevation images.

12. The system of claim 11 , wherein the instructions to apply the plurality of spectral embeddings and the plurality of temporal embeddings as input to the one or more upstream machine learning models include instructions to apply, as input to a remote sensing machine learning model included in the one or more upstream machine learning models, the synthetic image data and the sequence of high-elevation images to generate the plurality of remote sensing embeddings.

13. The system of claim 11 , wherein the instructions to infer the natural-language description of the status of the particular geographic area further include instructions to apply, as additional input to the one or more downstream machine learning models, the synthetic image data to generate the one or more inferred embeddings.

14. The system of claim 10 , wherein the plurality of spectral embeddings indicate temporal data included in the sequence of high-elevation images for each particular spectral band corresponding to the plurality of images, and wherein the plurality of temporal embeddings indicate spectral data included in the sequence of high-elevation images for each particular point in time corresponding to the plurality of timestamps.

15. The system of claim 10 , wherein the agricultural data is obtained from the local agricultural knowledge graph based on identifying one or more nodes, or one or more edges connecting the one or more nodes, included in the local agricultural knowledge graph that are associated with at least one spectral embedding of the plurality of spectral embeddings or at least one temporal embedding of the plurality of temporal embeddings.

16. The system of claim 9 , wherein the natural-language description of the status of the particular geographic area is generated based on user input of the agricultural entity.

17. At least one non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to:

obtain a sequence of high-elevation images capturing a particular geographic area during a particular time period, wherein the sequence of high-elevation images includes a plurality of images capturing the particular geographic area in a plurality of spectral bands;

apply the sequence of high-elevation images as input to one or more upstream machine learning models to generate a plurality of remote sensing embeddings indicating one or more terrain features of the particular geographic area remotely sensed based on processing the sequence of high-elevation images;

apply agricultural data obtained from a local agricultural knowledge graph associated with the particular geographic area as input to one or more additional upstream machine learning models to generate one or more agricultural knowledge embeddings;

infer a natural-language description of a status of the particular geographic area, wherein the instructions to infer include instructions to:

apply the plurality of remote sensing embeddings and the one or more agricultural knowledge embeddings as input to one or more downstream machine learning models to generate, as output, an aggregate representative embedding that semantically represents a plurality of agricultural conditions of the particular geographic area; and

generate, using a large language model, the natural-language description of one or more of the plurality of agricultural conditions of the particular geographic area; and

cause a user device associated with an agricultural entity to present the natural-language description of the status of the particular geographic area to the agricultural entity.

18. The at least one non-transitory computer-readable medium of claim 17 , wherein the sequence of high-elevation images includes a plurality of timestamps each associated with a respective image of the plurality of images, and wherein the instructions to apply the sequence of high-elevation images as input to the one or more upstream machine learning models to generate the plurality of remote sensing embeddings include instructions to:

preprocess the sequence of high-elevation images to generate:

a plurality of spectral embeddings each associated with a respective spectral band of the plurality of spectral bands; and

a plurality of temporal embeddings each associated with a respective timestamp of the plurality of timestamps; and

apply the plurality of spectral embeddings and the plurality of temporal embeddings as input to the one or more upstream machine learning models to generate the plurality of remote sensing embeddings.

19. The at least one non-transitory computer-readable medium of claim 18 , wherein the instructions to apply the plurality of spectral embeddings and the plurality of temporal embeddings as input to the one or more upstream machine learning models include instructions to apply the plurality of spectral embeddings and the plurality of temporal embeddings to at least one first machine learning model included in the one or more upstream machine learning models to generate, as output, synthetic image data, wherein the synthetic image data includes synthetic spectral band data or synthetic temporal data not included in the sequence of high-elevation images.

20. The at least one non-transitory computer-readable medium of claim 19 , wherein the instructions to apply the plurality of spectral embeddings and the plurality of temporal embeddings as input to the one or more upstream machine learning models include instructions to apply, as input to a remote sensing machine learning model included in the one or more upstream machine learning models, the synthetic image data and the sequence of high-elevation images to generate the plurality of remote sensing embeddings.

Assignments (4)
MERGER Recorded Jun 26, 2024
From: MINERAL EARTH SCIENCES LLC
To: DEERE & CO.
Reel/Frame 068055/0420 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE PREVIOUSLY RECORDED ON REEL 062797 FRAME 0625. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Apr 27, 2023
From: MA, HONGXU; QIAN, YUJING; MA, YUCHI; TIAN, YUANYUAN
To: MINERAL EARTH SCIENCES LLC
Reel/Frame 063902/0695 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 24, 2023
From: MA, HONGXU; QIAN, YUJING; MA, YUCHI; TIAN, YUANYUAN
To: X DEVELOPMENT LLC
Reel/Frame 062797/0625 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 24, 2023
From: X DEVELOPMENT LLC
To: MINERAL EARTH SCIENCES LLC
Reel/Frame 062850/0575 →