IP Library Granted Patent US 11,756,232
Granted Patent B2
US 11,756,232 · App. 17/960,432 · Granted Sep 12, 2023

Edge-based crop yield prediction

Inventors: Kathleen Watson (Sunnyvale, CA); Jie Yang (Sunnyvale, CA); Yueqi Li (San Jose, CA)
Assignee: MINERAL EARTH SCIENCES LLC
G06T7/97G06T2207/10032G06T2207/20081G06T2207/20084G06T2207/20212G06T2207/30188
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,756,232
App. No.
17/960,432
Granted
Sep 12, 2023
Kind
B2
Abstract

Implementations are described herein for edge-based real time crop yield predictions made using sampled subsets of robotically-acquired vision data. In various implementations, one or more robots may be deployed amongst a plurality of plants in an area such as a field. Using one or more vision sensors of the one or more robots, a superset of high resolution images may be acquired that depict the plurality of plants. A subset of multiple high resolution images may then be sampled from the superset of high resolution images. Data indicative of the subset of high resolution images may be applied as input across a machine learning model, with or without additional data, to generate output indicative of a real time crop yield prediction.

Claims (36)

1. A method for making a real time prediction of an agricultural metric for a crop grown in a field in real time, comprising:

retrieving a superset of high-resolution images that depict a plurality of plants in the field, wherein the superset of high-resolution images are acquired using one or more vision sensors carried by one or more robots;

sampling, by one or more edge computing devices associated with the field, from the acquired superset of high-resolution images, a subset of discrete high-resolution images;

applying, by one or more of the edge computing devices, data indicative of the sampled subset of discrete high-resolution images across a first machine learning model, along with local weather data, to generate output indicative of the real time prediction of the agricultural metric for the field, without applying data indicative of other acquired images of the superset outside of the sampled subset as input across the first machine learning model; and

generating, based on the output using one or more of the edge computing devices, for presentation at one or more computing devices, the real time prediction of the agricultural metric.

2. The method of claim 1 , wherein the agricultural metric comprises crop yield.

3. The method of claim 1 , further comprising processing the subset of discreet high-resolution images using a second machine learning model to generate one or more reduced-dimensionality image embeddings, wherein the data indicative of the sampled subset of discrete high-resolution images comprises the one or more reduced-dimensionality embeddings.

4. The method of claim 3 , wherein the processing comprises applying the subset of high-resolution images as input across a convolutional neural network.

5. The method of claim 4 , further comprising averaging or concatenating a plurality of constituent image embeddings together to form a composite embedding that represents crops across the field.

6. The method of claim 1 , wherein one or more of the sampling, applying, or generating are performed onboard one or more of the robots.

7. The method of claim 1 , further comprising selecting a portion of the field that is more predictive of the agricultural metric than another portion of the field, wherein the subset of discrete high-resolution images are sampled from another subset of the acquired superset of high-resolution images.

8. The method of claim 7 , further comprising:

analyzing one or more satellite images of the field to identify a plurality of distinct portions of the field that correspond to distinct portions of the one or more satellite images having different colors or infrared values;

wherein the portion of the field is selected based on the portions of the one or more satellite images.

9. The method of claim 7 , wherein the selecting is based on relative accuracies of past real time predictions of the agricultural metrics generated from respective sampled subsets of high-resolution images acquired by one or more of the robots while travelling through respective distinct portions of the field.

10. An edge computing system deployed at or near a field, the edge computing system comprising one or more processors and memory storing instructions that, in response to execution of the instructions by the one or more processors, cause the one or more processors to:

retrieve a superset of high-resolution images that depict a plurality of plants in the field, wherein the superset of high-resolution images are acquired using one or more vision sensors carried by one or more robots;

sample, from the acquired superset of high-resolution images, a subset of discrete high-resolution images;

apply data indicative of the sampled subset of discrete high-resolution images across a first machine learning model, along with local weather data, to generate output indicative of a real time prediction of an agricultural metric for the field, without applying data indicative of other acquired images of the superset outside of the sampled subset as input across the first machine learning model; and

generate, based on the output, for presentation at one or more computing devices, the real time prediction of the agricultural metric.

11. The system of claim 10 , wherein the real time prediction of the agricultural metric comprises crop yield.

12. The system of claim 10 , further comprising instructions to process the subset of discreet high-resolution images using a second machine learning model to generate one or more reduced-dimensionality image embeddings, wherein the data indicative of the sampled subset of discrete high-resolution images comprises the one or more reduced-dimensionality embeddings.

13. The system of claim 12 , wherein the instructions to process include instructions to apply the subset of high-resolution images as input across a convolutional neural network.

14. The system of claim 13 , further comprising instructions to average or concatenate a plurality of constituent image embeddings together to form a composite embedding that represents crops across the field.

15. The system of claim 10 , further comprising instructions to select a portion of the field that is more predictive of the field's agricultural metric as a whole than another portion of the field, wherein the subset of discrete high-resolution images are sampled from another subset of the acquired superset of high-resolution images.

16. The system of claim 15 , further comprising instructions to:

analyze one or more satellite images of the field to identify a plurality of distinct portions of the field that correspond to distinct portions of the one or more satellite images having different colors or infrared values;

wherein the portion of the field is selected \based on the portions of the one or more satellite images.

17. The system of claim 15 , wherein the portion of the field is selected based on relative accuracies of past real time predictions of the agricultural metrics generated from respective sampled subsets of high-resolution images acquired by one or more of the robots while travelling through respective distinct portions of the field.

18. At least one non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to:

retrieve a superset of high-resolution images that depict a plurality of plants in a field, wherein the superset of high-resolution images are acquired using one or more vision sensors carried by one or more robots;

sample, by one or more edge computing devices associated with the field, from the acquired superset of high-resolution images, a subset of discrete high-resolution images;

apply, by one or more of the edge computing devices, data indicative of the sampled subset of discrete high-resolution images across a first machine learning model, along with local weather data, to generate output indicative of a real time prediction of an agricultural metric for the field, without applying data indicative of other acquired images of the superset outside of the sampled subset as input across the first machine learning model; and

generating, based on the output using one or more of the edge computing devices, for presentation at one or more computing devices, the real time prediction of the agricultural metric.

19. The at least one non-transitory computer-readable medium of claim 18 , wherein the agricultural metric comprises crop yield.

20. The at least one non-transitory computer-readable medium of claim 10 , further comprising instructions to process the subset of discreet high-resolution images using a second machine learning model to generate one or more reduced-dimensionality image embeddings, wherein the data indicative of the sampled subset of discrete high-resolution images comprises the one or more reduced-dimensionality embeddings.

Assignments (3)
MERGER Recorded Jun 26, 2024
From: MINERAL EARTH SCIENCES LLC
To: DEERE & CO.
Reel/Frame 068055/0420 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 24, 2023
From: X DEVELOPMENT LLC
To: MINERAL EARTH SCIENCES LLC
Reel/Frame 062850/0575 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 6, 2022
From: WATSON, KATHLEEN; YANG, JIE; LI, YUEQI
To: X DEVELOPMENT LLC
Reel/Frame 061334/0476 →