IP Library Granted Patent US 11,403,846
Granted Patent B2
US 11,403,846 · App. 17/138,737 · Granted Aug 2, 2022

Crop boundary detection in images

Inventors: Cheng-en Guo (Santa Clara, CA); Jie Yang (Sunnyvale, CA); Elliott Grant (Woodside, CA)
Assignee: X Development LLC
G06V20/188A01B79/005G06N3/08G06Q50/02G06T7/11G06T2207/20081G06T2207/30188G06V20/68
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,403,846
App. No.
17/138,737
Granted
Aug 2, 2022
Kind
B2
Abstract

In embodiments, obtaining a plurality of image sets associated with a geographical region and a time period, wherein each image set of the plurality of image sets comprises multi-spectral and time series images that depict a respective particular portion of the geographical region during the time period, and predicting presence of a crop at particular locations within the particular portion of the geographical region associated with an image set of the plurality of image sets. Determining crop boundary locations within the particular portion of the geographical region based on the predicted presence of the crop at the particular locations, and generating a crop indicative image comprising at least one image of the multi-spectral and time series images of the image set overlaid with indication of crop areas, wherein the crop areas are defined by the determined crop boundary locations.

Claims (38)

1. A method comprising:

receiving, by a computing device, input containing one or more search parameters, wherein the one or more search parameters include one or more of a latitude, a longitude, a county, a size, a shape, and an identifier;

presenting, by the computing device, an image depicting a geographical region, wherein the image is selected based on the one or more search parameters; and

presenting, by the computing device, indications of crop areas overlaying the image, wherein the crop areas are defined by crop boundary locations determined by:

obtaining a plurality of image sets associated with the geographical region and a time period, wherein each image set of the plurality of image sets comprises multi-spectral and time series images that depict a respective particular portion of the geographical region during the time period;

determining probabilities that pixels of images in an image set of the plurality of image sets depict a crop versus a non-crop; and

determining crop boundary locations within the particular portion of the geographical region based on the determined probabilities.

2. The method of claim 1 , further comprising providing a user interface through which crop boundary locations are capable of being manually modified by a user;

receiving crop boundary location modifications from the user; and

updating a database storing the image and the crop areas with the crop boundary location modifications.

3. The method of claim 1 , wherein the crop boundary locations are determined at a sub-meter ground resolution.

4. The method of claim 1 , wherein determining the crop boundary locations includes determining the crop boundary locations based on the determined probabilities and one or more of application of prior knowledge of physical structures or agricultural practices proximate to particular portions of the geographical region associated with the image set, smoothing of crop boundary location pixels, and application of non-supervised clustering and region growing techniques.

5. The method of claim 1 , wherein determining the probabilities that pixels of images in an image set of the plurality of image sets depict the crop versus the non-crop comprises applying the image set to one or more machine learning systems configured to determine probabilities that pixels of images in image sets depict the crop versus the non-crop after supervised training on ground truth data.

6. The method of claim 5 , wherein the one or more machine learning systems include a convolutional neural network (CNN).

7. The method of claim 5 , wherein the ground truth data comprises one or more of government crop data, publicly available crop data, images with crop areas identified at low ground resolution, images with manually identified crop boundaries, crop survey data, sampled crop data, and farmer reports.

8. The method of claim 5 , wherein one or both of a first geographical region and a first time period associated with the ground truth data matches the geographical region and the time period associated with the plurality of image sets.

9. The method of claim 5 , wherein one or both of a first geographical region and a first time period associated with the ground truth data is different from the geographical region and the time period associated with the plurality of image sets.

10. The method of claim 1 , wherein the crop boundary locations are determined by further:

prior to determining the probabilities, filtering the plurality of image sets of one or more of clouds, shadows, haze, fog, and atmospheric obstructions, and wherein determining the probabilities that the pixels of the images in the image set of the plurality of image sets depict the crop versus the non-crop comprises using a filtered image set associated with the image set.

11. A non-transitory computer-readable medium having logic stored thereon that, in response to execution by one or more processors of a computing device, causes the computing device to perform actions comprising:

receiving, by the computing device, input containing one or more search parameters, wherein the one or more search parameters include one or more of a latitude, a longitude, a county, a size, a shape, and an identifier;

presenting, by the computing device, an image depicting a geographical region, wherein the image is selected based on the one or more search parameters; and

presenting, by the computing device, indications of crop areas overlaying the image, wherein the crop areas are defined by crop boundary locations determined by:

obtaining a plurality of image sets associated with the geographical region and a time period, wherein each image set of the plurality of image sets comprises multi-spectral and time series images that depict a respective particular portion of the geographical region during the time period;

determining probabilities that pixels of images in an image set of the plurality of image sets depict a crop versus a non-crop; and

determining crop boundary locations within the particular portion of the geographical region based on the determined probabilities.

12. The computer-readable medium of claim 11 , wherein the actions further comprise providing a user interface through which crop boundary locations are capable of being manually modified by a user;

receiving crop boundary location modifications from the user; and

updating a database storing the image and the crop areas with the crop boundary location modifications.

13. The computer-readable medium of claim 11 , wherein the crop boundary locations are determined at a sub-meter ground resolution.

14. The computer-readable medium of claim 11 , wherein determining the crop boundary locations includes determining the crop boundary locations based on the determined probabilities and one or more of application of prior knowledge of physical structures or agricultural practices proximate to particular portions of the geographical region associated with the image set, smoothing of crop boundary location pixels, and application of non-supervised clustering and region growing techniques.

15. The computer-readable medium of claim 11 , wherein determining the probabilities that pixels of images in an image set of the plurality of image sets depict the crop versus the non-crop comprises applying the image set to one or more machine learning systems configured to determine probabilities that pixels of images in image sets depict the crop versus the non-crop after supervised training on ground truth data.

16. The computer-readable medium of claim 15 , wherein the one or more machine learning systems include a convolutional neural network (CNN).

17. The computer-readable medium of claim 15 , wherein the ground truth data comprises one or more of government crop data, publicly available crop data, images with crop areas identified at low ground resolution, images with manually identified crop boundaries, crop survey data, sampled crop data, and farmer reports.

18. The computer-readable medium of claim 15 , wherein one or both of a first geographical region and a first time period associated with the ground truth data matches the geographical region and the time period associated with the plurality of image sets.

19. The computer-readable medium of claim 15 , wherein one or both of a first geographical region and a first time period associated with the ground truth data is different from the geographical region and the time period associated with the plurality of image sets.

20. The computer-readable medium of claim 11 , wherein the crop boundary locations are determined by further:

prior to determining the probabilities, filtering the plurality of image sets of one or more of clouds, shadows, haze, fog, and atmospheric obstructions, and wherein determining the probabilities that the pixels of the images in the image set of the plurality of image sets depict the crop versus the non-crop comprises using a filtered image set associated with the image set.

Assignments (3)
MERGER Recorded Jun 26, 2024
From: MINERAL EARTH SCIENCES LLC
To: DEERE & CO.
Reel/Frame 068055/0420 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 2, 2023
From: GUO, CHENG-EN; YANG, JIE; GRANT, ELLIOT
To: X DEVELOPMENT LLC
Reel/Frame 062863/0338 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 27, 2023
From: X DEVELOPMENT LLC
To: MINERAL EARTH SCIENCES LLC
Reel/Frame 062516/0456 →
Cited By (1)
US 12,548,131