IP Library Granted Patent US 12,089,110
Granted Patent B2
US 12,089,110 · App. 18/369,770 · Granted Sep 10, 2024

Hybrid vision system for crop land navigation

Inventor: Mark Young (San Francisco, CA)
Assignee: CLIMATE LLC
H04W4/021G05D1/0246G06F18/2413G06N20/00G06V10/87G06V20/10G06V20/188G06V20/56G06V30/2504
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,089,110
App. No.
18/369,770
Filed
Sep 18, 2023
Granted
Sep 10, 2024
Kind
B2
Art Unit
2645
USPC
382/110
Abstract

Autonomous vehicles with global positioning systems are used for field inspection. A vehicle may be programmed to traverse a field, while using sensors to detect objects in the field, and then capture low-resolution images of the objects. Machine vision techniques are used with the low-resolution images to recognize the objects as crops, non-crop plant material or undefined objects. Location data is used to correlate recognized objects with digitally stored field maps to resolve whether a particular object is in a location at which crop planting is expected or not expected. Depending on whether an object in a low-resolution digital image is recognized as a crop, and whether the object is in an expected geo-location for crops, the vehicle may switch to a second image capture mode, for example, capturing a high-resolution image of the object, and/or execute a disease analysis and/or weed analysis on the images of the objects.

Claims (64)

1. A method for determining a specific agricultural product type using autonomous mobile equipment, the method comprising:

obtaining digital image data from a digital camera on autonomous mobile equipment configured to traverse an agricultural field, the digital image data representing images of one or more plants in the agricultural field and one or more images of a specific plant to be classified, wherein the specific plant to be classified is planted at one or more field geolocations of the agricultural field and is associated with a specific agricultural product type to be determined;

receiving specific field space geolocation data representing a specific field space geolocation of the autonomous mobile equipment in the agricultural field;

comparing the specific agricultural product type to one or more pre-classified product types based on: a plant growth characteristic of the specific agricultural product type and a pre-classified plant growth characteristic corresponding to the pre-classified product type to which the specific agricultural product type matches;

comparing the specific field space geolocation to one or more pre-classified plant geolocations; and

based on the comparison of the specific agricultural product type to the one or more pre-classified product types and the comparison of the specific field space geolocation to the one or more pre-classified plant geolocations, performing at least one of:

executing a disease analysis on the one or more digital images of the specific plant associated with the specific agricultural product type;

automatically capturing at least one high-resolution digital image of the specific plant associated with the specific agricultural product type; and

executing a weed analysis on the one or more digital images of the specific plant associated with the specific agricultural product type.

2. The method of claim 1 , wherein comparing the specific field space geolocation to the one or more pre-classified plant geolocations includes determining that the specific field space geolocation does not match a pre-classified plant geolocation of the one or more pre-classified plant geolocations; and

wherein automatically capturing the at least one high-resolution digital image of the specific plant associated with the specific agricultural product type is in response to: the specific field space geolocation not matching a pre-classified plant geolocation of the one or more pre-classified plant geolocations.

3. The method of claim 1 , wherein comparing the specific agricultural product type to the one or more pre-classified product types includes determining that the specific agricultural product type matches the pre-classified product type of the one or more pre-classified product types; and

wherein comparing the specific field space geolocation to the one or more pre-classified plant geolocations includes determining that the specific field space geolocation does not match a pre-classified plant geolocation of the one or more pre-classified plant geolocations; and

wherein automatically capturing the at least one high-resolution digital image of the specific plant associated with the specific agricultural product type is in response to: the specific agricultural product type matching the pre-classified product type of the one or more pre-classified product types and the specific field space geolocation not matching a pre-classified plant geolocation of the one or more pre-classified plant geolocations.

4. The method of claim 3 , further comprising:

determining whether one or more attributes of the specific plant in the at least one high-resolution digital image are consistent with expected growth for the specific agricultural product type with which the specific plant is associated; and

in response to determining that the one or more attributes of the specific plant in the at least one high-resolution digital image are not consistent with expected growth for the specific agricultural product type with which the specific plant is associated, executing a weed analysis on the at least one high-resolution digital image of the specific plant associated with the specific agricultural product type.

5. The method of claim 1 , wherein comparing the specific agricultural product type to the one or more pre-classified product types includes determining that the specific agricultural product type matches the pre-classified product type of the one or more pre-classified product types; and

wherein comparing the specific field space geolocation to the one or more pre-classified plant geolocations includes determining that the specific field space geolocation does not match a pre-classified plant geolocation of the one or more pre-classified plant geolocations; and

wherein executing the weed analysis on the one or more digital images of the specific plant associated with the specific agricultural product type is in response to: the specific agricultural product type matching the pre-classified product type of the one or more pre-classified product types and the specific field space geolocation not matching a pre-classified plant geolocation of the one or more pre-classified plant geolocations.

6. The method of claim 5 , wherein executing the weed analysis comprises generating, via a trained machine learning classifier, a classification output specifying the digital image data representing the one or more images of the specific plant as a particular weed.

7. The method of claim 1 , wherein comparing the specific agricultural product type to the one or more pre-classified product types includes determining that the specific agricultural product type matches the pre-classified product type of the one or more pre-classified product types; and

wherein comparing the specific field space geolocation to the one or more pre-classified plant geolocations includes determining that the specific field space geolocation matches a pre-classified plant geolocation of the one or more pre-classified plant geolocations; and

wherein executing the disease analysis on the one or more digital images of the specific plant associated with the specific agricultural product type is in response to: the specific field space geolocation not matching a pre-classified plant geolocation of the one or more pre-classified plant geolocations.

8. The method of claim 7 , wherein executing the disease analysis comprises generating, via a trained machine learning classifier, a classification output specifying the digital image data representing the one or more images of the specific plant as a diseased plant candidate.

9. The method of claim 1 , wherein the plant growth characteristic of the specific agricultural product type is obtained from one or more sensor devices, each of the one or more sensor devices being located at a location in or near the agricultural field.

10. The method of claim 1 , further comprising determining a plant location match between the specific field space geolocation and one or more pre-classified agricultural product geolocations based on a digitally stored plot map of the agricultural field, the plot map including a geolocation of the specific agricultural product.

11. A non-transitory computer-readable storage medium comprising executable instructions for determining a specific agricultural product type using autonomous mobile equipment, which when executed by at least one processor, cause the at least one processor to:

obtain digital image data from a digital camera on autonomous mobile equipment configured to traverse an agricultural field, the digital image data representing images of one or more plants in the agricultural field and one or more images of a specific plant to be classified, wherein the specific plant to be classified is planted at one or more field geolocations of the agricultural field and is associated with a specific agricultural product type to be determined;

receive specific field space geolocation data representing a specific field space geolocation of the autonomous mobile equipment in the agricultural field;

compare the specific agricultural product type to one or more pre-classified product types based on: a plant growth characteristic of the specific agricultural product type and a pre-classified plant growth characteristic corresponding to the pre-classified product type to which the specific agricultural product type matches;

compare the specific field space geolocation to one or more pre-classified plant geolocations; and

based on the comparison of the specific agricultural product type to the one or more pre-classified product types and the comparison of the specific field space geolocation to the one or more pre-classified plant geolocations:

execute a disease analysis on the one or more digital images of the specific plant associated with the specific agricultural product type;

automatically capture at least one high-resolution digital image of the specific plant associated with the specific agricultural product type; and/or

execute a weed analysis on the one or more digital images of the specific plant associated with the specific agricultural product type.

12. The non-transitory computer-readable storage medium of claim 11 , wherein the executable instructions, when executed by the at least one processor to compare the specific field space geolocation to the one or more pre-classified plant geolocations, cause the at least one processor to determine that the specific field space geolocation does not match a pre-classified plant geolocation of the one or more pre-classified plant geolocations; and

wherein the executable instructions, when executed by the at least one processor, cause the at least one processor to automatically capture the at least one high-resolution digital image of the specific plant associated with the specific agricultural product type, in response to: the specific field space geolocation not matching a pre-classified plant geolocation of the one or more pre-classified plant geolocations.

13. The non-transitory computer-readable storage medium of claim 11 , wherein the executable instructions, when executed by the at least one processor to compare the specific agricultural product type to the one or more pre-classified product types, cause the at least one processor to determine that the specific agricultural product type matches the pre-classified product type of the one or more pre-classified product types; and

wherein the executable instructions, when executed by the at least one processor to compare the specific field space geolocation to the one or more pre-classified plant geolocations, cause the at least one processor to determine that the specific field space geolocation does not match a pre-classified plant geolocation of the one or more pre-classified plant geolocations; and

wherein the executable instructions, when executed by the at least one processor, cause the at least one processor to automatically capturing the at least one high-resolution digital image of the specific plant associated with the specific agricultural product type, in response to: the specific agricultural product type matching the pre-classified product type of the one or more pre-classified product types and the specific field space geolocation not matching a pre-classified plant geolocation of the one or more pre-classified plant geolocations.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the executable instructions, when executed by the at least one processor, further cause the at least one processor to:

determine whether one or more attributes of the specific plant in the at least one high-resolution digital image are consistent with expected growth for the specific agricultural product type with which the specific plant is associated; and

in response to determining that the one or more attributes of the specific plant in the at least one high-resolution digital image are not consistent with expected growth for the specific agricultural product type with which the specific plant is associated, execute a weed analysis on the at least one high-resolution digital image of the specific plant associated with the specific agricultural product type.

15. The non-transitory computer-readable storage medium of claim 11 , wherein the executable instructions, when executed by the at least one processor to compare the specific agricultural product type to the one or more pre-classified product types, cause the at least one processor to determine that the specific agricultural product type matches the pre-classified product type of the one or more pre-classified product types; and

wherein the executable instructions, when executed by the at least one processor to compare the specific field space geolocation to the one or more pre-classified plant geolocations, cause the at least one processor to determine that the specific field space geolocation does not match a pre-classified plant geolocation of the one or more pre-classified plant geolocations; and

wherein the executable instructions, when executed by the at least one processor, cause the at least one processor to execute the weed analysis on the one or more digital images of the specific plant associated with the specific agricultural product type, in response to the specific agricultural product type matching the pre-classified product type of the one or more pre-classified product types and based on the specific field space geolocation not matching a pre-classified plant geolocation of the one or more pre-classified plant geolocations.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the executable instructions, when executed by the at least one processor to execute the weed analysis, cause the at least one processor to generate, via a trained machine learning classifier, a classification output specifying the digital image data representing the one or more images of the specific plant as a particular weed.

17. The non-transitory computer-readable storage medium of claim 11 , wherein the executable instructions, when executed by the at least one processor to compare the specific agricultural product type to the one or more pre-classified product types, cause the at least one processor to determine that the specific agricultural product type matches the pre-classified product type of the one or more pre-classified product types; and

wherein the executable instructions, when executed by the at least one processor to compare the specific field space geolocation to the one or more pre-classified plant geolocations, cause the at least one processor to determine that the specific field space geolocation matches a pre-classified plant geolocation of the one or more pre-classified plant geolocations; and

wherein the executable instructions, when executed by the at least one processor, cause the at least one processor to execute the disease analysis on the one or more digital images of the specific plant associated with the specific agricultural product type, in response to: the specific field space geolocation not matching a pre-classified plant geolocation of the one or more pre-classified plant geolocations.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the executable instructions, when executed by the at least one processor to execute the disease analysis, cause the at least one processor to generate, via a trained machine learning classifier, a classification output specifying the digital image data representing the one or more images of the specific plant as a diseased plant candidate.

19. An autonomous mobile device configured to traverse an agricultural field, the autonomous mobile device comprising:

a digital camera;

one or more hardware processors coupled to the digital camera; and

one or more non-transitory computer-readable storage media coupled to the one or more hardware processors and storing machine-executable instructions which, when executed using the one or more hardware processors, cause the one or more hardware processors to:

obtain digital image data from the digital camera, the digital image data representing images of one or more plants in the agricultural field and one or more images of a specific plant to be classified, wherein the specific plant to be classified is planted at one or more field geolocations of the agricultural field and is associated with a specific agricultural product type to be determined;

receive specific field space geolocation data representing a specific field space geolocation of the autonomous mobile device in the agricultural field;

compare the specific agricultural product type to one or more pre-classified product types based on: a plant growth characteristic of the specific agricultural product type and a pre-classified plant growth characteristic corresponding to the pre-classified product type to which the specific agricultural product type matches;

compare the specific field space geolocation to one or more pre-classified plant geolocations; and

based on the comparison of the specific agricultural product type to the one or more pre-classified product types and the comparison of the specific field space geolocation to the one or more pre-classified plant geolocations:

execute a disease analysis on the one or more digital images of the specific plant associated with the specific agricultural product type;

automatically capture, via the digital camera, at least one high-resolution digital image of the specific plant associated with the specific agricultural product type; and/or

execute a weed analysis on the one or more digital images of the specific plant associated with the specific agricultural product type.