IP Library › Granted Patent US 11,237,563
Granted Patent B2
US 11,237,563 · App. 17/100,773 · Granted Feb 1, 2022

Method for analyzing individual plants in an agricultural field

Inventors: Michael Javault (San Francisco, CA); Pauline Canteneur (San Francisco, CA); Arthur Flajolet (San Francisco, CA); Sébastien Boyer (San Francisco, CA); Thomas Palomares (San Francisco, CA)
Assignee: FarmWise Labs, Inc.
G05D1/0219A01B69/008A01C21/005A01C21/007A01D46/30A01G25/09A01G25/167A01M7/0042A01M7/0089A01M21/02G05B19/4155G05D1/0088G05D1/0246G06K9/00657G06T7/0012G06T7/70B60R11/04G05B2219/45017G05D2201/0201G06T2207/30188
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,237,563
App. No.
17/100,773
Granted
Feb 1, 2022
Kind
B2
Abstract

A method includes: at an autonomous vehicle and during a first operating period, capturing a first set of images of a plant; calculating a location of the plant based on the first set of images; extracting an initial value of a plant metric of the plant based on the first set of images; predicting a predicted value of the plant metric of the plant at a time based on the initial value of the plant metric of the plant and a set of global condition data. The method also includes, at the autonomous vehicle and during a second operating period concurrent with the time: capturing a second set of images of the plant; identifying the plant based on the second set of images and the location of the plant; and executing an agricultural operation on the plant based on the predicted value of the plant metric.

Claims (136)

1. A method comprising:

at a vehicle, during a first operating period:

navigating within an agricultural field;

capturing a first set of images of a set of target plants within the agricultural field, the set of target plants comprising a first individual target plant and a second individual target plant;

capturing a first set of location data of the vehicle;

executing a first agricultural operation on the first individual target plant, the first agricultural operation characterized by a first magnitude of the first agricultural operation and a first timestamp of the first agricultural operation; and

executing a second agricultural operation on the second individual target plant, the second agricultural operation characterized by a second magnitude of the second agricultural operation and a second timestamp of the second agricultural operation;

detecting a position of the first individual target plant in the first set of images;

calculating a location of the first individual target plant based on the position of the first individual target plant in the first set of images and the first set of location data;

generating a first plant profile of the first individual target plant associating the first individual target plant with a first unique identifier;

extracting a first initial value of a plant metric characterizing the first individual target plant based on the first set of images;

populating the first plant profile with:

the location of the first individual target plant;

the first initial value of the plant metric;

the first magnitude of the first agricultural operation; and

the first timestamp of the first agricultural operation;

detecting a position of the second individual target plant in the first set of images;

calculating a location of the second individual target plant based on the position of the second individual target plant in the first set of images and the first set of location data;

generating a second plant profile of the second individual target plant associating the second individual target plant with a second unique identifier;

extracting a second initial value of the plant metric characterizing the second individual target plant based on the first set of images;

populating the second plant profile with:

the location of the second individual target plant;

the second initial value of the plant metric;

the second magnitude of the second agricultural operation; and

the second timestamp of the second agricultural operation;

accessing a set of global condition data associated with the agricultural field and captured during a target time interval between the first operating period and a target time;

predicting a first predicted value of the plant metric characterizing the first individual target plant at the target time based on:

the first initial value of the plant metric;

the first magnitude of the first agricultural operation;

the first timestamp of the first agricultural operation; and

the set of global condition data;

populating the first plant profile with the first predicted value of the plant metric in association with the target time;

predicting a second predicted value of the plant metric characterizing the second individual target plant at the target time based on:

the second initial value of the plant metric;

the second magnitude of the second agricultural operation;

the second timestamp of the second agricultural operation; and

the set of global condition data;

populating the second plant profile with the second predicted value of the plant metric in association with the target time; and

at the vehicle, during a second operating period at the target time:

navigating within the agricultural field;

capturing a second set of images of the set of target plants within the agricultural field;

capturing a second set of location data of the vehicle;

accessing the location of the first individual target plant and the first predicted value of the plant metric from the first plant profile;

identifying the first individual target plant based on the second set of images, the second set of location data, and the location of the first individual target plant;

executing a third agricultural operation on the first individual target plant based on the first predicted value of the plant metric;

accessing the location of the second individual target plant and the second predicted value of the plant metric from the second plant profile;

identifying the second individual target plant based on the second set of images, the second set of location data, and the location of the second individual target plant; and

executing a fourth agricultural operation on the second individual target plant based on the second predicted value of the plant metric, the fourth agricultural operation different from the third agricultural operation.

2. The method of claim 1 :

wherein detecting the position of the first individual target plant in the first set of images comprises:

classifying the first individual target plant as a target plant species based on the first set of images; and

identifying a stalk position of the first individual target plant based on the first set of images and the target plant species; and

wherein calculating the location of the first individual target plant based on the position of the first individual target plant in the first set of images and the first set of location data of the vehicle comprises calculating the location of the plant based on the stalk position of the first individual target plant in the first set of images and the first set of location data of the vehicle.

3. The method of claim 1 :

wherein extracting the first initial value of the plant metric characterizing the first individual target plant based on the first set of images comprises:

estimating a plant maturity of the first individual target plant based on a target plant species and the first set of images; and

calculating an initial value of a root diameter metric of the first individual target plant and an initial value of a root depth metric characterizing the first individual target plant based on the plant maturity of the first individual target plant;

wherein populating the first plant profile with the location of the first individual target plant and the first initial value of the plant metric comprises populating the first plant profile with:

the location of the first individual target plant;

the initial value of the root diameter metric; and

the initial value of the root depth metric;

wherein predicting the first predicted value of the plant metric characterizing the first individual target plant at the target time comprises:

predicting a predicted value of the root diameter metric characterizing the first individual target plant at the target time based on the initial value of the root diameter metric and the set of global condition data; and

predicting a predicted value of the root depth metric characterizing the first individual target plant at the target time based on the initial value of the root depth metric and the set of global condition data;

wherein populating the first plant profile with the first predicted value of the plant metric in association with the target time comprises:

populating the first plant profile with the predicted value of the root diameter metric in association with the target time; and

populating the first plant profile with the predicted value of the root depth metric in association with the target time; and

wherein executing the third agricultural operation on the first individual target plant based on the first predicted value of the plant metric comprises executing the third agricultural operation on the first individual target plant based on the predicted value of the root diameter metric and the predicted value of the root depth metric.

4. The method of claim 1 :

wherein extracting the first initial value of the plant metric characterizing the first individual target plant based on the first set of images comprises:

generating a three-dimensional model of the first individual target plant based on the first set of images; and

calculating an initial value of a plant volume metric characterizing the first individual target plant based on the three-dimensional model;

wherein populating the first plant profile with the location of the first individual target plant and the first initial value of the plant metric comprises populating the first plant profile with the location of the first individual target plant and the initial value of the plant volume metric;

wherein predicting the first predicted value of the plant metric characterizing the first individual target plant at the target time comprises predicting a predicted value of the plant volume metric characterizing the first individual target plant at the target time based on the initial value of the plant volume metric and the set of global condition data;

wherein populating the first plant profile with the first predicted value of the plant metric in association with the target time comprises populating the first plant profile with the predicted value of the plant volume metric in association with the target time; and

wherein executing the third agricultural operation on the first individual target plant based on the first predicted value of the plant metric comprises executing the third agricultural operation on the first individual target plant based on the predicted value of the plant volume metric.

5. The method of claim 1 :

wherein capturing the first set of images of the set of target plants within the agricultural field comprises capturing the first set of images of the set of target plants within the agricultural field, the first set of images comprising a first set of hyperspectral images;

wherein extracting the first initial value of the plant metric characterizing the first individual target plant based on the first set of images comprises extracting an initial value of a water content metric characterizing the first individual target plant based on the first set of hyperspectral images;

wherein populating the first plant profile with the location of the first individual target plant and the first initial value of the plant metric comprises populating the first plant profile with the location of the first individual target plant and the initial value of the water content metric;

wherein predicting the first predicted value of the plant metric characterizing the first individual target plant at the target time comprises predicting a predicted value of the water content metric characterizing the first individual target plant at the target time based on the initial value of the water content metric and the set of global condition data;

wherein populating the first plant profile with the first predicted value of the plant metric in association with the target time comprises populating the first plant profile with the predicted value of the water content metric in association with the target time; and

wherein executing the third agricultural operation on the first individual target plant based on the first predicted value of the plant metric comprises executing the third agricultural operation on the first individual target plant based on the predicted value of the water content metric.

6. The method of claim 1 :

wherein extracting the first initial value of the plant metric characterizing the first individual target plant based on the first set of images comprises extracting an initial value of a disease pressure metric characterizing the first individual target plant based on the first set of images;

wherein populating the first plant profile with the location of the first individual target plant and the first initial value of the plant metric comprises populating the first plant profile with the location of the first individual target plant and the initial value of the disease pressure metric;

wherein predicting the first predicted value of the plant metric characterizing the first individual target plant at the target time comprises predicting a predicted value of the disease pressure metric characterizing the first individual target plant at the target time based on the initial value of the disease pressure metric and the set of global condition data;

wherein populating the first plant profile with the first predicted value of the plant metric in association with the target time comprises populating the first plant profile with the predicted value of the disease pressure metric in association with the target time; and

wherein executing the third agricultural operation on the first individual target plant based on the first predicted value of the plant metric comprises executing the third agricultural operation on the first individual target plant based on the predicted value of the disease pressure metric.

7. The method of claim 1 :

further comprising:

at the vehicle, capturing a first set of local condition data, each local condition data point in the set of local condition data associated with a subregion of the agricultural field; and

populating the first plant profile with an adjacent local condition data point of the first individual target plant from the set of local condition data based on an adjacent region associated with the adjacent local data point, the adjacent region encompassing the location of the first individual target plant; and

wherein predicting the first predicted value of the plant metric characterizing the first individual target plant at the target time comprises predicting the first predicted value of the plant metric characterizing the first individual target plant at the target time based on the first initial value of the plant metric, the set of global condition data, and the adjacent local condition data point.

8. The method of claim 7 :

wherein populating the first plant profile with the adjacent local condition data point comprises appending the adjacent local condition data point to a time series of adjacent local condition data of the first individual target plant captured during a series of prior operating periods;

further comprising:

accessing a set of prior global condition data associated with the agricultural field and concurrent with the series of prior operating periods;

correlating the time series of adjacent local condition data with the set of prior global condition data to generate a local condition model; and

predicting a time series of local condition data for the target time interval between the first operating period and the target time based on the local condition model and the set of global condition data; and

wherein predicting the first predicted value of the plant metric characterizing the first individual target plant at the target time comprises predicting the first predicted value of the plant metric characterizing the first individual target plant at the target time based on the first initial value of the plant metric and the time series of local condition data.

9. The method of claim 1 :

wherein accessing the set of global condition data associated with the agricultural field and captured during the target time interval comprises accessing a series of satellite images of the agricultural field captured during the target time interval; and

wherein predicting the first predicted value of the plant metric characterizing the first individual target plant at the target time comprises predicting the first predicted value of the plant metric at the target time based on the first initial value of the plant metric and the series of satellite images of the agricultural field.

10. The method of claim 1 :

wherein accessing the set of global condition data associated with the agricultural field and captured during the target time interval comprises accessing a set of weather data associated with the agricultural field captured during the target time interval; and

wherein predicting the first predicted value of the plant metric characterizing the first individual target plant at the target time comprises predicting the first predicted value of the plant metric at the target time based on the first initial value of the plant metric and the set of weather data associated with the agricultural field.

11. The method of claim 1 , further comprising:

at the vehicle, during the second operating period capturing a set of operation data associated with the third agricultural operation executed on the first individual target plant; and

populating the first plant profile with the set of operation data associated with the third agricultural operation executed on the first individual target plant.

12. The method of claim 1 :

further comprising:

prior to the target time, generating an agricultural operation plan for the first individual target plant based on the first predicted value of the plant metric; and

at the vehicle and during the second operating period:

accessing the agricultural operation plan for the first individual target plant;

in response to identifying the first individual target plant based on the second set of images, calculating an adjusted value of the plant metric based on the second set of images; and

adjusting the third agricultural operation plan for the first individual target plant based on the adjusted value of the plant metric to generate an adjusted agricultural operation plan for the first individual target plant; and

wherein executing the third agricultural operation on the first individual target plant based on the first predicted value of the plant metric comprises executing the third agricultural operation on the first individual target plant in accordance with the adjusted agricultural operation plan.

13. The method of claim 1 , further comprising, at the vehicle, during the second operating period:

in response to identifying the first individual target plant based on the second set of images, calculating an adjusted value of the plant metric characterizing the first individual target plant based on the second set of images; and

detecting an anomaly associated with the first individual target plant based on a difference between the adjusted value of the plant metric and the first predicted value of the plant metric.

14. The method of claim 1 , wherein executing the third agricultural operation on the first individual target plant based on the first predicted value of the plant metric comprises:

setting a maximum blade gap for a weeding operation on the first individual target plant based on the first predicted value of the plant metric; and

executing the weeding operation on the first individual target plant in accordance with the maximum blade gap.

15. The method of claim 1 , wherein executing the third agricultural operation on the first individual target plant based on the first predicted value of the plant metric comprises:

generating a harvesting designation based on the first predicted value of the plant metric; and

in response to the harvesting designation indicating that the first individual target plant is designated for harvesting, executing a harvesting operation on the first individual target plant.

16. The method of claim 1 , wherein executing the third agricultural operation on the first individual target plant based on the first predicted value of the plant metric comprises:

calculating a fertilizer quantity based on the first predicted value of the plant metric; and

dispensing the fertilizer quantity at the location of the first individual target plant.

17. The method of claim 1 , wherein executing the third agricultural operation on the first individual target plant based on the first predicted value of the plant metric comprises:

calculating a pesticide quantity based on the first predicted value of the plant metric; and

dispensing the pesticide quantity at the location of the first individual target plant.

18. The method of claim 1 , wherein executing the third agricultural operation on the first individual target plant based on the first predicted value of the plant metric comprises:

calculating a water quantity based on the first predicted value of the plant metric; and

dispensing the water quantity at the location of the first individual target plant.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 23, 2021
From: JAVAULT, MICHAEL; CANTENEUR, PAULINE; FLAJOLET, ARTHUR; BOYER, SÉBASTIEN; PALOMARES, THOMAS
To: FARMWISE LABS, INC
Reel/Frame 057585/0214 →
Continuity (2)
Provisional Application 62938026 · Nov 20, 2019
Related Publication 20210144903A1 · May 20, 2021
Cited By (3)
US 12,201,044 US 12,493,120 US 12,670,136