Monitoring plant growth conditions by coupling near surface and remote sensing imageries
View Patent ↗One or more cameras or other image collecting devices are used at or near the surface of a field including crops to acquire images of the crop canopy. This can be done by tilting the cameras relative to the zenith. Additional imagery, such as airborne or satellite of the same crop canopies is also acquired, with at least some of the imagery having similar geographical and time-stamped information. A machined-learning model will take the near surface imagery and be trained to apply the information to the remote imagery of the air or satellite to be applied to a greater area, which provides plant growth information to the wide area of the remote imagery.
1 . A near surface system for monitoring plant growth conditions, comprising:
at least one tilted camera comprising a field of view covering an area ranging from the zenith direction towards a horizon; and
a processing system including a processor that facilitates performance of operations, the operations comprising:
obtaining from the at least one tilted camera tilted images that depict a portion of a plant canopy covering an area ranging from the zenith direction towards the horizon; and
delivering the tilted images to a non-transitory machine-readable storage;
wherein the at least one tilted camera comprises a below-canopy upward-viewing camera, wherein the below-canopy upward-viewing camera is mounted at a location below the canopy to a permanent support structure on the ground or is mounted to a fully below-canopy ground carrier.
2 . The system of claim 1 , wherein:
the at least one tilted camera obtains images of plant canopy from a position below the plant canopy; or
the at least one tilted camera obtains images of plant canopy from a position above the plant canopy.
3 . The system of claim 1 , wherein:
the below-canopy camera is movable along a surface to acquire images of plant canopy at a plurality of positions.
4 . The system of claim 3 , wherein:
the below-canopy camera is rotatable to acquire images of plant canopy from a plurality of orientations.
5 . The system of claim 4 , wherein:
the at least one tilted camera comprises an above-canopy camera that is movable along a surface to acquire images of plant canopy at a plurality of positions.
6 . The system of claim 5 , wherein:
the above-canopy camera is rotatable to acquire images of plant canopy from a plurality of orientations.
7 . The system of claim 6 , wherein:
the below-canopy camera is mounted to a tracked robot or vehicle, a wheeled robot or vehicle, or any first combination thereof.
8 . The system of claim 7 , wherein:
the above-canopy camera is mounted to a drone, a balloon, an airborne plane, or any second combination thereof.
9 . The system of claim 1 , further comprising:
a plurality of below-canopy cameras configured to acquire images of plant canopy at a plurality of positions; and
a plurality of above-canopy cameras configured to acquire images of plant canopy at a plurality of positions.
10 . The system of claim 9 , wherein:
the plurality of below-canopy cameras are mounted to a plurality of chassis on the ground; and
the plurality of above-canopy cameras are mounted to a plurality of poles on the ground.
11 . The system of claim 10 , wherein:
the plurality of below-canopy cameras are handheld by a plurality of operators; and
the plurality of above-canopy cameras are handheld by a plurality of operators.
12 . The system of claim 1 , wherein the at least one tilted camera comprises RGB cameras, near-infrared cameras, multi-spectral cameras, hyperspectral cameras, or any combination thereof.
13 . A method comprising:
collecting a plurality of tilted near surface images of plant canopy from at least one near surface device, and collecting remote sensing data comprising one or more remote sensing images from at least one remote sensing platform, wherein geographic footprints and acquisition dates of said tilted near surface images and said one or more remote sensing images are at least partially overlapping;
generating ecosystem variables from the tilted near surface images, the ecosystem variables comprising average leaf angle;
for the overlapped geographic footprints and acquisition dates of the near surface images and the one or more remote sensing images, configuring models and relationships to estimate ecosystem variables using the generated ecosystem variables as ground truths, and using the one or more remote sensing images as input; and
generating predictions of agroecosystem variables by applying the models or relationships over the whole or part of the geographic footprint of the remote sensing data.
14 . The method of claim 13 , wherein the ecosystem variables further comprise: leaf area index; plant area index; fraction of green leaves; fraction of plant cover; clumping index; fraction of absorbed photosynthetically active radiation by plants; fraction of absorbed photosynthetically active radiation by green leaves; chlorophyll content; nitrogen content; above ground biomass; crop growth stage; crop stress, disease condition, or any combination thereof.
15 . The method of claim 13 , wherein the models and relationships comprise physics-based canopy radiative transfer models or radiative transfer based machine learning models for the ecosystem variables, statistical, machine learning, deep learning, computer vision, physics, or other models.
16 . A computer-implemented method for monitoring plant growth conditions, comprising:
obtaining a plurality of tilted, near surface images of plant canopy from at least one near surface device;
obtaining a plurality of remote images of plant canopy from a remote device comprising a satellite wherein the remote images are obtained via the satellite;
generating a set of geographic footprints and acquisition dates for both the plurality of the tilted, near surface images and the plurality of remote images;
training a model using an overlapped value of the geographic footprints and acquisition dates of the tilted, near surface images and the plurality of remote images, wherein said training does not involve use of a biological sensor positioned in, at, or near the geographic footprints of the tilted, near surface images or the geographic footprints of the plurality of remote images;
processing the plurality of remote images of plant canopy using the trained model to predict plant growth conditions for the remote images of plant canopy.
17 . The method of claim 16 , wherein the at least one near surface device comprises at least one camera positioned at an angle relative to a zenith of the plant canopy.
18 . The method of claim 16 , further comprising associating ecosystem variables with the obtained plurality of tilted, near surface images.
19 . The method of claim 18 , wherein the ecosystem variables comprise: leaf area index, plant area index; fraction of green leaves; fraction of plant cover; average leaf angle; clumping index; fraction of absorbed photosynthetically active radiation by plants; fraction of absorbed photosynthetically active radiation by green leaves; chlorophyll content; nitrogen content; above ground biomass; crop growth stage; crop stress, disease condition, or any combination thereof.
20 . The method of claim 16 , wherein the step of training the model comprises identifying pixels in the tilted, near surface images, associating ecosystem variables within the pixels, and upscaling the pixels to the area covered by the remote images.