Systems and methods for real-time measurement and control of sprayed liquid coverage on plant surfaces
Presented herein are systems and methods for automatically determining liquid coverage on plant surfaces. More particularly, in certain embodiments, presented herein is a system for receiving an image depicting one or more plant surfaces, automatically identifying the plant surfaces in the image and distinguishing portions covered by liquid, and automatically determining a liquid coverage value. In some embodiments, the system determines changes to liquid spraying parameters to achieve desired liquid coverage values. In some embodiments, the system uses two cameras to cooperatively conduct background removal in images and determination of liquid coverage.
1 . A system for automatically quantifying liquid coverage on plant surfaces, the system comprising:
a processor of a computing device; and
a memory having instructions stored thereon, wherein the instructions, when executed by the processor, cause the processor to:
receive an image comprising a region of interest corresponding to one or more plant surfaces;
automatically identify one or more portions of the region of interest corresponding to liquid; and
use (i) one or more pre-spray images corresponding to a field of view comprising the one or more plant surfaces prior to spraying with a liquid and (ii) one or more post-spray images corresponding to the field of view comprising the one or more plant surfaces after spraying with the liquid
to automatically identify a liquid coverage value for the plant surfaces by:
(i) removing background pixels in the one or more pre-spray images so that leaves remain;
(ii) removing background pixels in the one or more post-spray images so that leaves remain; and
(iii) calculating the liquid coverage value based on pixels corresponding to leaves and sprayed liquid in the one or more pre-spray images and the one or more post-spray images.
2 . The system of claim 1 , further comprising one or more imaging devices and/or sensors for obtaining the image, wherein the one or more imaging devices and/or sensors comprises at least one member of the group consisting of a camera, a digital camera, a camera phone, a thermal imaging device, a night vision camera, a Light Detection and Ranging (LiDAR) device, an electronic image sensor, a charge-coupled device (CCD), an active-pixel sensor (CMOS sensor), a smart image sensor, an intelligent image sensor, and a short-wave infrared (SWIR) camera.
3 . The system of claim 2 , wherein the liquid on the plant surfaces comprises a sprayed-on liquid, wherein the one or more imaging devices and/or sensors comprises the short-wave infrared (SWIR) camera, and wherein sufficient detectable contrast is achieved for accurate liquid coverage value determination without the need for any dyes to be added to the sprayed-on liquid.
4 . The system of claim 1 , further comprising a first camera for receiving a first image corresponding to a field of view comprising the one or more plant surfaces and a second camera for receiving a second image corresponding to the field of view of the first image, wherein the instructions, when executed by the processor, cause the processor to automatically identify a background mask from the first image, said background mask corresponding to non-plant-surface portions of the first image, to apply the background mask to the second image, thereby eliminating non-plant surface portions from the second image, and to automatically identify the liquid coverage value for the plant surfaces depicted in the background-eliminated second image.
5 . The system of claim 4 , wherein the first camera is a red-green-blue (RGB) camera and the second camera is a shortwave infrared (SWIR) camera.
6 . The system of claim 5 , comprising an optical element that reflects infrared (IR) light and allows visible light to pass, said optical element positioned to allow alignment of fields of view of the first camera and the second camera.
7 . The system of claim 1 , wherein the instructions, when executed by the processor, cause the processor to automatically determine a series of liquid coverage values for regions in a sequence of images in real time, as the sequence of images is obtained.
8 . The system of claim 1 , the system further comprising:
a display comprising a display screen and a graphical user interface (GUI), wherein the instructions cause the processor to graphically render the liquid coverage value for viewing by a person via the display.
9 . The system of claim 1 , wherein the instructions, when executed by the processor, cause the processor to use the determined liquid coverage value or values to automatically determine an adjustment of one or more sprayer system parameters to achieve a desired level of liquid coverage, wherein the one or more sprayer system parameters comprises at least one member selected from the group consisting of a sprayer speed, a nozzle type, a nozzle positioning and/or orientation, a number of nozzles used, a spray pressure, an adjuvant and/or additive rate, an overall flow rate, and a boom orientation and/or height.
10 . The system of claim 9 , wherein the system comprises one or more environmental sensors for capturing environmental data corresponding to one or more environmental conditions at a location and at a time the image(s) is/are obtained, and
wherein the instructions, when executed by the processor, cause the processor to use the environmental data along with the identified liquid coverage value or values to automatically determine the adjustment of the one or more sprayer system parameters, wherein the one or more environmental sensors comprises one or more sensors selected from the group consisting of a temperature sensor, a humidity sensor, a pressure sensor, a wind sensor, a light sensor, an air quality sensor, a gas sensor, a rainfall sensor, a radiation sensor, and a soil sensor.
11 . The system of claim 9 , wherein the instructions, when executed by the processor, cause the processor to automatically determine a series of liquid coverage values for regions of interest in a sequence of images and use the automatically determined values to automatically determine the adjustment of the one or more sprayer system parameters to achieve the desired level of liquid coverage, wherein the instructions cause the processor to automatically implement the determined adjustment(s) in real time via a control system for controlling the one or more sprayer system parameters, thereby operating the sprayer system in real time to improve liquid coverage by accounting for one or more changing conditions.
12 . The system of claim 1 , wherein the liquid coverage value quantifies a total liquid volume on the depicted in the region of interest.
13 . The system of claim 1 , wherein the liquid coverage value quantifies a total liquid volume per unit area of the plant surfaces depicted in the region of interest.
14 . The system of claim 1 , wherein the liquid coverage value quantifies an absolute or relative surface area of the plant surfaces depicted in the region of interest that is covered by liquid.
15 . The system of claim 1 , wherein the system is mounted to a spraying mechanism.
16 . The system of claim 15 , wherein the spraying mechanism comprises at least one of a plow and an agricultural sprayer.
17 . The system of claim 16 , comprising the agricultural sprayer, wherein the agricultural sprayer comprises a boom sprayer, a boomless sprayer nozzle, a mist sprayer, a three-point hitch sprayer, a truck-bed sprayer, a towing-hitch sprayer, a Utility Task Vehicle (UTV) sprayer, an All-Terrain Vehicle (ATV) sprayer, a self-propelled sprayer, a towed sprayer, a robotic sprayer, a hand sprayer, or a backpack sprayer.
18 . The system of claim 16 , wherein the computing device is implemented as a portable device mounted on the spraying mechanism.
19 . The system of claim 1 , wherein the system is mounted to a drone.
20 . The system of claim 19 , wherein the drone comprises an unmanned aerial vehicle.
21 . The system of claim 1 , further comprising one or more imaging devices.
22 . The system of claim 21 , wherein the one or more imaging devices transmit data wirelessly.
23 . The system of claim 1 , wherein calculating the liquid coverage value comprises calculating an average histogram of a property of pixels of the one or more pre-spray images corresponding to leaves and sprayed liquid.
24 . The system of claim 23 , wherein calculating the liquid coverage value further comprises calculating the average histogram of the property of the pixels of the one or more post-spray images corresponding to leaves and sprayed liquid.
25 . The system of claim 24 , wherein calculating the liquid coverage value further comprises calculating a difference of average histograms between the average histogram of the property of the pixels of the pre-spray images corresponding to leaves and sprayed liquid and the average histogram of the property of the pixels of the post-spray images corresponding to leaves and sprayed liquid, and calculating an area under a curve of the difference of histograms to determine an area of spray coverage.
26 . The system of claim 1 , wherein the liquid on the plant surfaces comprises a sprayed-on solution comprising one or more members selected from the group consisting of water, an adjuvant, an additive, a crop-compatible dye, an agrochemical solution, a liquid solution of a pesticide, a liquid solution of a fertilizer, and a foliar fertilizer.
27 . The system of claim 24 , wherein the property of the pixels is a member selected from the group consisting of pixel color, pixel intensity, pixel greyscale value, and pixel brightness.
28 . A method for automatically quantifying liquid coverage on plant surfaces, the method comprising:
receiving, by a processor of a computing device, an image comprising a region of interest corresponding to one or more plant surfaces;
automatically identifying, by the processor, one or more portions of the region of interest corresponding to liquid; and
using (i) one or more pre-spray images corresponding to a field of view comprising the one or more plant surfaces prior to spraying with a liquid and (ii) one or more post-spray images corresponding to the field of view comprising the one or more plant surfaces after spraying with the liquid
to automatically identify a liquid coverage value for the plant surfaces by:
(i) removing background pixels in the one or more pre-spray images so that leaves remain;
(ii) removing background pixels in the one or more post-spray images so that leaves remain; and
(iii) calculating the liquid coverage value based on pixels corresponding to leaves and sprayed liquid in the one or more pre-spray images and the one or more post-spray images.
29 . The method of claim 28 , comprising a first camera for receiving a first image corresponding to a field of view comprising the one or more plant surfaces and a second camera for receiving a second image corresponding to the field of view of the first image;
wherein the instructions, when executed by the processor, cause the processor to:
automatically identify a background mask from the first image, said background mask corresponding to non-plant-surface portions of the first image;
apply the background mask to the second image, thereby eliminating non-plant surface portions from the second image; and
automatically identify the liquid coverage value for the plant surfaces depicted in the background-eliminated second image.
30 . The method of claim 28 , comprising automatically determining, by the processor, an adjustment of one or more sprayer system parameters to achieve a desired level of liquid coverage, using the determined liquid coverage value, wherein the one or more sprayer system parameters comprises at least one of a sprayer speed, a nozzle type, a nozzle positioning and/or orientation, a number of nozzles used, a spray pressure, an adjuvant and/or additive rate, an overall flow rate, and a boom orientation.
31 . The method of claim 30 , wherein the method comprises capturing environmental data corresponding to one or more environmental conditions at a location and at a time the image(s) is/are obtained using one or more environmental sensors, and automatically determining, by the processor, the adjustment of the one or more sprayer system parameters using the environmental data along with the determined liquid coverage value or values, wherein the one or more environmental sensors comprise one or more sensors selected from the group consisting of a temperature sensor, a humidity sensor, a pressure sensor, a wind sensor, a light sensor, an air quality sensor, a gas sensor, a rainfall sensor, a radiation sensor, and a soil sensor.
32 . The method of claim 30 , comprising automatically determining, by the processor, a series of liquid coverage values for regions of interest in a sequence of images and using the automatically determined liquid coverage values to automatically determine the adjustment of the one or more sprayer system parameters to achieve the desired level of liquid coverage, and automatically implementing the determined adjustment(s) in real time via a control system for controlling the one or more sprayer system parameters, thereby operating the sprayer system in real time to improve liquid coverage by accounting for one or more changing conditions.
33 . The method of claim 28 , wherein the liquid coverage value quantifies a total liquid volume on the plant surfaces depicted in the region of interest.
34 . The method of claim 28 , wherein the liquid coverage value quantifies a total liquid volume per unit area of the plant surfaces depicted in the region of interest.
35 . The method of claim 28 , wherein the liquid coverage value quantifies an absolute or relative surface area of the plant surfaces depicted in the region of interest that is covered by liquid.
36 . The method of claim 28 , wherein calculating the liquid coverage value comprises calculating an average histogram of a property of pixels of the one or more pre-spray images corresponding to leaves and sprayed liquid.
37 . The method of claim 36 , wherein calculating the liquid coverage value further comprises calculating the average histogram of the property of the pixels of the one or more post-spray images corresponding to leaves and sprayed liquid.
38 . The method of claim 37 , wherein calculating the liquid coverage value further comprises calculating a difference of average histograms between the average histogram of the property of the pixels of the pre-spray images corresponding to leaves and sprayed liquid and the average histogram of the property of the pixels of the post-spray images corresponding to leaves and sprayed liquid, and calculating an area under a curve of the difference of histograms to determine an area of spray coverage.
39 . The method of claim 37 , wherein the property of the pixels is a member selected from the group consisting of pixel color, pixel intensity, pixel greyscale value, and pixel brightness.