Systems and methods for spectral analysis of plants
Systems and methods for hyperspectral imaging of plants are provided. Multispectral images of plants are transformed, e.g. by interpolation along a spectral axis, to generate hyperspectral images of plants. The transformation can be based on spectral bases formed from hyperspectral sample images including images of plant matter. Plant characteristics, such as plant health, may be predicted based on the hyperspectral image. Plant health may be predicted by comparing derivatives of reflectance values with respect to wavelength for a plant of a given image relative to a reference derivative based on a reference hyperspectral image. The derivatives may be compared by determining a regression loss.
1 . A method for spectral characterization of plants, the method performed by a processor and comprising:
receiving an input multispectral image comprising a number m of spectral channels, at least one spectral channel comprising an infrared wavelength, the multispectral image representing at least one portion of at least one plant;
generating a multispectral image comprising a number n of multispectral channels based on the input multispectral image and a plurality of spectral bases, the number n of multispectral channels greater than the number m of input multispectral channels;
generating a determination for the at least one portion of the at least one plant based on the multispectral image based on a derivative of a plurality of reflectance values of the multispectral image with respect to wavelength.
2 . The method according to claim 1 wherein the determination comprises a prediction of plant health and generating the determination comprises determining the prediction of plant health based on the derivative of the plurality of reflectance values and a plurality of reference reflectance values.
3 . The method according to claim 2 further comprising generating the plurality of reference reflectance values based on a reference multispectral image representing at least a healthy portion of a reference plant, optionally wherein generating the plurality of reference reflectance values comprises determining an average of reflectance values for a plurality of spatial locations of the at least the healthy portion of the reference plant for each of a plurality of spectral channels of the reference multispectral image.
4 . The method according to claim 2 wherein determining the prediction of plant health comprises determining a difference between the derivative of the plurality of reflectance values and a derivative of the plurality of reference reflectance values with respect to wavelength.
5 . The method according to claim 4 wherein determining the difference comprises determining a regression loss metric based on the derivative of the plurality of reflectance values and a derivative of the plurality of reference reflectance values, wherein the regression loss optionally comprises at least one of: a mean square error, a mean absolute error, a Huber loss, a log-cosh loss, and a quantile loss.
6 . The method according to claim 2 wherein the plurality of reference reflectance values comprise a first plurality of reference reflectance values corresponding to at least a first portion of at least a first plant and a second plurality of reference reflectance values corresponding to at least a second portion of at least a second plant, the first and second portions differing in at least one of: species of plant, type of disease, type of damage, and degree of damage.
7 . The method according to claim 6 wherein determining the prediction of plant health comprises:
determining a first prediction of plant health based on the derivative of the plurality of reflectance values and the first plurality of reference reflectance values;
determining a second prediction of plant health based on the derivative of the plurality of reflectance values and the second plurality of reference reflectance values; and
selecting the first prediction based on the first prediction corresponding to at least one of: a greater likelihood of health, a greater measure of confidence, and/or a lesser regression loss metric than the second prediction.
8 . The method according to claim 1 wherein the spectral bases having been generated from one or more images comprising at least one image representing at least a further portion of at least one further plant, wherein optionally the plurality of spectral bases comprises at least four spectral bases.
9 . The method according to claim 1 wherein generating the multispectral image comprises interpolating at least one multispectral reflectance value for a wavelength of at least one of the n multispectral channels outside of the m input multispectral channels; or the method further comprising segmenting the input multispectral image into plant and non-plant regions; wherein generating the multispectral image comprises generating the multispectral image for the plant regions.
10 . The method according to claim 1 comprising:
receiving a calibration input multispectral image representing at least a portion of a calibration subject, the at least the portion of the calibration subject substantially non-reflective in one or more input multispectral channels of the m input multispectral channels; and
determining, for at least one of the one or more input multispectral channels, a corresponding calibration reflectance of at least a portion of the input multispectral image representing at least the portion of the calibration subject;
wherein generating the multispectral image comprises, for the at least one of the one or more input multispectral channels, subtracting the corresponding calibration reflectance.
11 . The method according to claim 1 wherein at least one of the m input multispectral channels comprises at least one wavelength in a range of about 525 nm to 575 nm, at least one wavelength in a range of about 600 nm to 700 nm, and/or at least one wavelength in a range of about 400 nm to 500 nm.
12 . The method according to claim 11 wherein the m input multispectral channels comprise at least four input multispectral channels, and/or wherein the m input multispectral channels comprise no more than ten input multispectral channels.
13 . The method according to claim 11 wherein receiving the input multispectral image comprises causing an imaging sensor having infrared sensitivity to capture one or more frames through one or more optical filters.
14 . The method according to claim 13 wherein:
the imaging sensor comprises at least one of: an RGB imaging sensor with NIR sensitivity and a monochrome imaging sensor;
the one or more optical filters comprise a plurality of optical filters; and
causing the imaging sensor to capture one or more frames comprises causing the imaging sensor to capture a plurality of frames by capturing at least one frame through each of the plurality of optical filters; optionally wherein
causing the imaging sensor to capture the plurality of frames by capturing at least one frame through each of the plurality of optical filters comprises causing the plurality of optical filters to revolve through a field of view of the imaging sensor while causing the imaging sensor to capture frames.
15 . The method according to claim 1 wherein generating the determination comprises:
determining, for a first spatial location of at least one of the input multispectral image and the multispectral image, that a measure of one or more reflectance values of the first spatial location at least one of: exceeds a specularity threshold and is less than non-illumination threshold; and
excluding the one or more reflectance values of the first spatial location from the determination based on said determining.
16 . The method according to claim 1 wherein generating the determination comprises determining the derivative based on a difference between a first reflectance value in a spectral region and a second reflectance value in the spectral region; optionally wherein the spectral region comprises a first wavelength approximately corresponding to a local maximum of reflectance with respect to wavelength and a second wavelength approximately corresponding to a local minimum of reflectance with respect to wavelength, the difference between the first and second reflectance values approximating a slope of reflectance values in the spectral region with respect to wavelength; further optionally wherein determining the derivative based on the difference between the first and second reflectance values comprises normalizing the difference based on a measure of total reflectance in the spectral region; further optionally wherein normalizing the difference based on the measure of total reflectance in the spectral region comprises determining a sum of reflectances at the first and second wavelengths and normalizing based on the sum; further optionally wherein determining the derivative comprises determining a sum comprising a reflectance value at a baseline wavelength outside of the spectral region.
17 . The method according to claim 16 wherein at least the portion of at least one plant comprises at least a portion of a wheat leaf.
18 . The method according to claim 17 wherein the spectral region comprises a region between approximately 570 nm and 670 nm, optionally wherein the first wavelength comprises approximately 570 nm and the second wavelength comprises approximately 670 nm, further optionally wherein determining the derivative comprises determining a sum comprising a reflectance value at a baseline wavelength comprising approximately 500 nm.
19 . A computer system comprising:
one or more processors; and
a memory storing instructions which cause the one or more processors to perform operations comprising:
performing the acts of the method according to claim 1 .