Colorizing x-ray images
Implementations are described herein for colorizing an X-ray image and predicting one or more phenotypic traits about a plant based on the colorized X-ray image. In various implementations, an X-ray image that depicts a plant with a canopy of the plant partially occluding a part-of-interest is obtained, where the part-of-interest is visible through the canopy in the X-ray image. The X-ray images is colorized to predict one or more phenotypic traits of the part-of-interest. The colorization includes processing the X-ray image based on a machine learning model to generate a colorized version of the X-ray image, and predicting the one or more phenotypic traits based on one or more visual features of the colorized version of the X-ray image.
1. A method implemented using one or more processors, the method comprising:
obtaining a monochrome X-ray image that depicts a plant, wherein a canopy of the plant at least partially occludes one or more plant-parts-of-interest from a vantage point at which an X-ray sensor captured the monochrome X-ray image, and wherein the one or more plant-parts-of-interest are visible through the canopy in the monochrome X-ray image;
obtaining environmental data of an agricultural area where the plant is grown, wherein the environmental data is obtained using one or more of a moisture sensor, a laser, a barometer, a photodiode, or a thermometer;
obtaining time-series data corresponding to the environmental data of the agricultural area where the plant is grown;
executing a machine learning model to generate a Red Green Blue (RGB) version of the X-ray image, wherein the monochrome X-ray image, the environmental data of the agricultural area where the plant is grown, and the time-series data are inputs to the machine learning model; and
predicting one or more phenotypic traits of the one or more plant-parts-of-interest based on one or more visual features of the RGB version of the X-ray image.
2. The method of claim 1 , wherein the machine learning model comprises a generator model of a generative adversarial network that also includes a discriminator model.
3. The method of claim 1 , wherein the environmental data of the agricultural area in which the plant is grown includes one or more of temperature, precipitation, sunlight exposure, fertilizer application, soil composition, pH levels, or pesticide application.
4. The method of claim 1 , wherein the plant comprises a melon plant with one or more melons, and the one or more visual features comprise one or more spatial dimensions of seams of the one or more melons.
5. The method of claim 1 , wherein the one or more plant-parts-of-interest include a berry portion of a berry plant.
6. The method of claim 1 , wherein predicting the one or more phenotypic traits is further based on one or more visual features of the canopy depicted in the X-ray image.
7. A system comprising one or more processors and memory storing instructions that, in response to execution of the instructions by the one or more processors, cause the one or more processors to:
obtain a monochrome X-ray image that depicts a plant, wherein a canopy of the plant at least partially occludes one or more plant-parts-of-interest from a vantage point at which an X-ray sensor captured the monochrome X-ray image, and wherein the one or more plant-parts-of-interest are visible through the canopy in the monochrome X-ray image;
obtain environmental data of an agricultural area where the plant is grown, wherein the environmental data is obtained using one or more of a moisture sensor, a laser, a barometer, a photodiode, or a thermometer;
obtain time-series data corresponding to the environmental data of the agricultural area where the plant is grown;
execute a machine learning model to generate a Red Green Blue (RGB) version of the X-ray image, wherein the monochrome X-ray image, the environmental data of the agricultural area where the plant is grown, and the time-series data are inputs to the machine learning model; and
predict one or more phenotypic traits of the one or more plant-parts-of-interest based on one or more visual features of the RGB version of the X-ray image.
8. The system of claim 7 , wherein the machine learning model comprises a generator model of a generative adversarial network that also includes a discriminator model.
9. The system of claim 7 , wherein the environmental data of the agricultural area in which the plant is grown includes one or more of temperature, precipitation, sunlight exposure, fertilizer application, soil composition, pH levels, or pesticide application.
10. The system of claim 7 , wherein the plant comprises a melon plant with one or more melons, and the one or more visual features comprise one or more spatial dimensions of seams of the one or more melons.
11. The system of claim 7 , wherein the one or more plant-parts-of-interest include a berry portion of a berry plant.
12. The system of claim 7 , the one or more phenotypic traits are further predicted based on one or more visual features of the canopy depicted in the X-ray image.