Hierarchal phenotyping using graphs
Implementations are described herein for integrating phenotyping machine learning (ML) models with an agricultural knowledge graph (AKG) to facilitate streamlined and flexible phenotypic inferences. In various implementations, the AKG may be traversed based on input including image(s) of crops growing in an agricultural plot until a destination node is reached. The AKG may include a phenotypic taxonomy of nodes representing a taxonomic hierarchy of organisms, with the destination node corresponding to a level of the phenotypic taxonomy that is commensurate in scope with the input. The phenotypic taxonomy of nodes may provide access to a corresponding taxonomy of phenotyping ML models, each trained to generate phenotypic inference(s) at a specificity that corresponds to a level of the phenotypic taxonomy. The crop image(s) may be processed using the phenotyping ML model that is accessible via the destination node to generate phenotypic inference(s) about the agricultural plot.
1 . A method implemented using one or more processors, the method comprising:
receiving input that includes one or more images of crops growing in an agricultural plot;
based on the input, traversing an agricultural knowledge graph until a destination node is reached, wherein:
the agricultural knowledge graph includes a phenotypic taxonomy of nodes representing a taxonomic hierarchy of organisms;
the destination node corresponds to a level of the phenotypic taxonomy of nodes that is commensurate in scope with the input; and
the phenotypic taxonomy of nodes provide access to a corresponding taxonomy of phenotyping machine learning (ML) models, wherein each phenotyping ML model is trained to generate one or more phenotypic inferences at a level of specificity that corresponds to a level of the phenotypic taxonomy of nodes that contains a respective node that provides access to a respective phenotyping ML model;
processing the one or more images of crops using a first phenotyping ML model that is accessible via the destination node, the processing using the first phenotyping ML model to produce a first phenotypic inference about the agricultural plot;
processing the one or more images of crops using a second phenotyping ML model that is accessible via a node below the destination node, the processing using the second phenotyping ML model to produce a second phenotypic inference about the agricultural plot, the second phenotypic inference including at least one confidence measure associated with an output of the second phenotyping ML model;
determining that the at least one confidence measure included in the output of the processing of the second phenotyping ML model fails to satisfy a threshold;
selecting the destination node based on a failure of the at least one confidence measure associated with the output of the processing of the second phenotyping ML model fails to satisfy the threshold;
based on a selection of the destination node, output the first phenotypic inference generated based on the first phenotyping ML model; and
causing an agricultural operation to be performed based on the first phenotypic inference, wherein the causing of the agricultural operation to be performed on the crops of the one or more images includes deploying a robot to perform a harvesting operation on the crops of the one or more images.
2 . The method of claim 1 , further including operating one or more agricultural vehicles in the agricultural plot in real time based on the first phenotypic inference about the agricultural plot.
3 . The method of claim 1 , wherein the one or more of the phenotypic inferences about the agricultural plot includes an instance count of edible plant matter.
4 . The method of claim 1 , wherein the destination node points to a memory location that stores the phenotyping ML model.
5 . The method of claim 1 , wherein the input includes natural language input, and the traversing includes selecting the destination node based on the natural language input having a scope that matches the level of the phenotypic taxonomy of nodes that contains the destination node.
6 . The method of claim 5 , wherein the selecting is based on the natural language input identifying a taxonomic rank of the destination node.
7 . The method of claim 1 , further including:
processing the one or more images of crops using a third phenotyping ML model and a fourth phenotyping ML model that are accessible via two or more of the phenotypic taxonomy of nodes above the destination node to generate lower-specificity phenotypic inferences about the agricultural plot;
comparing respective confidence measures associated with outputs of the lower-specificity phenotypic inferences; and
traversing the agricultural knowledge graph based at least in part on the comparison of the respective confidence measures.
8 . The method of claim 1 , wherein the input is received from an autonomous vehicle operating in the agricultural plot, and the method further includes providing data indicative of one or more of the phenotypic inferences about the agricultural plot to a computing device operated by a user associated with the agricultural plot, wherein the providing causes the computing device to surface the data.
9 . The method of claim 1 , wherein the robot is to perform an irrigation operation on the crops of the one or more images.
10 . The method of claim 1 , wherein the robot is to perform a trimming operation on the crops of the one or more images.
11 . A system comprising one or more processors and memory storing instructions that, in response to execution by the one or more processors, cause the one or more processors to:
receive input that includes one or more images of crops growing in an agricultural plot;
based on the input, traverse an agricultural knowledge graph until a destination node is reached, wherein:
the agricultural knowledge graph includes a phenotypic taxonomy of nodes representing a taxonomic hierarchy of organisms;
the destination node corresponds to a level of the phenotypic taxonomy of nodes that is commensurate in scope with the input; and
the phenotypic taxonomy of nodes provide access to a corresponding taxonomy of phenotyping machine learning (ML) models, wherein each phenotyping ML model is trained to generate one or more phenotypic inferences at a level of specificity that corresponds to a level of the phenotypic taxonomy of nodes that contains a respective node that provides access to a respective phenotyping ML model;
process the one or more images of crops using a first phenotyping ML model that is accessible via the destination node, the processing using the first phenotyping ML model to produce a first phenotypic inference about the agricultural plot;
process the one or more images of crops using a second phenotyping ML model that is accessible via a node below the destination node to generate a second phenotypic inference about the agricultural plot, the second phenotypic inference including at least one confidence measure associated with an output of the second phenotyping ML model;
determine that the at least one confidence measure included in the output of the processing of the second phenotyping ML model fails to satisfy a threshold;
select the destination node based on a failure of the at least one confidence measure associated with the output of the processing of the second phenotyping ML model to satisfy the threshold;
based on a selection of the destination node, output the first phenotypic inference generated based on the first phenotyping ML model; and
cause an agricultural operation to be performed based on the first phenotypic inference, wherein to cause the agricultural operation to be performed on the crops of the one or more images includes to deploy a robot to perform a harvesting operation on the crops of the one or more images.
12 . The system of claim 11 , wherein the instructions cause the one or more processors to operate one or more agricultural vehicles in the agricultural plot in real time based on the first phenotypic inference about the agricultural plot.
13 . The system of claim 11 , wherein the one or more of the phenotypic inferences about the agricultural plot includes an instance count of edible plant matter.
14 . The system of claim 11 , wherein the destination node points to a memory location that stores the phenotyping ML model.
15 . The system of claim 11 , wherein the input includes natural language input, and the instructions to traverse include instructions to select the destination node based on the natural language input having a scope that matches the level of the phenotypic taxonomy of nodes that contains the destination node.
16 . The system of claim 15 , wherein the destination node is selected is based on the natural language input identifying a taxonomic rank of the destination node.
17 . The system of claim 11 , wherein the instructions cause the one or more processors to:
process the one or more images of crops using a third phenotyping ML model and a fourth phenotyping ML model that are accessible via two or more of the phenotypic taxonomy of nodes above the destination node to generate lower-specificity phenotypic inferences about the agricultural plot;
compare respective confidence measures associated with outputs of the lower-specificity phenotypic inferences; and
traverse the agricultural knowledge graph based at least in part on the comparison of the respective confidence measures.
18 . The system of claim 11 , wherein the input is received from an autonomous vehicle operating in the agricultural plot, and the instructions include instructions to provide data indicative of one or more of the phenotypic inferences about the agricultural plot to a computing device operated by a user associated with the agricultural plot, wherein the providing causes the computing device to surface the data.
19 . At least one non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to:
receive input that includes one or more images of crops growing in an agricultural plot;
based on the input, traverse an agricultural knowledge graph until a destination node is reached, wherein:
the agricultural knowledge graph includes a phenotypic taxonomy of nodes representing a taxonomic hierarchy of organisms;
the destination node corresponds to a level of the phenotypic taxonomy of nodes that is commensurate in scope with the input; and
the phenotypic taxonomy of nodes provide access to a corresponding taxonomy of phenotyping machine learning (ML) models, wherein each phenotyping ML model is trained to generate one or more phenotypic inferences at a level of specificity that corresponds to a level of a corresponding phenotypic taxonomy of nodes that contains a respective node that provides access to a respective phenotyping ML model;
process the one or more images of crops using a first phenotyping ML model that is accessible via the destination node, the processing using the first phenotyping ML model to produce a first phenotypic inference about the agricultural plot;
process the one or more images of crops using a second phenotyping ML model that is accessible via a node below the destination node to generate a second phenotypic inference about the agricultural plot, the second phenotypic inference including at least one confidence measure associated with an output of the second phenotyping ML model;
determine that the at least one confidence measure included in the output of the processing of the second phenotyping ML model fails to satisfy a threshold;
select the destination node based on a failure of the at least one confidence measure associated with the output of the processing of the second phenotyping ML model to satisfy the threshold;
based on the selection of the destination node, output the first phenotypic inference generated based on the first phenotyping ML model; and
cause an agricultural operation to be performed based on the first phenotypic inference, wherein the causing the agricultural operation to be performed on the crops of the one or more images includes deploying a robot to perform a harvesting operation on the crops of the one or more images.
20 . The at least one non-transitory computer-readable medium of claim 19 , wherein the instructions cause the one or more processors to operate one or more agricultural vehicles in the agricultural plot in real time based on the first phenotypic inference about the agricultural plot.