Individual plant recognition and localization
Implementations are described herein for training and applying machine learning models to digital images capturing plants, and to other data indicative of attributes of individual plants captured in the digital images, to recognize individual plants in distinction from other individual plants. In various implementations, a digital image that captures a first plant of a plurality of plants may be applied, along with additional data indicative of an additional attribute of the first plant observed when the digital image was taken, as input across a machine learning model to generate output. Based on the output, an association may be stored in memory, e.g., of a database, between the digital image that captures the first plant and one or more previously-captured digital images of the first plant.
1. A method implemented using one or more processors, comprising:
obtaining a first digital image that captures a depiction of a first individual plant of a plurality of plants;
determining, based on a sensor signal generated by a sensor, additional data indicative of an additional attribute of the first individual plant;
applying both the first digital image and the additional data as inputs across a trained machine learning model to generate an embedding, wherein the machine learning model was previously trained using a plurality of training instances and triplet loss, and wherein each training instance includes one or more training digital images of a particular plant and additional data indicative of an additional attribute of the particular plant when the one or more training digital images were taken;
determining distances in latent space between the embedding and a plurality of reference embeddings generated from a plurality of digital images captured previously to the first digital image;
based on the distances, matching the depiction of the first individual plant captured in the first digital image to one or more depictions of the same first individual plant captured in a subset of one or more of the previously-acquired digital images; and
based on the matching, storing in memory an association between the first digital image that captures the depiction of the first individual plant and the subset of one or more previously-acquired digital images that capture the one or more matching depictions of the same first individual plant during an earlier stage of growth.
2. The method of claim 1 , wherein the obtaining includes:
operating an autonomous or semiautonomous vehicle equipped with a vision sensor to traverse along a path that is proximate to at least a subset of the plurality of plants; and
operating the vision sensor to acquire the digital image that captures the first individual plant.
3. The method of claim 1 , wherein the machine learning model comprises a convolutional neural network.
4. The method of claim 1 , wherein the sensor comprises a position coordinate sensor, and the additional attribute of the first individual plant comprises a position coordinate indicative of a location of the first individual plant.
5. The method of claim 1 , wherein the sensor comprises a vision sensor.
6. The method of claim 5 , wherein the additional attribute of the first individual plant that is applied as input across the machine learning model to generate the embedding comprises a spatial dimension of a bounding shape, determined from one or more digital images generated by the vision sensor, that encloses at least a portion of the first individual plant.
7. The method of claim 6 , wherein the bounding shape comprises a minimum bounding shape that encloses an entirety of the first individual plant.
8. The method of claim 5 , wherein the additional attribute of the first individual plant comprises an environmental context of the first individual plant captured in one or more digital images generated by the vision sensor.
9. The method of claim 5 , wherein the additional attribute of the first individual plant comprises a time interval since a milestone in a life of the first individual plant.
10. The method of claim 1 , further comprising:
receiving, from a graphical user interface rendered on a computing device, user selection of a user interface element that corresponds to the first individual plant; and
based on the receiving, causing the computing device to output first individual plant information based on the digital image that captures the depiction of the first individual plant and one or more of the previously-acquired digital images that capture the one or more matching depictions of the first individual plant.
11. The method of claim 10 , wherein the first individual plant information includes a time-based sequence of digital images that includes the digital image that captures the depiction of the first individual plant and one or more of the previously-acquired digital images that capture the one or more matching depictions of the first individual plant.
12. The method of claim 10 , wherein the first individual plant information includes first individual plant growth or health statistics generated based on the digital image that captures the depiction of the first individual plant and one or more of the previously-acquired digital images that capture the one or more matching depictions of the first individual plant.
13. The method of claim 1 , further comprising:
analyzing the digital image that captures the depiction of the first individual plant and one or more of the previously-acquired digital images that capture the one or more matching depictions of the first individual plant; and
based on the analyzing, predicting a growth rate or yield of the first individual plant.
14. A system comprising one or more processors and memory storing instructions that, in response to execution of the instructions by the one or more processors, cause the one or more processors to:
obtain a first digital image that captures a depiction of a first individual plant of a plurality of plants;
determine, based on a sensor signal generated by a sensor, additional data indicative of an additional attribute of the first individual plant;
applying both the first digital image and the additional data as inputs across a trained machine learning model to generate an embedding, wherein the machine learning model was previously trained using a plurality of training instances and triplet loss, and wherein each training instance includes one or more training digital images of a particular plant and additional data indicative of an additional attribute of the particular plant when the one or more training digital images were taken;
determine distances in latent space between the embedding and a plurality of reference embeddings generated from a plurality of digital images captured previously to the first digital image;
based on the distances, match the depiction of the first individual plant captured in the first digital image to one or more matching depictions of the same first individual plant captured in a subset of one or more of the previously-acquired digital images; and
based on the match, store in memory an association between the first digital image that captures the depiction of the first individual plant and the subset of one or more previously-acquired digital images that capture the one or more matching depictions of the same first individual plant during an earlier stage of growth.
15. The system of claim 14 , wherein the obtaining includes:
operating an autonomous or semiautonomous vehicle equipped with a vision sensor to traverse along a path that is proximate to at least a subset of the plurality of plants; and
operating the vision sensor to acquire the digital image that captures the depiction of the first individual plant.
16. The system of claim 14 , wherein the machine learning model comprises a convolutional neural network.
17. The system of claim 14 , wherein the sensor comprises a position coordinate sensor, and the additional attribute of the first individual plant comprises a position coordinate indicative of a location of the first individual plant.
18. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to perform the following operations:
obtaining a first digital image that captures a depiction of a first individual plant of a plurality of plants;
determining, based on a sensor signal generated by a sensor, additional data indicative of an additional attribute of the first individual plant;
applying both the first digital image and the additional data as inputs across a trained machine learning model to generate an embedding, wherein the machine learning model was previously trained using a plurality of training instances and triplet loss, and wherein each training instance includes one or more training digital images of a particular plant and additional data indicative of an additional attribute of the particular plant when the one or more training digital images were taken;
determining distances in latent space between the embedding and a plurality of reference embeddings generated from a plurality of digital images captured previously to the first digital image;
based on the distances, matching the depiction of the first individual plant captured in the first digital image to one or more depictions of the same first individual plant captured in a subset of one or more of the previously-acquired digital images; and
based on the matching, storing in memory an association between the first digital image that captures the depiction of the first individual plant and the subset of one or more previously-acquired digital images that capture the one or more matching depictions of the same first individual plant during an earlier stage of growth.