Plant detection and display system

1 . A server comprising:

at least one memory storing instructions; and

at least one processor that, upon execution of the stored instructions, is configured to

obtain, from an image capture device, an image that includes a plurality of plants growing in at least one grow tray and one or more tray markers identifying each of the at least one grow tray;

execute detection processing on the obtained image of the plants growing in the at least one grow tray using a trained model to classify each of the plurality of plants into a first category indicative of a defective plant or a second category indicative of a normal plant and generate information indicating a classification accuracy associated with each of the plants classified in the first category, wherein the trained model is a machine learning model trained using one or more visual characteristics of leaves of the plurality of plants indicating a defective plant;

determine, from the obtained image, position information for each of the plurality of plants classified in the first category, the position information being determined using the one or more tray markers associated with the at least one tray and captured in the obtained image;

generate, using the one or more tray markers in the position information, a map of the plurality of plants classified in the first category including an identifier around each of the plurality of plants classified in the first category; and

provide the generated map and a live view image of the at least one grow tray being captured by the image capture device to an application executing on a terminal device causing an augmented reality view to be generated for display on a display of the terminal device, wherein the augmented reality view includes the live view image of the plurality of plants in the at least one grow tray and the identifier providing visual representation of a location of each plant classified in the first class as a defective plant, wherein each identifier includes the generated information depicting an accuracy of the classification.

2 . The server according to claim 1 , wherein

the detection processing including a first analysis and a second analysis, the first analysis identifies a shape of leaves in the image of the plurality of plants, and the second analysis identifies defective leaves.

3 . The server according to claim 1 , wherein execution of the instructions further configures the at least one processor to

receive, from the terminal device, a request to change a respective plant classified in the first category to be classified in the second category;

update the generated map based on the received change request, and

use the updated map to cause the respective plant changed from the first category to the second category to not be identified within the image.

4 . The server according to claim 3 , wherein execution of the instructions further configures the at least one processor to

store, in the at least one memory, the updated map in association with obtained image as corrected image data, the updated map including plants classified in the first category using the trained model and plants having been corrected by a user;

providing the corrected image data to training module used in generating the trained model to generate an updated trained model.

5 . The server according to claim 1 , wherein execution of the instructions further configures the at least one processor to

in response to detecting that the terminal device is proximate to the image capture device, communicate the generated map to an application executing on the terminal device causing the application to generate an augmented reality view including a live view of the plurality of plants having one or more indicators from the map overlaid on the captured live view of the plurality of plants.

6 . The server according claim 1 , wherein execution of the instructions further configures the at least one processor to

control a picking device using the generated map to cause the picking device to remove each of the plants classified in the first category using the position information within the map.

7 . The server according to claim 1 , wherein execution of the instructions configures the at least one processor to

continually obtain over a period of time, from the image capture device, images of a plurality of plants;

for each of the continually obtained images, execute detection processing using the trained model to classify each of the plurality of plants into the first category indicative of a defective plant having a confidence score below a predetermined confidence threshold or a second category indicative of a normal plant, wherein the trained model is a machine learning model trained using one or more visual characteristics associated with the plurality of plants indicating a defective plant;

generating, a quality score representing the plurality of plants based on a number of respective ones of the plurality of plants being classified in the first category;

using the quality score to control one or more parameters used to grow the plurality of plants.

8 . A method of classifying plants comprising:

obtaining, from an image capture device, an image that includes of a plurality of plants growing in at least one grow tray and one or more tray markers identifying each of the at least one grow tray;

executing, by at least one processor, detection processing on the obtained image of the plants growing in the at least one grow tray using a trained model to classify each of the plurality of plants into a first category indicative of a defective plant or a second category indicative of a normal plant and generate information indicating a classification accuracy associated with each of the plants classified in the first category, wherein the trained model is a machine learning model trained using one or more visual characteristics of leaves of the plurality of plants indicating a defective plant;

determining, from the obtained image by the at least one processor, position information for each of the plurality of plants classified in the first category, the position information being determined using the one or more tray markers associated with the at least one tray and captured in obtain image;

generating, using the one or more tray markers in the position information by the at least one processor, a map of the plurality of plants classified in the first category including an identifier around each of the plurality of plants classified in the first category; and

providing the generated map and a live view image of the at least one grow tray being captured by the image capture device to an application executing on a terminal device causing an augmented reality view to be generated for display on a display of the terminal device, wherein the augmented reality view includes the live view image of the plurality of plants in the at least one grow tray and the identifier providing visual representation of a location of each plant classified in the first class as a defective plant, wherein each identifier includes the generated information depicting an accuracy of the classification.

9 . The method according to claim 8 , wherein

the detection processing including a first analysis and a second analysis, the first analysis identifies a shape of leaves in the image of the plurality of plants, and the second analysis identifies defective leaves.

10 . The method according to claim 8 , further comprising

receiving, from the terminal device, a request to change a respective plant classified in the first category to be classified in the second category;

updating the generated map based on the received change request, and

using the updated map to cause the respective plant changed from the first category to the second category to not be identified within the image.

11 . The method according to claim 10 , further comprising

storing, in the at least one memory, the updated map in association with obtained image as corrected image data, the updated map including plants classified in the first category using the trained model and plants having been corrected by a user;

providing the corrected image data to training module used in generating the trained model to generate an updated trained model.

12 . The method according to claim 8 , further comprising

in response to detecting that the terminal device is proximate to the image capture device, communicate the generated map to an application executing on the terminal device causing the application to generate an augmented reality view including a live view of the plurality of plants having one or more indicators from the map overlaid on the captured live view of the plurality of plants.

13 . The method according claim 8 , further comprising

controlling a picking device using the generated map to cause the picking device to remove each of the plants classified in the first category using the position information within the map.

14 . The method according to claim 8 , further comprising

continually obtaining over a period of time, from the image capture device, images of a plurality of plants;

for each of the continually obtained images, executing detection processing using the trained model to classify each of the plurality of plants into the first category indicative of a defective plant having a confidence score below a predetermined confidence threshold or a second category indicative of a normal plant, wherein the trained model is a machine learning model trained using one or more visual characteristics associated with the plurality of plants indicating a defective plant;

generating, a quality score representing the plurality of plants based on a number of respective ones of the plurality of plants being classified in the first category;

using the quality score to control one or more parameters used to grow the plurality of plants.

15 . A non-transitory storage medium that stores instructions, that when executed by one or more processers, configures the one or more processors to perform:

obtaining, from an image capture device, an image that includes a plurality of plants growing in at least one grow tray and one or more tray markers identifying each of the at least one grow tray;

executing, by at least one processor, detection processing on the obtained image of the plants growing in the at least one grow tray using a trained model to classify each of the plurality of plants into a first category indicative of a defective plant or a second category indicative of a normal plant and generate information indicating a classification accuracy associated with each of the plants classified in the first category, wherein the trained model is a machine learning model trained using one or more visual characteristics of leaves of the plurality of plants indicating a defective plant;

determining, from the obtained image by the at least one processor, position information for each of the plurality of plants classified in the first category, the position information being determined using the one or more tray markers associated with the at least one tray and captured in the obtained image;

generating, using the one or more tray markers in the position information by the at least one processor, a map of the plurality of plants classified in the first category including an identifier around each of the plurality of plants classified in the first category; and

providing the generated map and a live view image of the at least one grow tray being captured by the image capture device to an application executing on a terminal device causing an augmented reality view to be generated for display on a display of the terminal device, wherein the augmented reality view includes the live view image of the plurality of plants in the at least one grow tray and the identifier providing visual representation of a location of each plant classified in the first class as a defective plant, wherein each identifier includes the generated information depicting an accuracy of the classification.

16 . A server comprising:

at least one memory storing instructions; and

at least one processor that, upon execution of the stored instructions, is configured to

obtain, from an image capture device, an image that includes of a plurality of plants growing in at least one grow tray and one or more tray markers identifying each of the at least one grow tray;

execute detection processing on the obtained image of the plants growing in the at least one grow tray using a trained model to classify each of the plurality of plants into a first category indicative of a defective plant or a second category indicative of a normal plant and generate information indicating a classification accuracy associated with each of the plants classified in the first category,

wherein, the detection processing including a first analysis and a second analysis, the first analysis identifies a shape of leaves in the image of the plurality of plants, and the second analysis identifies defective leaves

wherein the trained model is a machine learning model trained using one or more visual characteristics of leaves of the plurality of plants indicating a defective plant;

determine, from the obtained image, position information for each of the plurality of plants classified in the first category, the position information being determined using the one or more tray markers associated with the at least one tray and captured in the obtained image;

generate, using the one or more tray markers in the position information, a map of the plurality of plants classified in the first category including an identifier around each of the plurality of plants classified in the first category; and

provide the generated map and a live view image of the at least one grow tray being captured by the image capture device to an application executing on a terminal device causing an augmented reality view to be generated for display on a display of the terminal device, wherein the augmented reality view includes the live view image of the plurality of plants in the at least one grow tray and the identifier providing visual representation of a location of each plant classified in the first class as a defective plant, wherein each identifier includes the generated information depicting an accuracy of the classification.