Plant group identification
A farming machine moves through a field and includes an image sensor that captures an image of a plant in the field. A control system accesses the captured image and applies the image to a machine learned plant identification model. The plant identification model identifies pixels representing the plant and categorizes the plant into a plant group (e.g., plant species). The identified pixels are labeled as the plant group and a location of the pixels is determined. The control system actuates a treatment mechanism based on the identified plant group and location. Additionally, the images from the image sensor and the plant identification model may be used to generate a plant identification map. The plant identification map is a map of the field that indicates the locations of the plant groups identified by the plant identification model.
1. A method comprising:
accessing images including plants, the plants classified into plant groups from a plurality of plant groups by at least one plant identification model;
accessing information indicating plant treatment actions applied to one or more of the plants based on the plant groups classified by the at least one plant identification model;
combining the accessed images to generate a plant identification map, the plant identification map being a geographical map indicating locations of individual plants of the plant groups and locations of plant treatment actions applied to one or more of the individual plants; and
sending the plant identification map to a device for display.
2. The method of claim 1 , wherein the plant identification map indicates a total area covered by one of the plant groups in a field.
3. The method of claim 1 , wherein the plant identification map indicates a total number of plants in one of the plant groups in a field.
4. The method of claim 1 , wherein the plant identification map is a spatially indexed geographical map of a field.
5. The method of claim 1 , wherein individual plants of a first plant group are highlighted with a first color in the plant identification map and individual plants of a second plant group different than the first plant group are highlighted with a second color different than the first color in the plant identification map, the highlights assisting a user to recognize locations of the individual plants in the first and second plant groups.
6. The method of claim 1 , further comprising: accessing extrinsic and intrinsic parameters of one or more image sensors that captured the accessed images,
wherein combining the accessed images to generate the plant identification map comprises:
based on the extrinsic and intrinsic parameters, determining geospatial coordinates for one or more of the accessed images; and
generating the plant identification map based on the determined geospatial coordinates for the one or more accessed images.
7. The method of claim 6 , wherein combining the accessed images to generate the plant identification map further comprises associating pixels in an accessed image to a geographic location on a ground plan.
8. The method of claim 1 , wherein:
the accessed images include first images captured by a first image sensor of a first type and second images captured by a second image sensor of a second type; and
the plant identification map includes a first layer formed by combining the first images and a second layer formed by combining the second images.
9. The method of claim 8 , wherein the first image sensor of the first type is a visible wavelength image sensor, and the second image sensor of the second type is an infrared image sensor.
10. The method of claim 1 , wherein the plant identification map is a heat map, wherein colors of the heat map indicate spatial densities of the plant groups.
11. The method of claim 1 , wherein the plant identification map indicates at least one of:
a total plant group area identified by the at least one plant identification model;
a total area treated by the plant treatment actions;
a time taken to capture an image of a plant;
a time taken to classify the plant into a plant group,
a time to determine a first plant treatment action to be applied to the plant;
a time to apply the first plant treatment action to the plant; or
a metric indicating an accuracy of a machine that applies the plant treatment actions to the one or more individual plants.
12. A non-transitory computer-readable storage medium storing instructions that, when executed by a control system, causes the control system to perform operations comprising:
accessing images including plants, the plants classified into plant groups from a plurality of plant groups by at least one plant identification model;
accessing information indicating plant treatment actions applied to one or more of the plants based on the plant groups classified by the at least one plant identification model;
combining the accessed images to generate a plant identification map, the plant identification map being a geographical map indicating locations of individual plants of the plant groups and locations of plant treatment actions applied to one or more of the individual plants; and
sending the plant identification map to a device for display.
13. The non-transitory computer-readable storage medium of claim 12 , wherein the plant identification map indicates a total area covered by one of the plant groups in a field.
14. The non-transitory computer-readable storage medium of claim 12 , wherein the plant identification map indicates a total number of plants in one of the plant groups in a field.
15. The non-transitory computer-readable storage medium of claim 12 , wherein the plant identification map is a spatially indexed geographical map of a field.
16. The non-transitory computer-readable storage medium of claim 12 , wherein individual plants of a first plant group are highlighted with a first color in the plant identification map and individual plants of a second plant group different than the first plant group are highlighted with a second color different than the first color in the plant identification map, the highlights assisting a user to recognize locations of the individual plants in the first and second plant groups.
17. The non-transitory computer-readable storage medium of claim 12 , further comprising: accessing extrinsic and intrinsic parameters of one or more image sensors that captured the accessed images,
wherein combining the accessed images to generate the plant identification map comprises:
based on the extrinsic and intrinsic parameters, determining geospatial coordinates for one or more of the accessed images; and
generating the plant identification map based on the determined geospatial coordinates for the one or more accessed images.
18. The non-transitory computer-readable storage medium of claim 17 , wherein combining the accessed images to generate the plant identification map further comprises associating pixels in an accessed image to a geographic location on a ground plan.
19. The non-transitory computer-readable storage medium of claim 12 , wherein:
the accessed images include first images captured by a first image sensor of a first type and second images captured by a second image sensor of a second type; and
the plant identification map includes a first layer formed by combining the first images and a second layer formed by combining the second images.
20. The non-transitory computer-readable storage medium of claim 19 , wherein the plant identification map indicates at least one of:
a total plant group area identified by the at least one plant identification model;
a total area treated by the plant treatment actions;
a time taken to capture an image of a plant;
a time taken to classify the plant into a plant group,
a time to determine a first plant treatment action to be applied to the plant;
a time to apply the first plant treatment action to the plant; or
a metric indicating an accuracy of a machine that applies the plant treatment actions to the one or more individual plants.