System and method of detection and identification of crops and weeds

1 . A detection and identification system for detecting and identifying plant species in an agricultural field that allows acting on them, comprising:

at least one camera comprising a lens, a bandpass filter for allowing wavelengths between 620 nm and 660 nm, and between 780 nm and 900 nm and an image sensor having a dynamic range of at least 60 dB placed after the bandpass filter, wherein each of the at least one camera is positioned at a height between 50 cm and 180 cm with respect to the ground, and wherein each of the at least one camera has an angle of inclination between 40 and 60 degrees towards the ground and forward, and wherein the image sensor of each of the at least one camera allows images with a wide aspect ratio to be captured;

an inertial measurement unit for each of the at least one camera, wherein each inertial measurement unit is rigidly attached to a respective camera;

a lighting source for each of the at least one camera, wherein each lighting source is positioned parallel to a respective camera;

a GPS receiver; and

at least one data processing unit comprising at least one processor and memory storing instructions, wherein each of the at least one data processing unit is in data communication with the at least one camera, the inertial measurement unit and lighting source of each camera, and with the GPS receiver,

wherein each of the at least one data processing unit comprises a camera calibration system for calibrating each of the at least one camera in terms of intrinsic and extrinsic parameters,

wherein each of the at least one camera captures and sends filtered images of the agricultural field to the at least one data processing unit,

wherein each of the at least one data processing unit is configured to:

correct an image using data from the inertial measurement unit, calculate a Normalized Difference Vegetation Index (NDVI) and subsequently separate the filtered image into patches so as to divide the image into a smaller size frame grid, and autonomously detect and identify plant species in each portion of the image using convolutional neural networks trained with a database containing images corresponding to the patches into which the filtered images were separated, discriminating between crops, weeds and soil, based on the images it receives, wherein the training of the deep learning algorithms includes data augmentation techniques,

wherein only the patches that have a probability exceeding a predetermined threshold of containing the plant species to be detected are retained and used for decision-making, according to what is detected and identified,

wherein each of the at least one data processing unit geolocates the detected plant species in each portion of the image, thereby allowing to generate weed and crop maps for each portion of the image that can be multi-layered, and

wherein each of the at least one data processing unit is in data communication with a valve controller so as to independently drive a set of spray valves, each valve covering a portion of the agricultural field corresponding to the portion of the image.

2 . The detection and identification system according to claim 1 , wherein the system comprises at least two cameras, wherein each of the at least two cameras is positioned at a height of 140 cm with respect to the ground, and wherein each of the at least two cameras has an angle of inclination of 50 degrees towards the ground and forward.

3 . The detection and identification system according to claim 1 , wherein each of the at least one camera is a high dynamic range camera having an image sensor with a dynamic range of 120 dB.

4 . The detection and identification system according to claim 1 , wherein the system is mounted on an agricultural vehicle.

5 . The detection and identification system according to claim 4 , wherein the agricultural vehicle is selected from an agrochemical spraying vehicle, a vehicle that pulls a trailed sprayer or an electric robot for agricultural operations.

6 . The detection and identification system according to claim 1 , wherein the system is mounted on a spray boom.

7 . The detection and identification system according to claim 1 , wherein each of the at least one camera is positioned at a height between 80 cm and 160 cm with respect to the ground.

8 . The detection and identification system according to claim 1 , wherein each of the at least one camera has an angle of inclination of 50 degrees towards the ground and forward.

9 . The detection and identification system according to claim 1 , wherein the system comprises at least two cameras spaced from each other between 0.5 m and 5 m.

10 . A detection and identification method that uses the detection and identification system of plant species according to claim 1 to detect and identify plant species in an agricultural field, comprising the following steps:

a sensing step, wherein an image or frame of the soil that can contain living or dead plant species is captured by a camera and filtered by a bandpass filter allowing wavelengths between 620 nm and 660 nm, and between 780 nm and 900 nm, and wherein a data processing unit comprising at least one processor and memory storing instructions receives data from the GPS receiver and the inertial measurement unit of said camera, and carries out configurations in real time and iteratively to each camera through the calibration system;

an image processing step, wherein the data processing unit performs any type of subsequent correction necessary to the captured image, calculates a Normalized Difference Vegetation Index (NDVI), and separates the image into patches or rectangular patches so as to divide the image into a grid of smaller frames;

a detection and identification step, wherein the content of each patch is detected and identified by the data processing unit which employs deep learning algorithms comprising convolutional neural networks that are trained with a database that contains patches comprising plant species and soil, said patches being of the same size as the patches of the previous step,

wherein the content of each of the patches of the database is labeled,

wherein the patches of the database are obtained from images captured by cameras under various conditions in the agricultural environment and from modified images that present changes with respect to said images captured, wherein the training of the deep learning algorithms allows the deep learning algorithms of the data processing unit to have filters to detect and identify plant species,

wherein the training of the deep learning algorithms includes data augmentation techniques, and

wherein only the patches that have a probability exceeding a predetermined threshold of containing the plant species to be detected are filtered;

a decision step, wherein the plant species detected and identified in the previous step are geolocated through the data processing unit and weed and crop maps for each portion of the image that can be multi-layered are generated,

wherein each of the at least one data processing unit sends data to a valve controller so as to independently drive a set of spray valves, each valve covering a portion of the agricultural field corresponding to the portion of the image, and

wherein decisions are made regarding the way to act on said detected plant species, and—an application step, wherein the data processing unit determines when, where and for how long each valve must act to carry out an action with respect to the detected and identified plant species, and carries out said action.