Compensating for occlusions in a detection system of a farming machine
A farming machine is configured to identify and compensate for occlusions in the field of view of its image acquisition system. To do so, the machine captures an image using a first set of capture parameters associated with a first set of treatment results. The farming machine identifies an occlusion in the first image that obstructs a portion of the first image and determines occlusion characteristics representative of the occlusion based on image data in the first image. The farming machine compensates for the identified occlusion based on the occlusion characteristics. The farming machine captures a second image using modified set of capture parameters that compensate for the occlusion. The second image is associated with a second set of treatment results. The farming machine transmits the second set of treatment results to a manager of the farming machine.
1 . A method of operating a farming machine comprising:
capturing a first image of plants in a field with an image acquisition system of the farming machine using an initial set of capture parameters, the farming machine performing spray treatment actions of a treatment plan, and the image acquisition system configured to capture images with different capture parameters;
determining that the farming machine is performing spray treatment actions of the treatment plan with a first set of treatment results, wherein treatment results are a representation of an accuracy of performed spray treatment actions of plants in the field;
identifying an occlusion in the first image by applying an object detection model to pixels in the first image, the occlusion obstructing a field of view of the image acquisition system in a portion of the first image;
determining occlusion characteristics representative of the occlusion based on image data in the first image;
generating, using the occlusion characteristics, a dynamic mask for applying to pixels of images captured by the image acquisition system, the dynamic mask implementing a modified set of capture parameters to compensate for the identified occlusion; and
responsive to generating the dynamic mask:
capturing a second image of plants in the field with the image acquisition system using the dynamic mask and the modified set of capture parameters that compensate for the occlusion;
determining that the farming machine is performing spray treatment actions of the treatment plan with a second set of treatment results, the second set of treatment results having are decreased in accuracy with occlusion compensation relative to the first set of treatment results without occlusion compensation by at least a threshold difference; and
transmitting the second set of treatment results to a manager of the farming machine.
2 . The method of claim 1 , wherein the dynamic mask is configured to remove pixels from the images obstructed by the occlusion.
3 . The method of claim 1 , wherein the dynamic mask is configured to remove outputs of the object detection model classified as the occlusion.
4 . The method of claim 1 , wherein identifying the occlusion in the first image comprises applying an object detection model to the first image, the object detection model configured to classify at least plants and occlusions in the first image.
5 . The method of claim 1 , wherein the occlusion is a part of the farming machine.
6 . The method of claim 1 , wherein the first set of treatment results and the second set of treatment results quantify an expected amount of treatment fluid applied in the field, and the second set of treatment results are higher than the first set of treatment results.
7 . The method of claim 1 , wherein the first set of treatment results and the second set of treatment results quantify an expected number of plants treated by the farming machine, the second set of treatment results are different from the first set of treatment results.
8 . A farming machine comprising:
an image acquisition system configured to capture images of plants in a field according to capture parameters;
one or more sensors configured to measure treatment results of the farming machine;
one or more processors physically attached to the farming machine; and
a non-transitory computer readable storage medium storing computer program instructions that, when executed by the one or more processors, cause the one or more processors to:
capture, using the image acquisition system, a first image of plants in the field using an initial set of capture parameters associated with a first set of treatment results, the farming machine performing spray treatment actions of a treatment plan;
determine that the farming machine is performing spray treatment actions of the treatment plan with a first set of treatment results, wherein treatment results are a representation of an accuracy of performed spray treatment actions of plants in the field;
identify an occlusion in the first image by applying an object detection model to pixels in the first image, the occlusion obstructing a field of view of the image acquisition system in a portion of the first image;
determine occlusion characteristics representative of the occlusion based on image data in the first image;
generate, using the occlusion characteristics, a dynamic mask for applying to pixels of images captured by the image acquisition system, the dynamic mask implementing a modified set of capture parameters to compensate for the identified occlusion; and
responsive to generating the dynamic mask:
capture a second image of plants in the field with the image acquisition system using the dynamic mask and the modified set of capture parameters that compensate for the occlusion;
determine that the farming machine is performing spray treatment actions of the treatment plan with a second set of treatment results, the second set of treatment results having are decreased in accuracy with occlusion compensation relative to the first set of treatment results without occlusion compensation by at least a threshold difference; and
transmit the second set of treatment results to a manager of the farming machine.
9 . The farming machine of claim 8 , wherein the dynamic mask is configured to remove pixels from the images obstructed by the occlusion.
10 . The farming machine of claim 8 , wherein the dynamic mask is configured to remove outputs of the object detection model classified as the occlusion.
11 . The farming machine of claim 8 , wherein computer program instructions identifying the occlusion in the first image further causes the one or more processors to apply an object detection model to the first image, the object detection model configured to classify at least plants and occlusions in the first image.
12 . The farming machine of claim 8 , wherein the occlusion is a part of the farming machine.
13 . The farming machine of claim 8 , wherein the first set of treatment results and the second set of treatment results quantify an expected amount of treatment fluid applied in the field, and the second set of treatment results higher than the first set of treatment results.
14 . The farming machine of claim 8 , wherein the first set of treatment results and the second set of treatment results quantify an expected number of plants treated by the farming machine, and the second set of treatment results are different from the first set of treatment results.
15 . A non-transitory computer readable storage medium storing computer program instructions for operating a farming machine that, when executed by one or more processors, cause the one or more processors to:
capture a first image of plants in a field with an image acquisition system of the farming machine using an initial set of capture parameters, the farming machine performing spray treatment actions of a treatment plan, and the image acquisition system configured to capture images with different capture parameters:
determine that the farming machine is performing spray treatment actions of the treatment plan with a first set of treatment results, wherein treatment results are a representation of an accuracy of performed spray treatment actions of plants in the field;
identify an occlusion in the first image by applying an object detection model to pixels in the first image, the occlusion obstructing a field of view of the image acquisition system in a portion of the first image;
determine occlusion characteristics representative of the occlusion based on image data in the first image;
generate, using the occlusion characteristics, a dynamic mask for applying to pixels of images captured by the image acquisition system, the dynamic mask implementing a modified set of capture parameters to compensate for the identified occlusion;
responsive to generating the dynamic mask:
capture a second image of plants in the field with the image acquisition system using the dynamic mask and the modified set of capture parameters that compensate for the occlusion;
determine that the farming machine is performing actions of the treatment plan with a second set of treatment results, the second set of treatment results having are decreased in accuracy with occlusion compensation relative to the first set of treatment results without occlusion compensation by at least a threshold difference; and
transmitting the second set of treatment results to a manager of the farming machine.