Autonomous agricultural treatment system using map based targeting of agricultural objects
Various embodiments of an apparatus, methods, systems and computer program products described herein are directed to an agricultural observation and treatment system and method of operation. The agricultural treatment system may determine a first real-world geo-spatial location of the treatment system. The system can receive captured images depicting real-world agricultural objects of a geographic scene. The system can associate captured images with the determined geo-spatial location of the treatment system. The treatment system can identify, from a group of mapped and indexed images, images having a second real-word geo-spatial location that is proximate with the first real-world geo-spatial location. The treatment system can compare at least a portion of the identified images with at least a portion of the captured images. The treatment system can determine a target object and emit a fluid projectile at the target object using a treatment device.
1. A method performed by an agricultural treatment system comprising one or more processors comprising hardware, one or more image sensors, and a treatment unit, the one or more processors configured to perform operations comprising:
storing a group of images on a data storage device, the group of images depicting views of real-world agricultural objects, the one or more stored images having associated real-world geo-spatial locations, and the one or more stored images including pre-identified agricultural objects;
determining a first real-world geo-spatial location of the agricultural treatment system;
receiving one or more captured images depicting real-world agricultural objects of a geographic scene;
associating the one or more captured images with the determined geo-spatial location of the agricultural treatment system;
identifying, from the group of one or more stored images, one or more images having a second real-word geo-spatial location that is proximate with the first real-world geo-spatial location, wherein the one or more captured images are obtained at a time later than when the group of images are stored on the storage device;
comparing at least a portion of the identified images with the one or more captured images;
determining a target object for treatment based on the comparing of the one or more captured images with the portions of the one or more identified images, the target object being determined where a portion of a captured image corresponds to a portion of an image from the identified one or more images; and
emitting a fluid projectile at the target object using the treatment unit.
2. The method of claim 1 , wherein the group of images comprise key frame images and sub-key frame images, the key frame images depicting agriculture objects of the geographical scene, and the sub-key frame images depicting a portion of a key frame image.
3. The method of claim 2 , wherein comparing at least a portion of the identified images comprises:
comparing pixels of the sub-key frame image to a portion of one of the captured images;
determining a confidence level of whether the sub-key frame image matches the portion of the captured image; and
identifying a match where the determined confidence level meets or exceeds a threshold value.
4. The method of claim 2 , wherein the key frame images were previously obtained by the one or more image sensors of the agricultural treatment system, and the captured images are obtained by the same one or more image sensors at a time subsequent to when the key frame images were taken.
5. The method of claim 1 , wherein comparing at least a portion of the identified images comprises:
identifying a landmark object in a captured image; and
determining a portion of the landmark object in the capture image matches a portion of an image from the group of images.
6. The method of claim 5 , wherein the target objects are real-world objects that are intended to be treated with the fluid projectile, and the landmark objects are real-world objects that aid in determining the location of a target object.
7. The method of claim 1 , wherein the one or more captured images are obtained by one or more image sensors and the real-world agricultural objects are located within a distance of 0.5 foot to 100 feet from the camera capturing the images.
8. A non-transitory computer storage medium comprising instructions that when executed by one or more process included of an agricultural treatment system, cause the agricultural treatment system to perform the operations comprising:
storing a group of images on a data storage device, the group of images depicting views of real-world agricultural objects, the one or more stored images having associated real-world geo-spatial locations, and the one or more stored images including pre-identified agricultural objects;
determining a first real-world geo-spatial location of the agricultural treatment system;
receiving one or more captured images depicting real-world agricultural objects of a geographic scene;
associating the one or more captured images with the determined geo-spatial location of the agricultural treatment system;
identifying, from the group of one or more stored images, one or more images having a second real-word geo-spatial location that is proximate with the first real-world geo-spatial location, wherein the one or more captured images are obtained at a time later than when the group of images are stored on the storage device;
comparing at least a portion of the identified images with the one or more captured images;
determining a target object for treatment based on the comparing of the one or more captured images with the portions of the one or more identified images, the target object being determined where a portion of a captured image corresponds with a portion of an image from the identified one or more images; and
emitting a fluid projectile at the target object using a treatment unit of the agricultural treatment system.
9. The non-transitory computer storage medium of claim 8 , wherein the group of images comprise key frame images and sub-key frame images, the key frame images depicting agriculture objects of the geographical scene, and the sub-key frame images depicting a portion of a key frame image.
10. The non-transitory computer storage medium of claim 9 , wherein comparing at least a portion of the identified images comprises:
comparing pixels of the sub-key frame image to a portion of one of the captured images;
determining a confidence level of whether the sub-key frame image matches the portion of the captured image; and
identifying a match where the determined confidence level meets or exceeds a predetermined confidence level threshold value.
11. The non-transitory computer storage medium of claim 9 , wherein the key frame images were previously obtained by the one or more image sensors of the agricultural treatment system, and the captured images are obtained by the same one or more image sensors at a time subsequent to when the key frame images were taken.
12. The non-transitory computer storage medium of claim 8 , wherein comparing at least a portion of the identified images comprises:
identifying a landmark object in a captured image; and
determining a portion of the landmark object in the capture image matches a portion of an image from the group of images.
13. The non-transitory computer storage medium of claim 12 , wherein the target objects are real-world objects that are intended to be treated with the fluid projectile, and the landmark objects are real-world objects that aid in determining the location of a target object.
14. The non-transitory computer storage medium of claim 8 , wherein the one or more captured images are obtained by one or more image sensors and the real-world agricultural objects are located within a distance of 0.5 foot to 100 feet from the camera capturing the images.
15. An agricultural treatment system, the system comprising:
one or more image sensors for obtaining imagery;
a treatment unit mounted to a gimble, the treatment unit configured to emit a fluid projectile;
one or more tanks, wherein the tanks provide a container for a fluid;
an electronically controlled pump fluidly connected to the one or more tanks and the treatment unit; and
one or more processors, memory, and a non-transitory computer readable storage medium comprising hardware, the one or more processors configured to control operation of the treatment unit, and control actuation of the electronically controlled pump and orientation of the treatment unit for emitting a fluid projectile via the treatment unit, wherein the one or more processors are configured to perform operations comprising:
storing a group of images on a data storage device, the group of images depicting views of real-world agricultural objects, the one or more stored images having associated real-world geo-spatial locations, and the one or more stored images including pre-identified agricultural objects;
determining a first real-world geo-spatial location of the agricultural treatment system;
receiving one or more captured images depicting real-world agricultural objects of a geographic scene;
associating the one or more captured images with the determined geo-spatial location of the agricultural treatment system;
identifying, from the group of one or more stored images, one or more images having a second real-word geo-spatial location that is proximate with the first real-world geo-spatial location, wherein the one or more captured images are obtained at a time later than when the group of images are stored on the storage device;
comparing at least a portion of the identified images with the one or more captured images;
determining a target object for treatment based on the comparing of the one or more captured images with the portions of the one or more identified images, the target object being determined where a portion of a captured image corresponds with a portion of an image from the identified one or more images; and
emitting a fluid projectile at the target object using the treatment unit.
16. The agricultural treatment system of claim 15 , wherein the group of images comprise key frame images and sub-key frame images, the key frame images depicting agriculture objects of the geographical scene, and the sub-key frame images depicting a portion of a key frame image.
17. The agricultural treatment system of claim 16 , wherein comparing at least a portion of the identified images comprises:
comparing pixels of the sub-key frame image to a portion of one of the captured images;
determining a confidence level of whether the sub-key frame image matches the portion of the captured image; and
identifying a match where the determined confidence level meets or exceeds a predetermined confidence level threshold value.
18. The agricultural treatment system of claim 16 , wherein the key frame images were previously obtained by the one or more image sensors of the agricultural treatment system, and the captured images are obtained by the same one or more image sensors at a time subsequent to when the key frame images were taken.
19. The agricultural treatment system of claim 15 , wherein comparing at least a portion of the identified images comprises:
identifying a landmark object in a captured image; and
determining a portion of the landmark object in the capture image matches a portion of an image from the group of images.
20. The agricultural treatment system of claim 15 , wherein the one or more captured images are obtained by one or more image sensors and the real-world agricultural objects are located within a distance of 0.5 foot to 100 feet from the camera capturing the images.
21. The method of claim 1 , wherein the group of images depict views of one or more unique agricultural objects in the real-world.
22. The method of claim 1 , further comprising:
determining whether the target object needs treatment; and
emitting the fluid projectile where the target object has been determined to need treatment.
23. The method of claim 1 , further comprising:
wherein the group of the one or more stored images includes associated information indicating a stage of growth of an agricultural object; and
dispensing a predetermined amount of the fluid based on the stage of growth of the target object.
24. The method of claim 1 , wherein the group of one or more stored images includes associated information indicating an object type of an agricultural object, and wherein emitting the fluid projectile is based on the object type of the target object.
25. The method of claim 1 , wherein the group of one or more stored images includes associated information indicating an object type of an agricultural object; and
wherein the emitting the fluid projectile is based on the treatment history of the target object.
26. The agricultural treatment system of claim 15 , wherein the group of images depict views of one or more unique agricultural objects in the real-world.
27. The agricultural treatment system of claim 15 , further comprising:
determining whether the target object needs treatment; and
emitting the fluid projectile where the target object has been determined to need treatment.
28. The agricultural treatment system of claim 15 , further comprising:
wherein the group of the one or more stored images includes associated information indicating a stage of growth of an agricultural object; and
dispensing a predetermined amount of the fluid based on the stage of growth of the target object.
29. The agricultural treatment system of claim 15 , wherein the group of one or more stored images includes associated information indicating an object type of an agricultural object, and wherein emitting the fluid projectile is based on the object type of the target object.
30. The agricultural treatment system of claim 15 , wherein the group of one or more stored images includes associated information indicating an object type of an agricultural object; and wherein the emitting the fluid projectile is based on the treatment history of the target object.