Systems and methods for point to point object matching and targeting
Disclosed herein are methods, devices, modules, and systems which may be employed for accurate targeting of objects. A method of point-to-point targeting may be implemented by systems with two or more sensors to locate an object and coordinate a handoff between the two or more sensors. These methods, devices, modules, and systems may be used for automated crop cultivation or maintenance. Devices disclosed herein may be configured to locate, identify, and autonomously target a weed with a beam, such as a laser beam, which may burn or irradiate the weed. The methods, devices, modules, and systems may be used for agricultural crop management or for at-home weed control.
1. A method comprising:
receiving from a first imaging sensor, a first image depicting a first region of a field;
identifying a plant of interest within the first region of the field depicted in the first image;
determining a predicted location of the plant of interest based on the first image;
directing a second imaging sensor to obtain a second image depicting a second region of the field, wherein the second region includes the predicted location of the plant of interest;
identifying the plant of interest within the second region of the field depicted in the second image; and
determining a target location of the plant of interest based on the second image.
2. The method of claim 1 , further comprising aiming the second imaging sensor toward the predicted location.
3. The method of claim 1 , further comprising aiming the second imaging sensor toward the target location.
4. The method of claim 3 , wherein aiming the second imaging sensor toward the target location comprises determining an offset between a first position of the second imaging sensor and a second position of the second imaging sensor.
5. The method of claim 1 , wherein the second imaging sensor is aimed toward the target location when the second imaging sensor is positioned at a second position of the second imaging sensor.
6. The method of claim 1 , wherein the target location is closer to the plant of interest than the predicted location.
7. The method of claim 1 , further comprising directing an implement toward the target location.
8. The method of claim 7 , wherein a direction of the implement is correlated with a direction of the second imaging sensor.
9. The method of claim 7 , wherein directing the implement toward the target location comprises determining an offset between a first position of the implement and a second position of the implement, and wherein the implement is directed toward the target location when the implement is positioned at the second position of the implement.
10. The method of claim 7 , further comprising manipulating the plant of interest the implement.
11. The method of claim 10 , wherein manipulating the plant of interest comprises one or more of irradiating the plant of interest with electromagnetic radiation, moving the plant of interest, spraying the plant of interest, or combinations thereof.
12. The method of claim 1 , wherein the first imaging sensor differs from the second imaging sensor in one or more parameters selected from the group consisting of sensor type, sensor resolution, magnification, field of view, color balance, color sensitivity, and positioning; wherein positioning comprises angle, distance, or both relative to the plant of interest.
13. The method of claim 1 , wherein the first imaging sensor and the second imaging sensor are coupled to a vehicle.
14. The method of claim 13 , wherein the predicted location accounts for motion of the vehicle relative to the plant of interest between a time the first image is collected and a time the second image is collected.
15. The method of claim 13 , wherein the target location accounts for motion of the vehicle relative to the plant of interest between a time the second image is collected and a time a manipulation of the plant of interest is performed.
16. The method of claim 1 , wherein the plant of interest is located on, above, or below a surface of the field.
17. The method of claim 16 , wherein the trained machine learning model accounts for variability in depth of the surface.
18. The method of claim 1 , wherein the target location is no more than 50 mm, no more than 25 mm, no more than 10 mm, no more than 5 mm, no more than 3 mm, no more than 2 mm, or no more than 1 mm from the plant of interest.
19. The method of claim 1 , wherein the first imaging sensor, the second imaging sensor, or both comprises one or more of a camera, a light detection and ranging (LIDAR) sensor, a photodetector, an active-pixel sensor, a semiconductor detector, an ultrasound sensor, a RADAR detector, a sonar sensor, or a photodiode array.
20. The method of claim 1 , comprising using a trained machine learning model to:
identify the plant of interest in the second region of the field depicted in the second image;
identify the plant of interest in the first region of the field depicted in the first image; and
match the plant of interest in the second image to the plant of interest in the first image.
21. The method of claim 1 , wherein the plant of interest is a weed or a crop.
22. The method of claim 1 , wherein the predicted location, target location, or both comprises a location in the first image, a position of the second image sensor, a position of an implement, a position of the first image sensor, a position of the plant of interest on a surface, a position of a vehicle, or any combination thereof.
23. A method of targeting an object comprising:
coordinating a handoff between a prediction sensor and a targeting sensor, comprising:
collecting, by the prediction sensor, a first image depicting a first region;
identifying, by a processor, plant of interest to be targeted within the first image;
determining, by the processor, a predicted location of the plant of interest based on the first image;
aiming the targeting sensor toward the predicted location of the plant of interest;
collecting, by the targeting sensor, a second image depicting a second region, wherein the second regions includes the predicted location of the plant of interest;
identifying, by the processor, the plant of interest within the second region depicted in the second image; and
determining, by the processor, a target location of the plant of interest based on the second image;
aiming the targeting sensor toward the target location;
directing an implement toward the target location; and
manipulating the plant of interest with the implement.
24. The method of claim 23 , wherein determining the target location of the plant of interest comprises using a trained machine learning model to:
identify the plant of interest in the second region depicted in the second image;
identify the plant of interest in the first region depicted in the first image; and
match the plant of interest in the second image to the plant of interest in the first image.