Systems and methods for object tracking and location prediction
Disclosed herein are methods, devices, modules, and systems which may be employed to accurately track, and subsequently target, objects of interest relative to a moving body, such as a vehicle. An object tracking method may be implemented by a detection system with a sensor coupled to the moving body. A targeting system may be used to target objects tracked by the detection system, such as for automated crop cultivation or maintenance. Devices disclosed herein may be configured to locate, identify, and autonomously target a weed with a beam, such as a laser beam, which may burn or irradiate the weed. The methods, devices, modules, and systems may be used for agricultural crop management or for at-home weed control.
1 . A method of tracking objects relative to a moving vehicle, the method comprising:
receiving, from an imaging sensor, a first image capturing a first region at a first time;
determining, in the first image, a first set of actual locations for a first set of objects relative to a moving vehicle;
receiving, from the imaging sensor, a second image capturing a second region at a second time, wherein the second region includes at least a portion of the first region;
determining, in the second image, a second set of actual locations for a second set of objects relative to the moving vehicle, wherein the second set of objects includes one or more of the same objects as the first set of objects;
identifying a tracked object included in the first set of objects;
applying a predicted shift to a first actual location of the tracked object in the first image at the first time to produce a shifted location of the tracked object at the second time;
designating a search area;
selecting a second actual location of the tracked object in the second image from the second set of objects in the second image at the second time from the second set of actual locations, wherein the second actual location of the tracked object in the second image at the second time is within the search area;
determining a vector displacement of the tracked object between the first time and the second time from the first actual location of the tracked object in the first image at the first time and the second actual location of the tracked object in the second image at the second time;
predicting a location of the tracked object based at least in part on the vector displacement, and
manipulating the tracked object at the predicted location.
2 . The method of claim 1 , further comprising designating the search area around the shifted location of the tracked object or the first actual location of the tracked object.
3 . The method of claim 1 , further comprising applying the predicted shift to each location of the first set of actual locations to produce a set of shifted locations comprising the shifted location of the tracked object.
4 . The method of claim 3 , further comprising applying a plurality of test shifts to the first set of shifted locations to obtain a set of test locations for the first set of objects at the second time, wherein each test shift of the plurality of test shifts is determined based on a distance between the shifted location and a second actual location of the second set of actual locations.
5 . The method of claim 4 , further comprising determining a plurality of offsets between each test location of the set of test locations and each second actual location of the second set of actual locations for the second set of objects at the second time, wherein each offset of the plurality of offsets corresponds to a test shift of the plurality of test shifts, and selecting a refined shift from the plurality of test shifts based on the plurality of offsets.
6 . The method of claim 1 , further comprising applying a second shift to the shifted location to produce a revised shifted location of the tracked object, wherein the second shift is based on a distance between the shifted location and the second actual location.
7 . The method of claim 6 , further comprising designating the search area around the revised shifted location of the tracked object.
8 . The method of claim 1 , further comprising determining a vector velocity of the moving vehicle, wherein the predicted shift is based on the vector velocity of the moving vehicle and a time difference between the first time and the second time.
9 . The method of claim 8 , wherein the vector velocity is determined using optical flow, a rotary encoder, a global positioning system, the vector displacement of the tracked object, or a combination thereof.
10 . The method of claim 8 , further comprising determining a trajectory of the tracked object over time based on the vector displacement of the tracked object between the first time and the second time and the vector velocity of the moving vehicle.
11 . The method of claim 8 , further comprising determining a predicted location of the tracked object at a third time based on the vector velocity of the moving vehicle and an elapsed time between the second time and the third time.
12 . The method of claim 1 , further comprising determining a predicted location of the tracked object at a third time based on the vector displacement of the tracked object between the first time and the second time and an elapsed time between the second time and the third time.
13 . The method of claim 1 , further comprising applying a plurality of test shifts to the first set of actual locations for the first set of objects to obtain a set of test locations for the first set of objects at the second time, wherein each test shift of the plurality of test shifts is determined based on a distance between a first actual location of a first object at the first time and a second actual location of a second object at the second time.
14 . The method of claim 13 , wherein the predicted shift is based on a test shift of the plurality of test shifts with the smallest corresponding offset, wherein the offset is between each of the test locations of the set of test locations for the first set of objects at the second time and each of the second set of actual locations for the second set of objects at the second time.
15 . The method of claim 1 , further comprising determining a second predicted location based on the predicted location and the first actual location of the tracked object in the first image at the first time, the second actual location of the tracked object in the second image at the second time, or both.
16 . The method of claim 1 , wherein the tracked object is positioned on a surface.
17 . The method of claim 16 , wherein the vehicle is moving perpendicular to the surface.
18 . The method of claim 16 , wherein the surface is a ground surface, a dirt surface, or an agricultural surface.
19 . The method of claim 1 , wherein manipulating the tracked object comprises targeting the tracked object at the predicted location.
20 . The method of claim 19 , wherein targeting the tracked object comprises correcting for parallax, distortions in the first image, distortions in the second image, targeting distortions, or a combination thereof.
21 . The method of claim 19 , wherein targeting the tracked object comprises aiming a targeting sensor, an implement, or both toward the predicted location.
22 . The method of claim 21 , wherein targeting the tracked object further comprises dynamically tracking the object with the targeting sensor, the implement, or both.
23 . The method of claim 22 , wherein dynamically tracking the object comprises moving the targeting sensor, the implement, or both to match the vector velocity of the moving vehicle such that the targeting sensor, the implement, or both remain aimed toward the predicted location while the moving vehicle moves.
24 . The method of claim 21 , wherein the implement comprises a laser, a sprayer, or a grabber.
25 . The method of claim 21 , wherein targeting the tracked object further comprises manipulating the tracked object with the implement, wherein manipulating the tracked object comprises irradiating the tracked object with electromagnetic radiation, moving the tracked object, spraying the tracked object, or combinations thereof.
26 . The method of claim 1 , wherein the tracked object is a plant.
27 . The method of claim 1 , wherein the tracked object is a weed or a crop.
28 . A method of tracking objects relative to a moving vehicle, the method comprising:
receiving, from an imaging sensor, a first image capturing a first region at a first time,
determining, in the first image, a first set of actual locations for a first set of objects relative to the moving vehicle;
receiving, from the imaging sensor, a second image capturing a second region at a second time, wherein the second region includes at least a portion of the first region,
determining, in the second image, a second set of actual locations for a second set of objects relative to the moving vehicle, wherein the second set of objects includes one or more of the same objects as the first set of objects;
applying a plurality of test shifts to the first set of actual locations for the first set of objects to obtain a set of test locations for the first set of objects at the second time, wherein each test shift of the plurality of test shifts is determined based on a distance between a first actual location of a first object at the first time and a second actual location of a second object at the second time;
determining a plurality of offsets between each of the test locations of the set of test locations for the first set of objects at the second time and each of the second set of actual locations for the second set of objects at the second time, wherein each offset of the plurality of offsets corresponds to a test shift of the plurality of test shifts;
selecting a refined shift from the plurality of test shifts based on the plurality of offsets;
identifying the tracked object included in both the first set of objects and the second set of objects;
predicting a loco cation of the tracked object; and
manipulating the tracked object at the predicted location.
29 . The method of claim 28 , further comprising determining a vector displacement of the tracked object between the first time and the second time from the first actual location of the tracked object in the first image at the first time and the second actual location of the tracked object in the second image at the second time.
30 . The method of claim 29 , further comprising determining a predicted location of the tracked object at a third time based on the vector displacement of the tracked object between the first time and the second time and an elapsed time between the second time and the third time.
31 . The method of claim 29 , further comprising applying the refined shift to the first set of actual locations for a first set of objects at the first time to establish a set of shifted locations for the first set of objects at the second time.
32 . The method of claim 29 , wherein the refined shift is a test shift from the plurality of test shifts with the smallest corresponding offset.