Systems and methods to determine object position using images captured from mobile image collection vehicle
An object identification method is disclosed. The method includes obtaining images of a target geographical area and telemetry information of an image-collection vehicle at a time of capture, analyzing each image to identify objects, and determining a position of the objects. The method further includes determining an image capture height, determining a position of the image using the capture height and the telemetry information, performing a transform on the image based on the capture height and the telemetry information, identifying the objects in the transformed image, determining first pixel locations of the objects within the transformed image, performing a reverse transform on the first pixel locations to determine second pixel locations in the image, and determining positions of the objects within the area based on the second pixel locations within the captured image and the determined image position.
1. A method, comprising:
obtaining a first image of a geographical area;
performing a homography transform on the first image to generate a second image having uniform-pixel-distances based on a capture height and avionic telemetry information associated with an aerial vehicle that captured the first image;
performing image recognition on the second image to identify an object in the second image using an artificial neural network trained on a dataset of trained object parameters, wherein the second image is divided into a plurality of tiles, wherein each of the plurality of tiles is input into an artificial neural network trained on trained object parameters, and wherein a bounding box is generated for the object based on results from the artificial neural network;
determining a first pixel location of the object within the second image based on the bounding box;
performing a reverse homography transform on the first pixel location to determine a second pixel location in the first image for the object;
determining a position of the object within the geographical area based on the second pixel location within the first image and an image position associated with the aerial vehicle capturing the image; and
storing the determined position of the object.
2. The method of claim 1 , wherein obtaining the first image of the geographical comprises:
capturing, via a camera on the aerial vehicle, the first image; and
recording the avionic telemetry information of the aerial vehicle at a time of capture of the first image.
3. The method of claim 1 , wherein obtaining the first image of the geographical area comprises:
employing the aerial vehicle to traverse over the geographical area;
capturing, via a camera on the aerial vehicle, the first image as the aerial vehicle traverses over the geographical area; and
capturing the avionic telemetry information of the aerial vehicle when first image is captured.
4. The method of claim 1 , wherein obtaining the first image of the geographical area comprises:
selecting a target pixel-to-physical distance resolution for the first image;
determining a maximum travel height for the aerial vehicle based on the target pixel-to-physical distance resolution and one or more sensing characteristics of a camera on the aerial vehicle;
employing the aerial vehicle to traverse over the geographical area at the maximum travel height relative to a low point on the geographical area; and
capturing the first image as the aerial vehicle traverses over the geographical area.
5. The method of claim 4 , wherein the target pixel-to-physical distance resolution is an image portion of 15 pixels by 15 pixels that corresponds to a four inch square of the geographical area.
6. The method of claim 1 , wherein the avionic telemetry information includes at least two of: global positioning system location, pitch of the image-collection vehicle, roll of the image-collection vehicle, yaw of the image-collection vehicle, heading of the image-collection vehicle, and altitude of the image-collection vehicle.
7. The method of claim 1 , further comprising:
selecting the trained object parameters from a plurality of datasets of trained object parameters based on at least one of: time of year, type of crop planted in the geographical area, status of the crop, expected type of object, or expected type of non-cultivated vegetation.
8. The method of claim 1 , further comprising:
performing the image recognition on the second image to identify a second object in the second image based on the trained object parameters;
determining a third pixel location of the second object within the second image; and
ignoring the second object as a duplicate object in response to the third pixel location being within a threshold distance from the first pixel location of the object.
9. The method of claim 1 , further comprising:
employing an object-collection system to pick up the object based on the stored location of the object.
10. A system, comprising:
an aerial vehicle including:
a camera;
a first processor; and
a first memory that stores first computer instructions that, when executed by the first processor, cause the first processor to:
control movement of the aerial vehicle over a geographical area;
capture, via the camera, a first image of the geographical area; and
record a capture height of the aerial vehicle and avionic telemetry information of the aerial vehicle when the first image is captured;
a server including:
a second processor; and
a second memory that stores second computer instructions that, when executed by the second processor, cause the second processor to:
obtain the first image and the avionic telemetry information;
perform a homography transform on the first image to generate a second image having uniform-pixel-distances per ground unit based on the capture height and the avionic telemetry information;
perform image recognition on the second image to identify an object in the second image using an artificial neural network trained on a dataset of trained object parameters, wherein the second image is divided into a plurality of tiles, wherein each of the plurality of tiles is input into an artificial neural network trained on trained object parameters, and wherein a bounding box is generated for the object based on results from the artificial neural network;
determine a first pixel location of the object within the second image based on the bounding box;
perform a reverse homography transform on the first pixel location to determine a second pixel location in the first image for the object;
determine a position of the object within the geographical area based on the second pixel location within the first image and the capture height and the avionic telemetry information; and
store the determined position of the object.
11. The system of claim 10 , wherein execution of the first computer instructions by the first processor to control movement of the aerial vehicle causes the first processor to:
receive a travel plan indicating a travel path for the aerial vehicle to move over the geographical area; and
control movement of the aerial vehicle along the travel path over the geographical area based on the travel plan.
12. The system of claim 10 , wherein execution of the first computer instructions by the first processor causes the first processor to:
select a target pixel-to-physical distance resolution for the first image;
determine a maximum travel height for the aerial vehicle based on the target pixel-to-physical distance resolution and one or more sensing characteristics of the camera on the aerial vehicle;
employ the aerial vehicle to traverse over the geographical area at the maximum travel height relative to a low point on the geographical area; and
capture the first image as the aerial vehicle traverses over the geographical area.
13. The system of claim 12 , wherein the target pixel-to-physical distance resolution is an image portion of 15 pixels by 15 pixels that corresponds to a four inch square of the geographical area.
14. The system of claim 10 , wherein execution of the second computer instructions by the second processor causes the second processor to:
select a dataset of trained object parameters from a plurality of datasets of trained object parameters based on at least one of: time of year, type of crop planted in the geographical area, status of the crop, expected type of object, or expected type of non-cultivated vegetation.
15. The system of claim 10 , wherein execution of the second computer instructions by the second processor causes the second processor to:
perform the image recognition on the second image to identify a second object in the second image;
determine a third pixel location of the second object within the second image; and
ignore the second object as a duplicate object in response to the third pixel location being within a threshold distance from the first pixel location of the object.
16. The system of claim 10 , wherein execution of the second computer instructions by the second processor causes the second processor to:
determine an orientation of a third image of the geographic area relative to the first image, wherein the first image includes the object and the third image includes a second object;
determine if the second object is a duplicate of the object based on a pixel distance between the object and the second object and the determined orientation; and
in response to determining that the second object is a duplicate, ignoring storage of a position of the second object.
17. The system of claim 10 , wherein the second processor on the object-detection server is the first processor on the image-collection vehicle.
18. An object-detection computing device, comprising:
a memory that stores second computer instructions; and
a processor that is configured to execute the computer instructions to:
obtain a first image of a geographic area;
perform a homography transform on the first image to generate a second image having a uniform-pixel-distance per ground unit based on an image height and avionic telemetry information associated with an aerial vehicle that captured the first image;
perform image recognition on the second image to identify an object in the second image using an artificial neural network trained on a dataset of trained object parameters, wherein the second image is divided into a plurality of tiles, wherein each of the plurality of tiles is input into an artificial neural network trained on trained object parameters, and wherein a bounding box is generated for the object based on results from the artificial neural network;
determine a first pixel location of the object within the second image based on the bounding box or segmented search zones that are searched by the trained artificial neural network;
perform a reverse homography transform on the first pixel location to determine a second pixel location in the first image;
determine a position of the object within the geographical area based on the second pixel location within the first image and the image height and the avionic telemetry information; and
store the determined position of the object.