Unmanned aerial vehicle sensor activation and correlation system
An unmanned aerial vehicle (UAV) logs first UAV information at a first frequency. The UAV triggers a camera associated with the UAV to capture an image. In response to triggering the camera to capture the image, the UAV logs second UAV information at a second frequency that is higher than the first frequency. A device that is separate from the UAV identifies a location of the UAV corresponding to the image based on a capture timestamp of the image received from the camera, the first UAV information, and the second UAV information. The device generates a geo-rectified imagery based on the image and the location of the UAV.
1 . A method, comprising:
logging, by an unmanned aerial vehicle (UAV), first UAV information at a first frequency;
triggering, by the UAV, a camera associated with the UAV to capture an image;
in response to triggering the camera to capture the image, logging, by the UAV, second UAV information at a second frequency that is higher than the first frequency;
identifying, by a device that is separate from the UAV, a location of the UAV corresponding to the image based on a capture timestamp of the image received from the camera, the first UAV information, and the second UAV information; and
generating, by the device, geo-rectified imagery based on the image and the location of the UAV.
2 . The method of claim 1 , wherein the first UAV information includes at least one of location information of the UAV or inertial navigation system (INS) data.
3 . The method of claim 2 , further comprising:
determining, by the device, an optimal time offset for the capture timestamp, wherein the optimal time offset is determined from amongst a time series obtained based on the first UAV information, and the second UAV information, the optimal time offset indicating, for the image, a difference in value between respective metadata timestamps in the time series and the capture timestamp; and
determining, at the device and based on the optimal time offset, the location and INS data of the UAV at the time the image was captured, wherein the location is an estimated location.
4 . The method of claim 1 , wherein identifying the location of the UAV corresponding to the image based on the capture timestamp, the first UAV information, and the second UAV information comprises:
obtaining the image and the capture timestamp from the UAV;
accessing logs describing Global Positioning System (GPS) coordinates of the UAV and associated GPS timestamps;
correlating the GPS timestamps with the capture timestamp, wherein the correlation includes determining that the capture timestamp is between two GPS timestamps; and
determining a likely GPS coordinate corresponding to the capture timestamp based on the correlation, thereby determining the location of the UAV at a time the image was captured.
5 . The method of claim 1 , further comprising:
determining, at the UAV, to re-capture a captured image based on a comparison of a speed of the UAV and a shutter speed of the camera; and
triggering the camera, at the UAV, to re-capture the captured image.
6 . The method of claim 1 , wherein the image is a first image, further comprising:
triggering, at the UAV, the camera to capture a second image;
moving the UAV into a holding position while determining whether the second image is to be re-captured; and
triggering the camera to re-capture the second image responsive to a determination that the second image is to be re-captured.
7 . The method of claim 1 , further comprising:
determining, at the UAV, whether to re-capture the image based on a comparison that includes capabilities of the camera and a height associated with the image.
8 . An unmanned aerial vehicle (UAV) system comprising a non-transitory computer storage medium storing instructions that are operable, when executed by a processor, to cause the processor to:
receive first UAV information logged at a first frequency and second UAV information logged at a second frequency that is higher than the first frequency, wherein the first UAV information and the second UAV information are separated by an image capture event;
identify a location of the UAV corresponding to an image based on a capture timestamp of the image, the first UAV information, and the second UAV information; and
generate geo-rectified imagery based on the image and the location of the UAV.
9 . The UAV system of claim 8 , wherein the first UAV information includes at least one of first location information of the UAV and first inertial navigation system (INS) data.
10 . The UAV system of claim 9 , wherein the instructions are further operable, when executed by the processor, to cause the processor to:
determine an optimal time offset for the capture timestamp, wherein the optimal time offset is determined from amongst a time series obtained based on the first UAV information, and the second UAV information, the optimal time offset indicating, for the image, a difference in value between respective metadata timestamps in the time series and the capture timestamp; and
determine, based on the optimal time offset, the location and INS data of the UAV at the time the image was captured, wherein the location is an estimated location.
11 . The UAV system of claim 8 , wherein the instructions to identify the location of the UAV corresponding to the image based on the capture timestamp of the image received from a camera, the first UAV information, and the second UAV information, comprise instructions that, when executed by the processor, cause the processor to:
obtain the image and the capture timestamp from the UAV;
access logs describing Global Positioning System (GPS) coordinates of the UAV and associated GPS timestamps;
correlate the GPS timestamps with the capture timestamp by determining that the capture timestamp is between two GPS timestamps; and
determining a likely GPS coordinate corresponding to the capture timestamp based on the correlation, thereby determining the location of the UAV at a time the image was captured.
12 . The UAV system of claim 8 ,
wherein the non-transitory computer storage medium is a first non-transitory computer storage medium, the instructions are first instructions, and the processor is a first processor, and
wherein the UAV system further comprises:
a second non-transitory computer storage medium storing second instructions that are operable, when executed by a second processor, to cause the second processor to:
log, at the first frequency, the first UAV information;
trigger a camera to capture the image; and
subsequent to triggering the camera to capture the image, log the second UAV information at the second frequency.
13 . The UAV system of claim 12 , wherein the second instructions are further operable, when executed by the second processor, to cause the second processor to:
determine to re-capture a captured image based on a comparison of a speed of the UAV and a shutter speed of the camera; and
trigger the camera to re-capture the captured image.
14 . The UAV system of claim 12 , wherein the image is a first image, and wherein the second instructions are further operable, when executed by the second processor, to cause the second processor to:
trigger the camera to capture a second image;
move the UAV into a holding position while determining whether the second image is to be re-captured; and
trigger the camera to re-capture the second image responsive to a determination that the second image is to be re-captured.
15 . An unmanned aerial vehicle (UAV), comprising:
a processor, the processor configured to:
log, at a first frequency, first UAV information;
trigger a camera to capture an image; and
in response to triggering the camera to capture the image, log second UAV information at a second frequency that is higher than the first frequency,
wherein a capture timestamp of the image, the first UAV information, and the second UAV information are used to identify a location of the UAV corresponding to the image; and
wherein geo-rectified imagery is generated based on the image and the location of the UAV.
16 . The UAV of claim 15 , wherein the first UAV information includes at least one of first location information of the UAV and first inertial navigation system (INS) data.
17 . The UAV of claim 15 , wherein the processor is further configured to:
determine to re-capture the image based on a comparison of a speed of the UAV and a shutter speed of the camera; and
trigger the camera to re-capture the image.
18 . The UAV of claim 15 , wherein the image is a first image, and wherein the processor is further configured to:
trigger the camera to capture a second image;
move the UAV into a holding position while determining whether the second image is to be re-captured; and
trigger the camera to re-capture the second image responsive to a determination that the second image is to be re-captured.
19 . The UAV of claim 15 , wherein the processor is further configured to:
determine whether to re-capture the image based on a comparison that includes capabilities of the camera and a height associated with the image.
20 . The method of claim 1 , wherein identifying the location comprises selecting UAV records immediately before and after an image's capture time and determining the location based on the selected UAV records.
21 . The method of claim 20 , further comprising:
computing an attitude of the UAV at the capture timestamp by integrating inertial measurements recorded in a UAV log between the selected UAV records and associating the attitude with the image.
22 . The method of claim 1 , further comprising:
embedding, in image metadata, information including the image's capture timestamp, the identified location, and a quality indicator associated with the image.
23 . The method of claim 3 , wherein the optimal time offset is selected by minimizing an error metric between metadata timestamps and logged timestamps.
24 . The method of claim 3 , wherein the optimal time offset is determined based on a relationship between metadata timestamps and logged timestamps that compensates for differences between the timestamps.
25 . The method of claim 3 , further comprising:
computing respective optimal time offsets for multiple camera payloads, and applying, for each image, the offset corresponding to the payload that captured the image.
26 . The UAV system of claim 10 , wherein the processor is configured to write the optimal time offset and per-image adjusted timestamps to a data structure for consumption by geo-rectification software.
27 . The UAV system of claim 10 , wherein the instructions are further operable, when executed by the processor, to cause the processor to output a quality metric with the estimated location of the UAV.
28 . The UAV of claim 15 , wherein the processor is further configured to log location data and inertial data at the second frequency in response to triggering the camera to capture the image.
29 . The unmanned aerial vehicle of claim 15 , wherein the processor determines a camera pose for each image by applying a fixed boresight transform between an inertial sensor frame and a camera frame based on a relationship between an inertial-sensor frame and camera frame and associates the pose with the image.