Detecting optical discrepancies in captured images
View Patent ↗Embodiments are described for detecting optical discrepancies associated with image capture analyzing pixels in multiple images corresponding to common points of reference in a physical environment. In an embodiment, photometric error values are averaged over time to compute the mean error at each pixel. Once the estimate of the mean error has a sufficient number of updates above a specified value, the estimate is thresholded to provide a mask of any optical discrepancies occurring in the stereo pair of images. Applications include detecting optical discrepancies in images captured for use by a visual navigation system in guiding an autonomous vehicle (e.g., an unmanned aerial vehicle).
1. An unmanned aerial vehicle UAV) configured for autonomous flight through a physical environment, the UAV comprising:
a first image capture device;
a second image capture device, the second image capture device different than the first image capture device;
a propulsion system;
a visual navigation system configured to:
process a first plurality of images received from the first image capture device; and
generate control commands based at least in part on the first plurality of images received from the first image capture device, the control commands configured to maneuver the UAV using the propulsion system;
a subject tracking system configured to automatically adjust an orientation of the second image capture device relative to the UAV such that a subject in the physical environment remains in a field of view of the second image capture device while the UAV is in flight; and
an optical discrepancy detection system configured to:
receive, from the first image capture device, a first image of the physical environment from a first position;
receive, from the second image capture device, a second image of the physical environment from a second position;
process the first image and the second image to compare photometric characteristics of pixels in the first image and the second image corresponding to a common point of reference in the physical environment; and
detect an optical discrepancy associated with the capture of the first image and/or the second image based on the processing.
2. The UAV of claim 1 , wherein the first image capture device includes an array of stereoscopic cameras arranged along a perimeter of the UAV so as to provide a stereoscopic view at a plurality of angles around the UAV.
3. The UAV of claim 1 , wherein the subject tracking system includes a hybrid mechanical-digital gimbal system, the hybrid mechanical digital gimbal system including:
a mechanical gimbal configured to rotate the second image capture device about a first axis relative to the UAV; and
an image processing system configured to rotate one or more of a second plurality of images captured by the second image capture device about a second axis relative to the UAV.
4. The UAV of claim 1 , further comprising:
an image processing system configured to automatically adjust the first image and/or second image to correct for the detected optical discrepancy.
5. The UAV of claim 1 , further comprising:
a notification system configured to:
generate a notification in response to detecting the optical discrepancy; and
transmit the notification, via a wireless communication link, to a remote computing device.
6. The UAV of claim 1 , wherein the optical discrepancy detection system is further configured to:
identify a region of the first image and/or second image corresponding to the detected optical discrepancy; and
apply a mask to the region of the first image and/or second image corresponding to the detected optical discrepancy.
7. The UAV of claim 6 , wherein applying the mask to the region of the first image and/or second image causes the visual navigation system to ignore the region of the first image and/or second image when generating control commands to maneuver the UAV.