Detecting optical discrepancies in captured images
Embodiments are described for detecting optical discrepancies associated with image capture analyzing pixels in multiple images corresponding to common points of reference in a physical environment. In an embodiment, photometric error values are averaged over time to compute the mean error at each pixel. Once the estimate of the mean error has a sufficient number of updates above a specified value, the estimate is thresholded to provide a mask of any optical discrepancies occurring in the stereo pair of images. Applications include detecting optical discrepancies in images captured for use by a visual navigation system in guiding an autonomous vehicle (e.g., an unmanned aerial vehicle).
1 . A method for autonomously navigating an aerial vehicle through a physical environment, the method comprising:
obtaining a first image at a first position of the aerial vehicle and a second image at a second position of the aerial vehicle different than the first position;
for a pixel of the first image, computing a photometric error with respect to candidate pixels of the second image along an epipolar correspondence between the first and second positions;
detecting, based at least in part on the photometric error, an optical discrepancy associated with a contiguous region in a field of view of an image capture device of the aerial vehicle; and
generating control commands configured to autonomously maneuver the aerial vehicle through the physical environment by excluding the region from a visual-navigation control computation used to generate the control commands such that the region is ignored when estimating motion and/or scene structure.
2 . The method of claim 1 , wherein computing the photometric error comprises searching along the epipolar line for a minimum residual between a first-image pixel value and second-image candidate pixel values.
3 . The method of claim 1 , further comprising:
generating a threshold map of photometric errors across the field of view; and
detecting the optical discrepancy by thresholding the map.
4 . The method of claim 1 , wherein excluding the region comprises applying a binary mask to image data before visual-odometry pose estimation.
5 . The method of claim 1 , wherein excluding the region comprises down-weighting pixels of the region in a photometric cost function using a robust loss.
6 . The method of claim 1 , wherein the optical discrepancy corresponds to a dynamic object identified by temporal persistence of above-threshold photometric error over successive image pairs.
7 . The method of claim 1 , wherein the epipolar correspondence is determined from an estimated relative pose between the first and second positions.
8 . The method of claim 1 , further comprising:
causing display of the threshold map and the detected region on a remote device.
9 . The method of claim 1 , wherein the detected optical discrepancy comprises a specular highlight or saturation artifact, and excluding the region prevents bias in pose estimation due to illumination changes.
10 . A vehicle comprising:
an image capture device configured to capture images of a physical environment;
a propulsion system configured to maneuver the vehicle through the physical environment; and
a visual navigation system configured to:
obtain a first image at a first position of the vehicle and a second image at a second position of the vehicle different than the first position;
for a pixel of the first image, compute a photometric error with respect to candidate pixels of the second image along an epipolar correspondence between the first and second positions;
detect, based at least in part on the photometric error, an optical discrepancy associated with a contiguous region in a field of view of an image capture device of the vehicle; and
generate control commands configured to autonomously maneuver the vehicle through the physical environment by excluding the region from control computation used to generate the control commands such that the region is ignored when estimating motion and/or scene structure.
11 . The vehicle of claim 10 , wherein computing the photometric error comprises searching along the epipolar line for a minimum residual between a first-image pixel value and second-image candidate pixel values.
12 . The vehicle of claim 10 , wherein the visual navigation system is further configured to:
generate a threshold map of photometric errors across the field of view and detecting the optical discrepancy by thresholding the map.
13 . The vehicle of claim 10 , wherein excluding the region comprises applying a binary mask to image data before visual-odometry pose estimation.
14 . The vehicle of claim 10 , wherein excluding the region comprises down-weighting pixels of the region in a photometric cost function using a robust loss.
15 . The vehicle of claim 10 , wherein the optical discrepancy corresponds to a dynamic object identified by temporal persistence of above-threshold photometric error over successive image pairs.
16 . The vehicle of claim 10 , wherein the epipolar correspondence is determined from an estimated relative pose between the first and second positions.
17 . The vehicle of claim 10 , wherein the visual navigation system is further configured to:
cause display of the threshold map and the detected region on a remote device.
18 . The vehicle of claim 10 , wherein the detected optical discrepancy comprises a specular highlight or saturation artifact, and excluding the region prevents bias in pose estimation due to illumination changes.
19 . An apparatus comprising:
one or more computer-readable media; and
program instructions stored on the one or more computer-readable storage media that, when executed by one or more processors of a vehicle, direct the one or more processors to at least:
obtain a first image at a first position of the vehicle and a second image at a second position of the vehicle that is different from the first position;
compute, for a pixel of the first image, a photometric error with respect to candidate pixels of the second image along an epipolar correspondence between the first and second positions;
detect, based at least in part on the photometric error, an optical discrepancy associated with a contiguous region in a field of view of an image capture device of the vehicle; and
generate control commands configured to autonomously maneuver the vehicle through the physical environment by excluding the region from control computation used to generate the control commands such that the region is ignored when estimating the motion and/or scene structure.
20 . The apparatus of claim 19 , wherein the program instructions, when executed by the one or more processors of the vehicle, further direct the one or more processors to:
generate a threshold map of photometric errors across the field of view; and
detect the optical discrepancy by thresholding the map.