Unmanned aerial image capture platform
Methods and systems are disclosed for an unmanned aerial vehicle (UAV) configured to autonomously navigate a physical environment while capturing images of the physical environment. In some embodiments, the motion of the UAV and a subject in the physical environment may be estimated based in part on images of the physical environment captured by the UAV. In response to estimating the motions, image capture by the UAV may be dynamically adjusted to satisfy a specified criterion related to a quality of the image capture.
1. An unmanned aerial vehicle (UAV) configured for autonomous flight through a physical environment, the UAV including:
multiple image sensors configured to capture images of a physical environment surrounding the UAV;
a gimbaled image sensor configured to capture images of a subject in the physical environment; and
a control system configured to continually:
process the images of the physical environment surrounding the UAV to detect images with overlapping fields of view;
process the images with overlapping field of view to identify dense correspondences between the images;
estimate a distance to each pixel represented in each of the identified images with overlapping field of view using the dense correspondences; and
generate and continually update a three-dimensional (3D) model of the physical environment based on the distance estimates.
2. The UAV of claim 1 , wherein the control system is further configured to:
cause the UAV to autonomously navigate through the physical environment based, at least in part, on the 3D model of the physical environment.
3. The UAV of claim 1 , wherein the images of the physical environment surrounding the UAV are captured during sequential time steps.
4. The UAV of claim 1 , wherein the images of the physical environment surrounding the UAV are stereoscopic images captured during a same timestep.
5. The UAV of claim 4 , wherein the multiple image sensors are configured with pre-defined spatial offsets from each other.
6. The UAV of claim 1 , wherein the control system is further configured to continually estimate relative positions and/or orientations of the UAV and/or the subject within the physical environment while simultaneously continually updating the 3D model.
7. The UAV of claim 1 , wherein the multiple image sensors are arranged to provide a full 360 degree view around the UAV.
8. The UAV of claim 7 , wherein the multiple image sensors are arranged such that at least two image sensors are provided with overlapping fields of view.
9. The UAV of claim 1 , wherein the multiple image sensors are arranged such that at least one image sensor comprises a fisheye lens.
10. The UAV of claim 1 , further comprising:
one or more additional proximity sensors configured to estimate distance to pixels represented in the identified images with overlapping field of view.
11. The UAV of claim 10 , wherein the one or more additional proximity sensors comprise at least one of an electromagnetic sensor, an acoustic sensor, or an optics based sensor to estimate distance with more accuracy than the image sensors.
12. A method comprising:
capturing, by multiple navigation image sensors of an unmanned aerial vehicle (UAV), images of a physical environment surrounding the UAV;
capturing, by a gimbaled image sensor of the UAV, images of a subject in the physical environment;
processing the images of the physical environment surrounding the UAV to detect images with overlapping fields of view;
processing the images with overlapping field of view to identify dense correspondences between the images;
estimating a distance to each pixel represented in each of the identified images with overlapping field of view using the dense correspondences; and
generating and continually updating a three-dimensional (3D) model of the physical environment based on the distance estimates.
13. The method of claim 12 , further comprising:
causing the UAV to autonomously navigate through the physical environment based, at least in part, on the 3D model of the physical environment.
14. The method of claim of claim 12 , wherein the images of the physical environment surrounding the UAV are captured during sequential time steps.
15. The method of claim of claim 12 , wherein the images of the physical environment surrounding the UAV are stereoscopic images captured during a same timestep and the multiple image sensors are configured with pre-defined spatial offsets from each other.
16. The method of claim of claim 12 , wherein the control system is further configured to continually estimate relative positions and/or orientations of the UAV and/or the subject within the physical environment while simultaneously continually updating the 3D model.
17. The method of claim of claim 12 , wherein the multiple image sensors are arranged to provide a full 360 degree view around the UAV, and wherein the multiple image sensors are arranged such that at least two image sensors are provided with overlapping fields of view.
18. The method of claim of claim 12 , wherein at least one of the multiple image sensors comprises a fisheye lens.
19. An apparatus, comprising:
one or more memory units storing instructions that, when executed by one or more processors of an aerial vehicle, cause the one or more processors to:
process images of the physical environment surrounding the aerial vehicle to detect images with overlapping fields of view;
process the images with overlapping field of view to identify dense correspondences between the images;
estimate a distance to each pixel represented in each of the identified images with overlapping field of view using the dense correspondences; and
generate and continually update a three-dimensional environment using the distance estimates.
20. The apparatus of claim 19 , wherein the instructions, when executed by the one or more processors of the aerial vehicle, further cause the one or more processors to:
generate control commands to cause the aerial vehicle to autonomously navigate through the physical environment based, at least in part, on the 3D model of the physical environment.