Unmanned aerial image capture platform
Methods and systems are disclosed for an unmanned aerial vehicle (UAV) configured to autonomously navigate a physical environment while capturing images of the physical environment. In some embodiments, the motion of the UAV and a subject in the physical environment may be estimated based in part on images of the physical environment captured by the UAV. In response to estimating the motions, image capture by the UAV may be dynamically adjusted to satisfy a specified criterion related to a quality of the image capture.
1. An unmanned aerial vehicle (UAV) configured for autonomous flight through a physical environment, the UAV including:
multiple image sensors configured to capture images of a view of a physical environment surrounding the UAV;
a gimbaled image sensor configured to capture images of a subject in the physical environment; and
a control system configured to:
estimate a motion of the UAV and the subject in the physical environment relative to a three-dimensional (3D) map of the physical environment using visual inertial odometry based on the captured images of the physical environment surrounding the UAV; and
generate control commands to dynamically adjust image capture of the subject by the gimbaled image capture device to satisfy a specified criterion.
2. The UAV of claim 1 , wherein the specified criterion is related to a quality of the image capture.
3. The UAV of claim 1 , wherein the control system is further configured to:
generate, while in flight through the physical environment, the 3D map of the physical environment based on the captured images of the physical environment surrounding the UAV.
4. The UAV of claim 3 , wherein the control system is further configured to:
continually update the 3D map of the physical environment based on images of the physical environment surrounding the UAV continually captured by the multiple image sensors.
5. The UAV of claim 3 , wherein generating and updating the 3D map of the physical environment comprises calculating position and/or pose of features in the physical environment in addition to position and/or pose of the UAV.
6. The UAV of claim 1 , wherein the control commands to dynamically adjust image capture include control commands to dynamically adjust a flight path of the UAV.
7. The UAV of claim 6 , wherein the control commands to dynamically adjust the flight path of the UAV include control commands to cause the UAV to avoid a collision with another physical object in the physical environment.
8. The UAV of claim 6 , further including:
a flight controller configured to cause the UAV to autonomously maneuver based on the flight path.
9. The UAV of claim 1 , wherein to satisfy the specified criterion, the control commands to dynamically adjust image capture are configured such that, at a given time, image capture of the subject by the UAV is not obstructed by another physical object in the physical environment.
10. The UAV of claim 1 , wherein to satisfy the specified criterion, the control commands to dynamically adjust image capture are configured such that, at a given time, the UAV is positioned substantially between a light source and the subject so as to avoid backlighting in the captured images of the subject.
11. The UAV of claim 1 , wherein the control system is further configured to:
identify the subject in the physical environment by comparing captured images of the subject against stored two-dimensional (2D) or three-dimensional (3D) appearance models representing a neural network;
determine, based on the identification of the subject and the estimated motion of the subject, that the image capture of the subject is intended for a particular type of scene;
wherein, to satisfy the specified criterion, the control commands to dynamically adjust image capture are configured such that the UAV follows a pre-scripted flight path associated with the particular type of scene.
12. The UAV of claim 1 , wherein the control system is further configured to:
predict a future motion of the subject based any of the estimated motion of the subject a previously estimated motion of the subject, the captured images, or a 3D map of the physical environment;
wherein the control commands to dynamically adjust image capture are further based on the predicted future motion of the subject.
13. The UAV of claim 1 , wherein the specified criterion is based on quantified metrics associated with subjective image capture quality standards.
14. A method comprising:
generating, while in autonomous flight within a physical environment, a three-dimensional (3D) map of the physical environment based on images captured by image capture devices coupled to an unmanned aerial vehicle (UAV);
estimating a motion of the UAV and a subject in the physical environment relative to the 3D map of the physical environment using visual inertial odometry with the images captured by the image capture devices; and
generating a control command to dynamically adjust image capture of the subject by a gimbaled image capture device coupled to the UAV to satisfy a specified criterion related to the image capture.
15. The method of claim 14 , wherein the specified criterion is related to a quality of the image capture.
16. The method of claim 14 , further comprising:
generating, while in flight through the physical environment, the 3D map of the physical environment based on the captured images of the physical environment surrounding the UAV.
17. The method of claim 16 , further comprising:
continually updating the 3D map of the physical environment based on images of the physical environment surrounding the UAV continually captured by the multiple image sensors.
18. The method of claim 16 , wherein generating and updating the 3D map of the physical environment comprises calculating position and/or pose of features in the physical environment in addition to position and/or pose of the UAV.
19. An apparatus, comprising:
one or more memory units storing instructions that, when executed by one or more processors of an unmanned aerial vehicle (UAV), cause the one or more processors to:
generate, while in autonomous flight within a physical environment, a three-dimensional (3D) map of the physical environment based on images captured by image capture devices coupled to the UAV;
estimate a motion of the UAV and a subject in the physical environment relative to the 3D map of the physical environment using visual inertial odometry with the images captured by the image capture devices; and
generate a control command to dynamically adjust image capture of the subject by a gimbaled image capture device coupled to the UAV to satisfy a specified criterion related to the image capture.
20. The apparatus of claim 19 , wherein the specified criterion is related to a quality of the image capture.