Unmanned aerial image capture platform
Methods and systems are disclosed for an unmanned aerial vehicle (UAV) configured to autonomously navigate a physical environment while capturing images of the physical environment. In some embodiments, the motion of the UAV and a subject in the physical environment may be estimated based in part on images of the physical environment captured by the UAV. In response to estimating the motions, image capture by the UAV may be dynamically adjusted to satisfy a specified criterion related to a quality of the image capture.
1. A method comprising:
generating and continually updating, while in autonomous flight within a physical environment, a three-dimensional (3D) map of the physical environment based on images captured by an array of image capture devices coupled to an unmanned aerial vehicle (UAV);
estimating a motion of both the UAV and a subject in the physical environment relative to the 3D map of the physical environment using visual inertial odometry with the images captured by the array of image capture devices; and
in response to estimating the motions of the UAV and the subject, generating control commands to dynamically adjust image capture of the subject by a gimbaled image capture device coupled to the UAV to satisfy a specified criterion related to a quality of the image capture.
2. The method of claim 1 , wherein the control commands to dynamically adjust image capture include control commands to dynamically adjust the motion of the UAV.
3. The method of claim 2 , wherein the control commands to dynamically adjust the motion of the UAV include control commands configured to cause the UAV to avoid a collision with another physical object in the physical environment.
4. The method of claim 2 , wherein the control commands to dynamically adjust the motion of the UAV include control commands configured to cause the UAV to fly along a continually updated flight path through the physical environment.
5. The method of claim 2 , wherein the control commands to dynamically adjust the motion of the UAV include control commands to configured to cause the UAV to fly a pre-scripted flight path.
6. The method of claim 2 , wherein the control commands to dynamically adjust the motion of the UAV include control commands configured to cause the UAV to fly a path of maximum energy efficiency.
7. The method of claim 1 , wherein to satisfy the specified criterion, the control commands to dynamically adjust image capture are configured such that, at a given time, image capture of the subject by the UAV is not obstructed by another physical object in the physical environment.
8. The method of claim 1 , wherein to satisfy the specified criterion, the control commands to dynamically adjust image capture are configured such that, at a given time, the UAV is positioned substantially between a light source and the subject so as to avoid backlighting in the captured images of the subject.
9. The method of claim 8 , wherein the light source is the Sun.
10. The method of claim 1 , wherein estimating the motion of the UAV and the subject is further based in part on the generated 3D map.
11. The method of claim 1 , further comprising:
identifying the subject in the physical environment by comparing captured images of the subject against stored two-dimensional (2D) and/or three-dimensional (3D) appearance models representing a neural network;
determining based on the identification of the subject and the estimated motion of the subject that the image capture of the subject is intended for a particular type of scene;
wherein, to satisfy the specified criterion, the control commands to dynamically adjust image capture are configured such that the UAV follows a pre-scripted flight path associated with the particular type of scene.
12. The method of claim 1 , further comprising:
predicting a future motion of the subject based any of the estimated motion of the subject a previously estimated motion of the subject, the captured images, or a 3D map of the physical environment;
wherein the control commands to dynamically adjust image capture are further based on the predicted future motion of the subject.
13. The method of claim 1 , wherein the specified criterion is based on quantified metrics associated with subjective image capture quality standards.
14. The method of claim 13 , wherein the subjective image capture quality standards are based on models generated by applying machine learning to historical data.
15. The method of claim 1 , wherein the control commands to dynamically adjust image capture include control commands to adjust any of focus, aperture, shutter speed, light sensitivity (ISO), frame rate, color balance, image framing, or image stabilization.
16. The method of claim 1 , wherein the gimbaled image capture device utilizes a motorized gimbal.
17. The method of claim 16 , wherein the motorized gimbal is a hybrid mechanical digital gimbal.
18. The method of claim 1 , wherein the images captured by the image capture device include video.
19. A system comprising:
a processing unit; and
a memory unit, the memory unit having instructions stored thereon, which when executed by the processing unit cause the system to:
generate and continually update, while in autonomous flight within a physical environment, a three-dimensional (3D) map of the physical environment based on images captured by an array of image capture devices coupled to an unmanned aerial vehicle (UAV);
estimate a motion of both the UAV and a subject in the physical environment relative to the 3D map of the physical environment using visual inertial odometry with the images captured by the array of image capture devices; and
in response to estimating the motions of the UAV and the subject, generate control commands to dynamically adjust image capture of the subject by a gimbaled image capture device coupled with the UAV to satisfy a specified criterion related to a quality of the image capture.
20. An unmanned aerial vehicle (UAV) comprising:
an array of a plurality of image capture devices configured to capture images of a physical environment surrounding the UAV;
a gimbaled image capture device mounted to a hybrid mechanical digital gimbal configured to capture images of a subject in the physical environment;
a processing unit communicatively coupled to the array of the plurality of image capture devices and the gimbaled image capture device; and
a memory unit having instructions stored thereon, which when executed by the processing unit, cause the processing unit to:
generating and continually updating, while in autonomous flight within the physical environment, a three-dimensional (3D) map of the physical environment based on the captured images of the physical environment by the array of image capture devices;
estimate a motion of both the UAV and the subject in the physical environment relative to the 3D map using visual inertial odometry based on the captured images of the physical environment;
in response to estimating the motions of the UAV and the subject, generate control commands to dynamically adjust image capture of the subject by the gimbaled image capture device to satisfy a specified criterion related to a quality of the image capture.