Backup navigation system for unmanned aerial vehicles
Described is a method that involves operating an unmanned aerial vehicle (UAV) to begin a flight, where the UAV relies on a navigation system to navigate to a destination. During the flight, the method involves operating a camera to capture images of the UAV's environment, and analyzing the images to detect features in the environment. The method also involves establishing a correlation between features detected in different images, and using location information from the navigation system to localize a feature detected in different images. Further, the method involves generating a flight log that includes the localized feature. Also, the method involves detecting a failure involving the navigation system, and responsively operating the camera to capture a post-failure image. The method also involves identifying one or more features in the post-failure image, and determining a location of the UAV based on a relationship between an identified feature and a localized feature.
1. A computer-implemented method comprising:
during a flight along a first flight path to a destination by an unmanned aerial vehicle (UAV), operating an image capture device coupled to the UAV to capture a first set of images of a field-of-view (FOV) of an environment of the UAV;
analyzing the first set of images to identify one or more first visual features within the environment;
generating a local flight log for the first flight path, wherein the local flight log associates the identified first visual features with location information from a primary navigation system of the UAV; and
detecting a failure event involving the primary navigation system of the UAV and responsively:
capturing at least one post-failure image using the image capture device;
identifying, in the at least one post-failure image, one or more second visual features within the environment;
matching two or more of the first visual features to two or more of the second visual features;
based on respective coordinates indicated by the flight log for the two or more first visual features, determining coordinates of the two or more second visual features; and
using the coordinates of the two or more second visual features as a basis for determining the current location of the UAV.
2. The method of claim 1 , wherein the image capture device is part of a backup navigation system of the UAV, and wherein the backup navigation system replaces navigational functionality of the primary navigation system upon detecting the failure event.
3. The method of claim 1 , wherein the primary navigation system is reliant upon a global positioning system (GPS).
4. The method of claim 1 , further comprising:
responsive to detecting the failure event, determining to fly the UAV to a safe zone; and
based on the current location of the UAV, determining at least a portion of a second flight path to the safe zone.
5. The method of claim 4 , wherein the first flight path and the second flight path are the same flight path.
6. The method of claim 4 , wherein the UAV is deployed from a home base to the destination, and wherein the safe zone is the home base of the UAV.
7. The method of claim 4 , wherein the second flight path has a substantially similar altitude to the first flight path such that the image capture device during the second flight path has a substantially similar FOV of the environment to the FOV of the environment that the image capture device has during the first flight path.
8. The method of claim 1 , wherein operating the image capture device to capture the first set of images along the flight path comprises:
using position data from the primary navigation system to associate each image with a respective geolocation.
9. The method of claim 1 , further comprises localizing the one or more first visual features.
10. The method of claim 9 , wherein localizing the one or more first visual features comprises:
comparing a given visual feature identified in a first image from the first image set to one or more visual features identified in other images from the first set;
detecting a match between the given visual feature and a corresponding visual feature of at least one other image from the first set; and
determining a location of the given visual feature.
11. The method of claim 10 , wherein determining the location of the first visual feature comprises:
using a first geolocation of the first image and a second geolocation of the second image to determine the location of the first visual feature.
12. The method of claim 1 , further comprising:
determining a time of the flight and environmental conditions along the first flight path;
selecting a map associated with at least one of the time of the flight and the environmental conditions, wherein the map is representative of at least an area along the first flight path; and
providing the map to the UAV for local storage on memory coupled to the UAV, before the UAV begins the flight.
13. The method of claim 1 , further comprising, in response to detecting the failure event, ceasing flight to the destination and initiating flight to a home location.
14. The method of claim 13 , further comprising navigating to the home location by, repeatedly updating the current location of the UAV, where updating the current location comprises:
capturing at least one additional post-failure image using the image capture device;
identifying, in the at least one additional post-failure image, one or more third visual features within the environment; and
detecting a match between at least one of the third visual features and at least one of the first visual features, and responsively determining an updated current location of the UAV based at least in part on (i) location information for the at least one first visual feature indicated by the local flight log, and (ii) an orientation of the UAV corresponding to the capture of the at least one additional post-failure image.
15. The UAV of claim 13 , wherein the primary navigation system comprises a global positioning system (GPS) device configured to receive a satellite signal, and wherein the failure event is at least one of: (i) a malfunction of the GPS device, and (ii) a loss of the satellite signal.
16. An unmanned aerial vehicle (UAV) system comprising:
a primary navigation system providing for navigation of the UAV;
an image capture device configured to capture images of an environment of the UAV; and
a controller, wherein the controller is configured to:
use the primary navigation system to begin a flight of the UAV along a first flight path to a destination;
during the flight along the first flight path, operate the image capture device to capture a first set of images of a field-of-view (FOV) of an environment of the UAV;
analyze the first set of images to identify one or more first visual features within the environment;
generate a local flight log for the first flight path, wherein the local flight log associates the identified first visual features with location information generated by the primary navigation system; and
detect a failure event involving the primary navigation system of the UAV and responsively:
(a) capture at least one post-failure image using the image capture device;
(b) identify, in the at least one post-failure image, one or more second visual features within the environment;
(c) match two or more of the first visual features to two or more of the second visual features;
(d) based on respective coordinates indicated by the flight log for the two or more first visual features, determining coordinates of the two or more second visual features; and
(e) using the coordinates of the two or more second visual features as a basis for determining the current location of the UAV.
17. The UAV of claim 16 , wherein the controller is further configured to:
responsive to detecting the failure event, initiate flight of the UAV to a safe zone; and
based on the current location of the UAV, determine at least a portion of a second flight path to the safe zone.
18. A non-transitory computer-readable medium, having instructions stored thereon that when executed by one or more processors cause a computing device to perform functions comprising:
during a flight along a first flight path to a destination by an unmanned aerial vehicle (UAV), operating an image capture device coupled to the UAV to capture a first set of images of a field-of-view (FOV) of an environment of the UAV;
analyzing the first set of images to identify one or more first visual features within the environment;
generating a local flight log for the first flight path, wherein the local flight log associates the identified first visual features with location information from a primary navigation system of the UAV; and
detecting a failure event involving the primary navigation system of the UAV and responsively:
capturing at least one post-failure image using the image capture device;
identifying, in the at least one post-failure image, one or more second visual features within the environment;
matching two or more of the first visual features to two or more of the second visual features;
based on respective coordinates indicated by the flight log for the two or more first visual features, determining coordinates of the two or more second visual features; and
using the coordinates of the two or more second visual features as a basis for determining the current location of the UAV.