IP Library Granted Patent US 12,494,138
Granted Patent B2
US 12,494,138 · App. 18/404,697 · Granted Dec 9, 2025

Map including data for routing aerial vehicles during GNSS failure

Inventors: Martin Kubie (San Francisco, CA); Dinuka Abeywardena (Mountain View, CA)
Assignee: Wing Aviation LLC
G08G5/74B64U10/25G05D1/0038G05D1/0044G05D1/0094G05D1/222G05D1/225G05D1/689G08G5/55G08G5/57H04W4/029B64U2101/30B64U2201/104
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,494,138
App. No.
18/404,697
Granted
Dec 9, 2025
Kind
B2
Abstract

An unmanned aerial vehicle (UAV) includes a propulsion system, a global navigation satellite system (GNSS) sensor, a camera and a controller. The controller includes logic that, in response to execution by the controller, causes the UAV to in response to detecting a loss of tracking by the GNSS sensor determine an estimated location of the UAV on a map based on a location image captured by the camera, determine a route to a destination using tracking parameters embedded in the map, wherein the map is divided into a plurality of sections and the tracking parameters indicate an ease of determining a location of the UAV using images captured by the camera with respect to each section, and control the propulsion system to cause the UAV to follow the route to the destination.

Claims (41)

1 . At least one non-transitory computer readable medium having computer-executable instructions stored thereon that, in response to execution by a computer, cause the computer to:

obtain at least one image of an area;

determine features in the at least one image and a feature uniqueness for each of the features, wherein the feature uniqueness indicates a relative ease by which a given one of the features is uniquely identifiable;

generate a map of the area;

divide the map into map sections; and

assign a tracking parameter to each of the map sections based on a combination of a quantity of the features associated with each of the map sections and the feature uniqueness of the features associated with each of the map sections.

2 . The at least one non-transitory computer readable medium of claim 1 , wherein the map sections form a grid.

3 . The at least one non-transitory computer readable medium of claim 1 , wherein the map is a three-dimensional map with a plurality of two-dimensional layers together forming the three-dimensional map, wherein each of the two-dimensional layers is divided into the map sections, and wherein each two-dimensional layer of the map corresponds to a UAV at a different height.

4 . The at least one non-transitory computer readable medium of claim 3 , wherein the map sections have different sizes between different two-dimensional layers of the map, and wherein the quantity of features and the feature uniqueness for each of the features in the at least one image is determined for each of the two-dimensional layers of the map.

5 . The at least one non-transitory computer readable medium of claim 1 , wherein the tracking parameter of each of the map sections is a numerical value.

6 . The at least one non-transitory computer readable medium of claim 1 , further comprising identifying temporary and permanent features in the at least one image, wherein the determining the quantity of features and the feature uniqueness for each of the features in the at least one image is performed on the permanent features and not the temporary features.

7 . The at least one non-transitory computer readable medium of claim 1 , wherein at least one of the at least one image is taken from a first height and another of the at least one image is taken at a second height.

8 . The at least one non-transitory computer readable medium of claim 1 , further comprising transmitting the map to an unmanned aerial vehicle (UAV) for autonomous navigation.

9 . The at least one non-transitory computer readable medium of claim 1 , wherein the features include roads, intersections, and buildings.

10 . At least one non-transitory computer readable medium having computer-executable instructions stored thereon that, in response to execution by a controller of an unmanned aerial vehicle (UAV), causes the UAV to perform operations comprising:

in response to detecting a loss of tracking by a global navigation satellite system (GNSS) sensor of the UAV:

determining an estimated location of the UAV on a map based on a location image captured by a camera of the UAV;

determining a route to a destination using tracking parameters embedded in the map, wherein the map is divided into a plurality of sections and the tracking parameters indicate an ease of determining a location of the UAV using images captured by the camera with respect to each of the sections, wherein determining the route includes planning the route to avoid passing over one of the sections of the map having a corresponding tracking parameter that does not meet a threshold value of ease for determining the location of the UAV using one or more of the images; and

navigating the UAV along the route to the destination.

11 . A computer implemented method comprising:

obtaining at least one image of an area;

determining a quantity of features and a feature uniqueness in the at least one image;

generating a map of the area;

dividing the map into map sections; and

assigning a tracking parameter to each of the map sections based on at least one of the quantity of features or the feature uniqueness in the at least one image,

wherein the map is a three-dimensional map with a plurality of two-dimensional layers together forming the three-dimensional map, wherein each of the two-dimensional layers is divided into the map sections, and wherein each two-dimensional layer of the map corresponds to a different height for flying an unmanned aerial vehicle (UAV).

12 . The computer implemented method of claim 11 , wherein the map sections form a grid.

13 . The computer implemented method of claim 11 , wherein each two-dimensional layer of the map has sections has a different size, and wherein the quantity of features and feature uniqueness in the at least one image is determined for each two-dimensional layer of the map.

14 . The computer implemented method of claim 11 , wherein the tracking parameter of each section of the map is a numerical value.

15 . The computer implemented method of claim 11 , further comprising identifying temporary and permanent features in the at least one image, wherein the determining the quantity of features and the feature uniqueness in the at least one image is performed on the permanent features.

16 . The computer implemented method of claim 11 , wherein at least one of the at least one image is taken from a first height and another of the at least one image is taken at a second height.

17 . The computer implemented method of claim 11 , further comprising transmitting the map to an unmanned aerial vehicle (UAV) for autonomous navigation.

18 . A method for navigating an unmanned aerial vehicle (UAV), comprising:

determining an estimated location of the UAV on a map based on a location image captured by a camera of the UAV;

determining a route to a destination using tracking parameters embedded in the map, wherein the map is divided into a plurality of sections and the tracking parameters indicate an ease of determining a location of the UAV using images captured by the camera with respect to each of the sections, wherein determining the route includes planning the route to avoid passing over one of the sections of the map having a corresponding tracking parameter that does not meet a threshold value of ease for determining the location of the UAV using one or more of the images; and

navigating the UAV along the route to the destination.

19 . The method of claim 18 , wherein the method comprises a backup method of navigation for the UAV when a primary method of navigation for the UAV fails.

20 . A method of navigation for an unmanned aerial vehicle (UAV), the method comprising:

determining an estimated location of the UAV on a map based on a location image captured by a camera of the UAV;

determining a route to a destination using tracking parameters embedded in the map, wherein the map is divided into a plurality of sections and the tracking parameters indicate an ease of determining a location of the UAV using images captured by the camera with respect to each of the sections, wherein determining the route includes planning the route with a preference for passing the route over first sections of the map deemed easier, as indicated by the tracking parameters, to determine the location of the UAV using the images while avoiding second sections of the map deemed more difficult, as indicated by the tracking parameters, to determine the location of the UAV using the images; and

navigating the UAV along the route to the destination.

Continuity (2)
Division 16689872 · Nov 20, 2019
Related Publication 20240177619A1 · May 30, 2024
References Cited (32)
US 9665100B2 · Shashua et al. · 2017 [cited by applicant]
US 11900820B2 · Kubie · 2024 [cited by examiner]
US 20060064202A1 · Gutmann · 2006 [cited by examiner]
US 20150347872A1 · Taylor et al. · 2015 [cited by applicant]
US 20160371985A1 · Kotecha · 2016 [cited by applicant]
US 20170328716A1 · Ma · 2017 [cited by applicant]
US 20180157255A1 · Halverson et al. · 2018 [cited by applicant]
US 20180292214A1 · Zhang · 2018 [cited by applicant]
US 20190080142A1 · Abeywardena et al. · 2019 [cited by applicant]
US 20190258883A1 · Britto Mattos Lima et al. · 2019 [cited by applicant]
CN 106875486A · 2017 [cited by applicant]
CN 108871351A · 2018 [cited by applicant]
CN 109642796A · 2019 [cited by applicant]
CN 109765930A · 2019 [cited by applicant]
CN 109839921A · 2019 [cited by applicant]
Australian Notice of Acceptance for Patent Application mailed Jul. 4, 2023, in corresponding Patent Application No. 2020388371, 3 pages. [cited by applicant]
European Communication under Rule 71(3) EPC, Intention to Grant mailed Oct. 31, 2023, in corresponding Patent Application No. 20775539.8, 35 pages. [cited by applicant]
Vetrella et al., “Differential GNSS and Vision-Based Tracking to Improve Navigation Performance in Cooperative Multi-UAV Systems”, Article, Dec. 17, 2016, 26 pages, Sensors 2016, vol. 16(12), 2164, www.mdpi.com/journal/… [cited by applicant]
Zahran, et al., “Enhanced Drone Navigation in GNSS Denied Environment Using VDM and Hall Effect Sensor”, Journal, Apr. 2, 2019, 18 pages, International Journal of Geo-Information 2019, vol. 8(169), www.mdpi.com/journal/… [cited by applicant]
Partial Search Report, issued Nov. 24, 2020, in corresponding International Patent Application No. PCT/US2020/049664, 10 pages. [cited by applicant]
International Search Report and Written Opinion mailed Feb. 5, 2021, in corresponding International Patent Application No. PCT/US2020/049664, 15 pages. [cited by applicant]
Australian Office Action mailed Mar. 10, 2023, in corresponding Australian Patent Application No. 2020388371, 3 pages. [cited by applicant]
Conte et al., “An Integrated UAV Navigation System Based on Aerial Image Matching”, 2007, IEEE, IEEEAC Paper #1276 , updated Oct. 23, 2007, 10 pages. [cited by applicant]
Lu et al., ‘A Survey on vision-based UAV navigation’, 2018, Geo-spatial Information Science, 21:1, 21-32. [cited by applicant]
USPTO Office Action, Restriction Requirement mailed Nov. 9, 2021, in corresponding U.S. Appl. No. 16/689,872, 5 pages. [cited by applicant]
USPTO Office Action mailed Mar. 18, 2022, in corresponding U.S. Appl. No. 16/689,872, 11 pages. [cited by applicant]
USPTO Office Action mailed Jul. 21, 2022, in corresponding U.S. Appl. No. 16/689,872, 12 pages. [cited by applicant]
USPTO Office Action mailed Nov. 25, 2022, in corresponding U.S. Appl. No. 16/689,872, 14 pages. [cited by applicant]
USPTO Office Action mailed Apr. 12, 2023, in corresponding U.S. Appl. No. 16/689,872, 14 pages. [cited by applicant]
USPTO Notice of Allowance mailed Oct. 18, 2023, in corresponding U.S. Appl. No. 16/689,872, 16 pages. [cited by applicant]
Chinese Notice of Allowance mailed Nov. 6, 2024, in corresponding CN Patent Application No. 202080079526.7, 8 pages. [cited by applicant]
Singapore Written Opinion dated May 28, 2025, in corresponding Singapore Patent Application No. 11202204589P, 8 pages. [cited by applicant]