IP Library Granted Patent US 12,276,501
Granted Patent B2
US 12,276,501 · App. 18/351,301 · Granted Apr 15, 2025

Control point identification and reuse system

Inventors: Bernard J. Michini (San Francisco, CA); Brett Michael Bethke (Millbrae, CA); Hui Li (San Francisco, CA)
Assignee: Skydio, Inc.
G01C15/02G01C11/02B64U2101/30
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,276,501
App. No.
18/351,301
Granted
Apr 15, 2025
Kind
B2
Abstract

Landmarks are identified based on images of an area. A path is generated for a vehicle to traverse the area that includes the landmarks. Precise locations information indicative of locations of the vehicle and captured by a high accuracy position receiver of the vehicle while the vehicle traverses the path are received from the vehicle. At least some of the precise locations information are associated with the landmarks. Unmanned aerial vehicle (UAV) data are received from a UAV. The UAV data include aerial images of the area captured by the UAV and UAV location information corresponding to the aerial images. A three-dimensional model of the area is generated based on at least some of the precise locations information and the aerial images.

Claims (54)

1. A method, comprising:

identifying landmarks based on images of an area;

generating a path for a vehicle to traverse the area that includes the landmarks;

receiving, from the vehicle, precise locations information indicative of locations of the vehicle and captured by a high accuracy position receiver of the vehicle while the vehicle traverses the path;

associating at least some of the precise locations information with the landmarks;

subsequently receiving unmanned aerial vehicle (UAV) data from a UAV, wherein the UAV data comprises aerial images of the area captured by the UAV and UAV location information corresponding to the aerial images, wherein the UAV location information is determined, at least in part, based on the precise locations information associated with the landmarks; and

generating a three-dimensional model of the area based on at least some of the precise locations information and the aerial images.

2. The method of claim 1 , further comprising:

generating a flight plan for the UAV based on the landmarks.

3. The method of claim 2 , wherein the flight plan includes waypoints corresponding to the landmarks.

4. The method of claim 1 , further comprising:

receiving, from the vehicle, distances of the vehicle to a surface, wherein the distances are captured by a distance sensor of the vehicle and the distances correspond to the precise locations information.

5. The method of claim 4 , wherein generating the three-dimensional model of the area based on the at least some of the precise locations information and the aerial images comprises:

utilizing at least some of the distances to vertically project the UAV location information onto the surface; and

generating at least one of geo-rectified or ortho-rectified imagery based on the projecting.

6. The method of claim 1 , wherein the landmarks are identified as waypoints in the path.

7. The method of claim 2 , wherein the path configures the vehicle to be vertically positioned over at least one of the landmarks.

8. The method of claim 1 , wherein the path comprises at least one of a figure eight pattern, a back and forth pattern, or a random pattern.

9. A system, comprising:

a device, wherein the device is configured to:

store landmarks based on images of an area;

associate with the landmarks at least some of precise locations information received from a vehicle, wherein the precise locations information are indicative of locations of the vehicle while the vehicle traverses a path in the area that includes the landmarks;

subsequently receive unmanned aerial vehicle (UAV) data from a UAV, wherein the UAV data comprises aerial images of the area captured by the UAV and UAV location information corresponding to the aerial images, and wherein the UAV location information is determined, at least in part, based on the precise locations information associated with the landmarks; and

generate a three-dimensional model of the area based on at least some of the precise locations information and the aerial images;

the vehicle, wherein the vehicle is configured to:

navigate the path; and

transmit the precise locations information to the device, wherein the precise locations information are captured by a high accuracy position receiver of the vehicle while the vehicle traverses the path; and

the UAV, wherein the UAV is configured to:

capture, while navigating a flight plan, the UAV data; and

transmit the UAV data to the device.

10. The system of claim 9 , wherein the UAV is further configured to:

receive, from the device, the flight plan.

11. The system of claim 10 , wherein the flight plan includes waypoints corresponding to at least some of the landmarks.

12. The system of claim 9 , wherein the vehicle is further configured to:

transmit distances of the vehicle to a surface, wherein the distances are captured by a distance sensor of the vehicle and the distances correspond to the precise locations information.

13. The system of claim 12 , wherein to generate the three-dimensional model of the area based on the at least some of the precise locations information and the aerial images comprises:

utilize at least some of the distances received from the vehicle to vertically project the location of the UAV location information onto the surface; and

generate geo-rectified or ortho-rectified imagery based on the projecting.

14. The system of claim 9 , wherein the landmarks are identified as waypoints in the path.

15. The system of claim 10 , wherein the path configures the vehicle to be vertically positioned over at least one of the landmarks.

16. The system of claim 9 , wherein the path comprises one or more of a figure eight pattern, a back and forth pattern, or a random pattern.

17. Non-transitory computer-readable storage media, comprising executable instructions that, when executed by one or more processors, facilitate performance of operations comprising:

generating a path for a vehicle to traverse an area that includes landmarks, wherein the landmarks identified based on images of the area;

receiving, from the vehicle, precise locations information indicative of locations of the vehicle and captured by a high accuracy position receiver of the vehicle while the vehicle traverses the path;

associating at least some of the precise locations information with the landmarks;

subsequently receiving unmanned aerial vehicle (UAV) data from a UAV, wherein the UAV data comprises aerial images of the area captured by the UAV and UAV location information corresponding to the aerial images, wherein the UAV location information is determined, at least in part, based on the precise locations information associated with the landmarks; and

generating a three-dimensional model of the area based on at least some of the precise locations information and the aerial images.

18. The non-transitory computer-readable storage media of claim 17 , wherein the operations further comprise:

transmitting, to the UAV, a flight plan that includes waypoints corresponding to the landmarks.

19. The non-transitory computer-readable storage media of claim 17 , wherein the operations further comprise:

receiving, from the vehicle, distances of the vehicle to a surface, wherein the distances are captured by a distance sensor of the vehicle.

20. The non-transitory computer-readable storage media of claim 19 , wherein generating the three-dimensional model of the area based on the at least some of the precise locations information and the aerial images comprises:

utilize at least some of the distances to vertically project the location of the UAV onto the surface; and

generating geo-rectified or ortho-rectified imagery based on the projecting.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 13, 2023
From: MICHINI, BERNARD J.; BETHKE, BRETT MICHAEL; LI, HUI
To: UNMANNED INNOVATION INC.
Reel/Frame 064240/0658 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 13, 2023
From: UNMANNED INNOVATION, INC.
To: AIRWARE, LLC
Reel/Frame 064240/0715 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 13, 2023
From: AIRWARE, LLC
To: SKYDIO, INC.
Reel/Frame 064240/0764 →
Continuity (4)
Continuation 17478220 · Sep 17, 2021
Continuation 15396096 · Dec 30, 2016
Provisional Application 62273822 · Dec 31, 2015
Related Publication 20230358538A1 · Nov 9, 2023
References Cited (24)
US 6437708B1 · Brouwer · 2002 [cited by examiner]
US 8315794B1 · Strelow et al. · 2012 [cited by applicant]
US 9052571B1 · Lapstun · 2015 [cited by examiner]
US 9087451B1 · Jarrell · 2015 [cited by applicant]
US 9415869B1 · Chan et al. · 2016 [cited by applicant]
US 9529360B1 · Melamed et al. · 2016 [cited by applicant]
US 11150089B2 · Michini et al. · 2021 [cited by applicant]
US 20140316616A1 · Kugelmass · 2014 [cited by applicant]
US 20150032295A1 · Stark et al. · 2015 [cited by applicant]
US 20150148988A1 · Fleck · 2015 [cited by examiner]
US 20150269438A1 · Samarasekera et al. · 2015 [cited by applicant]
US 20150321758A1 · Sarna, II · 2015 [cited by applicant]
US 20160004795A1 · Novak · 2016 [cited by applicant]
US 20160023760A1 · Goodrich · 2016 [cited by applicant]
US 20170109577A1 · Wang · 2017 [cited by examiner]
US 20170234966A1 · Naguib et al. · 2017 [cited by applicant]
US 20180155023A1 · Choi et al. · 2018 [cited by applicant]
US 20180305012A1 · Ichihara · 2018 [cited by examiner]
CN 204854730U · 2015 [cited by applicant]
CN 107615358A · 2018 [cited by applicant]
R. Sharma, “Observability based control for cooperative localization,” 2014 International Conference on Unmanned Aircraft Systems (ICUAS), Orlando, FL, USA, 2014, pp. 134-139, doi: 10.1109/ICUAS.2014.6842248. (Year: 201… [cited by examiner]
L. Jayatilleke and N. Zhang, “Landmark-based localization for Unmanned Aerial Vehicles,” 2013 IEEE International Systems Conference (SysCon), Orlando, FL, USA, 2013, pp. 448-451, doi: 10.1109/SysCon.2013.6549921. (Year:… [cited by examiner]
S. Minaelan, “Vision-Based Target Detection and Localization via a Team of Cooperative UAV and UGVs”, 2015 (Year: 2015). [cited by applicant]
Douterloigne, Koen et al., “On the Accuracy of 3D Landscapes From UAV Image Data,” 2010 IEEE International Geoscience and Remote Sensing Symposium, pp. 589-592, Jul. 25-30, 2010. [cited by applicant]