IP Library Granted Patent US 11,725,940
Granted Patent B2
US 11,725,940 · App. 17/478,220 · Granted Aug 15, 2023

Unmanned aerial vehicle control point selection system

Inventors: Bernard J. Michini (San Francisco, CA); Brett Michael Bethke (Millbrae, CA); Hui Li (San Francisco, CA)
Assignee: Skydio, Inc.
G01C15/02G01C11/02B64C39/024B64U2101/30
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,725,940
App. No.
17/478,220
Granted
Aug 15, 2023
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for an unmanned aerial vehicle control point selection system. A first unmanned aerial vehicle can navigate about a geographic area while a second unmanned aerial vehicle can capture images of the first unmanned aerial vehicle. Location information of the first unmanned aerial vehicle can be recorded, and a system can identify the first unmanned aerial vehicle in the captured images. Utilizing the recorded location information, the system can identify landmarks proximate to the first unmanned aerial vehicle identified in the images, and determine location information of the landmarks. The landmarks can be assigned as ground control points for subsequent flight plans.

Claims (49)

1. A system, comprising:

a processor; and

a memory storing instructions that when executed by the processor, cause the processor to:

receive location information identifying one or more locations associated with a first unmanned vehicle being navigated within a first area during a first time period;

receive one or more images of the first area including the first unmanned vehicle and one or more landmarks, wherein the one or more images are captured by a second unmanned vehicle when the second unmanned vehicle is flying at a higher altitude than the first unmanned vehicle; and

correlate a first captured image of the one or more images that includes the first unmanned vehicle and a first landmark of the one or more landmarks with first location information of the one or more locations based on a timestamp associated with the first captured image and a timestamp associated with the first location information.

2. The system of claim 1 , wherein the instructions executed by the processor, cause the processor to:

store the one or more locations associated with the first unmanned vehicle in association with timestamps indicating when a respective location was determined; and

store the one or more images captured by the second unmanned vehicle in association with timestamps indicating when a respective image was captured.

3. The system of claim 1 , wherein the first unmanned vehicle is a first unmanned aerial vehicle with a first satellite navigation receiver and the second unmanned vehicle is a second unmanned aerial vehicle with a second satellite navigation receiver, wherein the second satellite navigation receiver has lower precision and lower accuracy than the first satellite navigation receiver.

4. The system of claim 1 , wherein the instructions executed by the processor, cause the processor to:

select the first landmark as a control point; and

store an identification of the first landmark in association with the first location information.

5. The system of claim 4 , wherein to select the first landmark as the control point comprises to:

generate a score for the first landmark based on at least one of landmark dimensionality, landmark shape, landmark contrast, landmark movement, landmark absolute size, or landmark image size.

6. The system of claim 4 , wherein to select the first landmark as the control point comprises to:

determine if the first landmark corresponds to a preferred landmark type.

7. The system of claim 1 , wherein the second unmanned vehicle is on a mission unrelated to an identification of control points.

8. The system of claim 1 , wherein the first unmanned vehicle is the same as the second unmanned vehicle navigated at separate times with different payload configurations.

9. The system of claim 1 , wherein the one or more locations are recorded by the first unmanned vehicle with greater than a threshold accuracy.

10. The system of claim 9 , wherein the threshold accuracy is 30 centimeters.

11. The system of claim 1 , wherein the one or more locations are recorded by the first unmanned vehicle using a Carrier-Phase Enhanced GNSS receiver.

12. The system of claim 1 , wherein timestamps corresponding to the one or more locations recorded by the first unmanned vehicle are obtained using flight logs of the first unmanned vehicle.

13. The system of claim 1 , wherein timestamps corresponding to the one or more images captured by the second unmanned vehicle are obtained using flight logs of the second unmanned vehicle.

14. A system, comprising:

one or more computer processors; and

one or more computer storage media storing instructions that when executed by the one or more computer processors cause the one or more computer processors to:

obtain images and corresponding image time data captured by a first unmanned aerial vehicle (UAV) positioned above a second UAV during implementation of a flight plan, each image including the second UAV and one or more landmarks representing real-world geographic areas;

obtain location information and corresponding location time data recorded by the second UAV while proximate to the one or more landmarks;

assign, for each image and based on the corresponding image time data and the corresponding location time data, the second UAV included in the image as a ground control point;

obtain resulting images based on the images and corresponding assigned ground control points;

determine location information associated with the one or more landmarks in the resulting images; and

assign, for each resulting image and based on location information associated with the one or more landmarks, the one or more landmarks included in the resulting image as a ground control point.

15. The system of claim 14 , wherein the assigned ground control points are used during implementation of subsequent flight plans.

16. The system of claim 14 , wherein to obtain resulting images comprises to:

provide the images and the corresponding assigned ground control points to an outside system; and

receiving the resulting images from the outside system.

17. The system of claim 16 , wherein the outside system is a photogrammetry system, and wherein the resulting images comprise at least one of a geo-rectified image or an ortho-rectified image.

18. The system of claim 14 , wherein to assign the second UAV included in the image as a ground control point comprises to:

determine, based on the corresponding image time data and the corresponding location time data, location information of the second UAV that is included in the image; and

assign, for the image, the second UAV as a ground control point associated with the determined location information.

19. The system of claim 18 , wherein to determine the location information of the second UAV that is included in the image comprises to:

determine estimated location information based on recorded location information associated with the corresponding location time data that is within a threshold amount of time of the corresponding image time data.

20. A method comprising:

receiving one or more locations associated with a first unmanned vehicle being navigated within a first area during a first time period;

receiving one or more images of the first area captured by a second unmanned vehicle during at least a portion of the first time period, wherein the one or more images are captured when the second unmanned vehicle is flying above the first unmanned vehicle and which include the first unmanned vehicle and one or more landmarks;

correlating a first captured image of the one or more images that includes the first unmanned vehicle and a first landmark of the one or more landmarks, with first location information of the one or more locations based on a timestamp associated with the first captured image and a timestamp associated with the first location information;

selecting the first landmark as a control point; and

storing an identification of the first landmark in association with the first location information.

Assignments (7)
RELEASE OF SECURITY INTEREST Recorded Jul 21, 2025
From: SILICON VALLEY BANK, A DIVISION OF FIRST-CITIZENS BANK & TRUST COMPANY
To: SKYDIO, INC.
Reel/Frame 072107/0066 →
RELEASE OF SECURITY INTEREST Recorded Jul 21, 2025
From: HERCULES CAPITAL, INC.
To: SKYDIO, INC.
Reel/Frame 072128/0698 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 10, 2022
From: AIRWARE, LLC
To: SKYDIO, INC.
Reel/Frame 058592/0222 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 10, 2022
From: MICHINI, BERNARD J.; BETHKE, BRETT MICHAEL; LI, HUI
To: UNMANNED INNOVATION INC.
Reel/Frame 058591/0828 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 10, 2022
From: UNMANNED INNOVATION INC.
To: AIRWARE, LLC
Reel/Frame 058592/0114 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Nov 9, 2021
From: SKYDIO, INC.
To: HERCULES CAPITAL, INC., AS COLLATERAL AND ADMINISTRATIVE AGENT
Reel/Frame 058081/0677 →
SECURITY INTEREST Recorded Nov 8, 2021
From: SKYDIO, INC.
To: SILICON VALLEY BANK
Reel/Frame 058053/0768 →
Continuity (3)
Continuation 15396096 · Dec 30, 2016
Provisional Application 62273822 · Dec 31, 2015
Related Publication 20220074744A1 · Mar 10, 2022
Cited By (2)
US 12,504,454 US 12,624,944