IP Library Granted Patent US 12,148,316
Granted Patent B2
US 12,148,316 · App. 17/313,754 · Granted Nov 19, 2024

Unmanned aerial vehicle visual point cloud navigation

Inventors: Joseph Moster (San Francisco, CA); Donna Okazaki (San Francisco, CA); Bernard J. Michini (San Francisco, CA)
Assignee: Skydio, Inc.
G08G5/0069B64U10/13G01C11/02G05D1/0094G08G5/0034G08G5/0039B64U30/20B64U2101/26B64U2101/30B64U2201/10B64U2201/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,148,316
App. No.
17/313,754
Granted
Nov 19, 2024
Kind
B2
Abstract

Methods, systems and apparatus, including computer programs encoded on computer storage media for unmanned aerial vehicle flight operations near physical structures or objects. In particular, a point cloud of the physical structure is generated using aerial images of the structure. The point cloud is then referenced to determine a flight path for the UAV to follow around the physical structure, determine whether a planned flight path to desired locations around the structure is possible, determine the fastest route to return home and land from a given position around the physical structure, determine possibility of inflight collision to surface represented in point cloud, or determine an orientation of a fixed or gimballed camera given a position of the UAV relative the point cloud.

Claims (52)

1. A method, comprising:

obtaining a flight plan including waypoints for navigating an aerial vehicle about a physical structure;

determining, using a point cloud representing the physical structure, a likelihood of the aerial vehicle colliding with the physical structure based on a likelihood of a first buffer associated with the aerial vehicle overlapping a second buffer associated with the point cloud,

wherein the first buffer is determined based on one or more sources of error associated with the aerial vehicle and the second buffer is determined based on one or more sources of error associated with the physical structure or the point cloud;

modifying one or more of the waypoints based on the likelihood of the aerial vehicle colliding with the physical structure exceeding a threshold to establish an updated flight plan; and

autonomously navigating the aerial vehicle about the physical structure according to the updated flight plan.

2. The method of claim 1 , wherein the first buffer represents a volumetric perimeter determined for the aerial vehicle and the second buffer represents a volumetric perimeter determined for the physical structure.

3. The method of claim 2 , wherein determining whether the likelihood of the first buffer associated with the aerial vehicle overlapping the second buffer associated with the point cloud comprises:

comparing three-dimensional coordinates associated with the volumetric perimeter determined for the aerial vehicle against three-dimensional coordinates associated with the volumetric perimeter determined for the physical structure.

4. The method of claim 1 , the method comprising:

changing a size of one or both of the first buffer or the second buffer based on information detected using the aerial vehicle.

5. The method of claim 1 , the method comprising:

generating the point cloud based on one or more images captured by the aerial vehicle and one or more geo-spatial readings of the aerial vehicle.

6. The method of claim 1 , wherein modifying the one or more of the waypoints based on the likelihood of the aerial vehicle colliding with the physical structure exceeding a threshold to establish the updated flight plan comprises:

changing a location of a waypoint of the one or more of the waypoints based on the determined likelihood of the aerial vehicle colliding with the physical structure indicating that a collision is likely at the waypoint.

7. The method of claim 1 , the method comprising:

transmitting the updated flight plan to the aerial vehicle to configure the aerial vehicle to navigate about the physical structure according to the updated flight plan.

8. The method of claim 1 , wherein the waypoints are selected by an operator of the aerial vehicle.

9. An aerial vehicle, comprising:

one or more sensors configured to produce output associated with the aerial vehicle and a physical structure; and

a flight planning system configured to:

determine, using a point cloud representing the physical structure, a likelihood of the aerial vehicle colliding with the physical structure based on a flight plan, wherein the flight plan includes waypoints for navigating the aerial vehicle about the physical structure, and the likelihood of the aerial vehicle colliding with the physical structure is based on a likelihood of a first buffer overlapping a second buffer, wherein the first buffer is determined based on at least one source of error associated with the aerial vehicle and the second buffer is determined based on at least one source of error associated with the physical structure or the point cloud; and

modify one or more of the waypoints when the likelihood of the aerial vehicle colliding with the physical structure exceeds a threshold to establish an updated flight plan; and

autonomously navigate the aerial vehicle about the physical structure according to an updated flight plan.

10. The aerial vehicle of claim 9 , wherein, the flight planning system is further configured to determine the likelihood of the first buffer overlapping the second buffer.

11. The aerial vehicle of claim 10 , wherein to determine the likelihood of the first buffer overlapping the second buffer, the flight planning system is configured to compare three-dimensional coordinates associated with the first buffer against three-dimensional coordinates associated with the second buffer.

12. The aerial vehicle of claim 9 , wherein the flight planning system is configured to:

obtain the flight plan from a ground control station in communication with the aerial vehicle.

13. The aerial vehicle of claim 9 , further comprising:

a camera,

wherein the flight planning system is configured to:

receive one or more images captured using the camera; and

generate the point cloud based on the one or more images.

14. The aerial vehicle of claim 9 , wherein, to modify the one or more of the waypoints according to the likelihood of the aerial vehicle colliding with the physical structure to establish the updated flight plan, the flight planning system is configured to:

remove a waypoint of the waypoints from the flight plan based on the likelihood of the aerial vehicle colliding with the physical structure indicating that a collision is likely at the waypoint.

15. The aerial vehicle of claim 9 , wherein the first buffer represents a volumetric perimeter determined for the aerial vehicle and the second buffer represents a volumetric perimeter determined for the physical structure.

16. The aerial vehicle of claim 9 , wherein the flight planning system is further configured to:

change a size of one or both of the first buffer or the second buffer based on information detected by the aerial vehicle.

17. An apparatus, comprising:

one or more processors of an aerial vehicle; and

a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to:

determine, using a point cloud representing a physical structure, a likelihood that an aerial vehicle is to collide with the physical structure based on a flight plan for navigating the aerial vehicle about the physical structure by determining whether a first buffer associated with the aerial vehicle overlaps a second buffer associated with a point cloud, wherein the first buffer is determined according to one or more sources of error associated with the aerial vehicle and the second buffer is determined according to one or more sources of error associated with the physical structure;

responsive to the likelihood that the aerial vehicle is to collide with the physical structure exceeding a threshold, navigate the aerial vehicle according to a modified flight plan; and

responsive to the likelihood that the aerial vehicle is to collide with the physical structure not exceeding the threshold, navigate the aerial vehicle according to the flight plan.

18. The apparatus of claim 17 , wherein the instructions to determine whether the first buffer associated with the aerial vehicle overlaps the second buffer associated with the point cloud, when executed by the one or more processors, cause the one or more processors to:

determine whether the likelihood that the first buffer overlaps the second buffer exceeds a threshold.

19. The apparatus of claim 17 , wherein the instructions, when executed by the one or more processors, cause the one or more processors to:

generate the point cloud based on one or more images received from the aerial vehicle.

20. The apparatus of claim 17 , wherein the instructions, when executed by the one or more processors, cause the one or more processors to:

select one or more waypoints within the flight plan based on input received from an operator.

21. The apparatus of claim 17 , wherein the instructions to determine whether the first buffer associated with the aerial vehicle overlaps the second buffer associated with the point cloud, when executed by the one or more processors, cause the one or more processors to:

compare three-dimensional coordinates associated with the first buffer against three-dimensional coordinates associated with the second buffer.

Assignments (7)
RELEASE OF SECURITY INTEREST Recorded Jul 21, 2025
From: SILICON VALLEY BANK, A DIVISION OF FIRST-CITIZENS BANK & TRUST COMPANY
To: SKYDIO, INC.
Reel/Frame 072107/0066 →
RELEASE OF SECURITY INTEREST Recorded Jul 21, 2025
From: HERCULES CAPITAL, INC.
To: SKYDIO, INC.
Reel/Frame 072128/0698 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Nov 9, 2021
From: SKYDIO, INC.
To: HERCULES CAPITAL, INC., AS COLLATERAL AND ADMINISTRATIVE AGENT
Reel/Frame 058081/0677 →
SECURITY INTEREST Recorded Nov 8, 2021
From: SKYDIO, INC.
To: SILICON VALLEY BANK
Reel/Frame 058053/0768 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 4, 2021
From: AIRWARE, LLC
To: SKYDIO, INC.
Reel/Frame 056442/0320 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 4, 2021
From: MOSTER, JOSEPH; OKAZAKI, DONNA; MICHINI, BERNARD J.
To: UNMANNED INNOVATION, INC.
Reel/Frame 056441/0939 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 4, 2021
From: UNMANNED INNOVATION, INC.
To: AIRWARE, LLC
Reel/Frame 056442/0219 →
Continuity (3)
Continuation 15869659 · Jan 12, 2018
Provisional Application 62446334 · Jan 13, 2017
Related Publication 20210358315A1 · Nov 18, 2021
Cited By (1)
US 12,682,488