IP Library Granted Patent US 11,455,895
Granted Patent B2
US 11,455,895 · App. 16/987,336 · Granted Sep 27, 2022

Roof scan using unmanned aerial vehicle

Inventors: Peter Henry (San Francisco, CA); Jack Zhu (San Francisco, CA); Brian Richman (San Francisco, CA); Harrison Zheng (Palo Alto, CA); Hayk Martirosyan (San Francisco, CA); Matthew Donahoe (Redwood City, CA); Abraham Galton Bachrach (Redwood City, CA); Adam Bry (Redwood City, CA)
Assignee: Skydio, Inc.
G08G5/003B64C39/024B64D47/08G01S13/89G01S17/89G05D1/0016G05D1/0038G05D1/106G06F3/04817G06F3/04845G06F3/04847G06T17/05G06T17/10G06T19/006G06V20/13G06V20/176G08G5/04H04N5/272B64C2201/027B64C2201/123B64C2201/127B64C2201/14B64C2201/146G05D2201/0207H04N13/204
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,455,895
App. No.
16/987,336
Granted
Sep 27, 2022
Kind
B2
Abstract

Described herein are systems for roof scan using an unmanned aerial vehicle. For example, some methods include capturing, using an unmanned aerial vehicle, an overview image of a roof of a building from above the roof; presenting a suggested bounding polygon overlaid on the overview image to a user; determining a bounding polygon based on the suggested bounding polygon and user edits; based on the bounding polygon, determining a flight path including a sequence of poses of the unmanned aerial vehicle with respective fields of view at a fixed height that collectively cover the bounding polygon; fly the unmanned aerial vehicle to a sequence of scan poses with horizontal positions matching respective poses of the flight path and vertical positions determined to maintain a consistent distance above the roof; and scanning the roof from the sequence of scan poses to generate a three-dimensional map of the roof.

Claims (66)

1. An unmanned aerial vehicle comprising:

a propulsion mechanism,

one or more image sensors, and

a processing apparatus, wherein the processing apparatus is configured to:

capture, using the one or more image sensors, an overview image of a roof of a building from a first pose of the unmanned aerial vehicle positioned above the roof;

present a graphical representation of a suggested bounding polygon overlaid on the overview image to a user, wherein the suggested bounding polygon includes vertices corresponding to respective vertex icons of the graphical representation that enable the user to move the vertices within a plane;

determine a bounding polygon based on the suggested bounding polygon and data encoding user edits of one or more of the vertices of the suggested bounding polygon; and

determine a flight path based on the bounding polygon, wherein the flight path includes a sequence of poses of the unmanned aerial vehicle with respective fields of view at a fixed height that collectively cover the bounding polygon.

2. The unmanned aerial vehicle of claim 1 , wherein the processing apparatus is configured to:

control the propulsion mechanism to cause the unmanned aerial vehicle to fly to assume a sequence of scan poses with horizontal positions matching respective poses of the flight path and vertical positions determined to maintain a consistent distance above the roof; and

scan the roof from the sequence of scan poses to generate a three-dimensional map of the roof.

3. The unmanned aerial vehicle of claim 1 , wherein the processing apparatus is configured to:

present a graphical representation of the unmanned aerial vehicle overlaid on the overview image, wherein the graphical representation of the unmanned aerial vehicle corresponds to a current horizontal position of the unmanned aerial vehicle.

4. The unmanned aerial vehicle of claim 2 , wherein the graphical representation of the unmanned aerial vehicle includes a three-dimensional rendering of the unmanned aerial vehicle.

5. The unmanned aerial vehicle of claim 2 , wherein the processing apparatus is configured to:

present indications of progress along the flight path overlaid on the overview image.

6. The unmanned aerial vehicle of claim 1 , wherein the processing apparatus is configured to:

determine the flight path based on one or more scan parameters presented for selection by the user, including one or more parameters from a set of parameters including a grid size, a nominal height above a surface of the roof, and a top flight speed.

7. The unmanned aerial vehicle of claim 1 , wherein the flight path is determined as a lawn-mower pattern.

8. The unmanned aerial vehicle of claim 1 , wherein the one or more image sensors are configured to support stereoscopic imaging to provide range data, and wherein the processing apparatus is configured to:

scan the roof using the one or more image sensors to generate the three-dimensional map of the roof.

9. The unmanned aerial vehicle of claim 7 , wherein the roof is scanned to generate the three-dimensional map from a distance greater than a consistent distance later used for facet imaging.

10. The unmanned aerial vehicle of claim 1 , comprising a radar sensor configured to provide range data, and wherein the processing apparatus is configured to:

scan the roof using the radar sensor to generate the three-dimensional map of the roof.

11. The unmanned aerial vehicle of claim 1 , comprising a lidar sensor configured to provide range data, and wherein the processing apparatus is configured to:

scan the roof using the lidar sensor to generate the three-dimensional map of the roof.

12. The unmanned aerial vehicle of claim 1 , wherein the processing apparatus is configured to:

detect, while flying between poses in the sequence of scan poses, an obstacle, wherein the detection is performed based on images captured using the one or more image sensors; and

dynamically adjust a pose of the flight path to avoid the obstacle.

13. The unmanned aerial vehicle of claim 1 , wherein the processing apparatus is configured to:

store a scan state indicating a next pose of the sequence of poses of the flight path;

control the propulsion mechanism to cause the unmanned aerial vehicle to fly to a landing spot and land the unmanned aerial vehicle;

after landing, control the propulsion mechanism to cause the unmanned aerial vehicle to take off from the landing spot;

access the scan state; and

based on the scan state, control the propulsion mechanism to cause the unmanned aerial vehicle to fly to assume a pose in the sequence of scan poses corresponding to the next pose and continue scanning the roof to generate the three-dimensional map.

14. A method comprising:

capturing, using one or more image sensors of an unmanned aerial vehicle, an overview image of a roof of a building from a first pose of the unmanned aerial vehicle positioned above the roof;

presenting a graphical representation of a suggested bounding polygon overlaid on the overview image to a user, wherein the suggested bounding polygon includes vertices corresponding to respective vertex icons of the graphical representation that enable the user to move the vertices within a plane;

accessing data encoding user edits of one or more of the vertices of the suggested bounding polygon;

determining a bounding polygon based on the suggested bounding polygon and the data encoding user edits;

determining a flight path based on the bounding polygon, wherein the flight path includes a sequence of poses of the unmanned aerial vehicle with respective fields of view at a fixed height that collectively cover the bounding polygon;

controlling a propulsion mechanism to cause the unmanned aerial vehicle to fly to assume a sequence of scan poses with horizontal positions matching respective poses of the flight path and vertical positions determined to maintain a consistent distance above the roof; and

scanning the roof from the sequence of scan poses to generate a three-dimensional map of the roof.

15. The method of claim 14 , comprising:

presenting a graphical representation of the unmanned aerial vehicle overlaid on the overview image, wherein the graphical representation of the unmanned aerial vehicle corresponds to a current horizontal position of the unmanned aerial vehicle.

16. The method of claim 14 , comprising:

determining the flight path based on one or more scan parameters presented for selection by the user, including one or more parameters from a set of parameters including a grid size, a nominal height above a surface of the roof, and a top flight speed.

17. The method of claim 14 , wherein the flight path is determined as a lawn-mower pattern.

18. The method of claim 14 , comprising:

detecting, while flying between poses in the sequence of scan poses, an obstacle, wherein the detection is performed based on images captured using the one or more image sensors; and

dynamically adjusting a pose of the flight path to avoid the obstacle.

19. The method of claim 14 , comprising:

storing a scan state indicating a next pose of the sequence of poses of the flight path;

controlling the propulsion mechanism to cause the unmanned aerial vehicle to fly to a landing spot and land the unmanned aerial vehicle;

after landing, controlling the propulsion mechanism to cause the unmanned aerial vehicle to take off from the landing spot;

accessing the scan state; and

based on the scan state, controlling the propulsion mechanism to cause the unmanned aerial vehicle to fly to assume a pose in the sequence of scan poses corresponding to the next pose and continue scanning the roof to generate the three-dimensional map.

20. A non-transitory computer-readable storage medium that includes instructions that, when executed by a processor, facilitate performance of operations comprising:

capturing, using one or more image sensors of an unmanned aerial vehicle, an overview image of a roof of a building from a first pose of the unmanned aerial vehicle positioned above the roof;

presenting a graphical representation of a suggested bounding polygon overlaid on the overview image to a user, wherein the suggested bounding polygon includes vertices corresponding to respective vertex icons of the graphical representation that enable the user to move the vertices within a plane;

accessing data encoding user edits of one or more of the vertices of the suggested bounding polygon;

determining a bounding polygon based on the suggested bounding polygon and the data encoding user edits;

determining a flight path based on the bounding polygon, wherein the flight path includes a sequence of poses of the unmanned aerial vehicle with respective fields of view at a fixed height that collectively cover the bounding polygon;

controlling a propulsion mechanism to cause the unmanned aerial vehicle to fly to assume a sequence of scan poses with horizontal positions matching respective poses of the flight path and vertical positions determined to maintain a consistent distance above the roof;

scanning the roof from the sequence of scan poses to generate a three-dimensional map of the roof; and

presenting indications of progress along the flight path overlaid on the overview image.

Assignments (6)
RELEASE OF SECURITY INTEREST Recorded Jul 21, 2025
From: SILICON VALLEY BANK, A DIVISION OF FIRST-CITIZENS BANK & TRUST COMPANY
To: SKYDIO, INC.
Reel/Frame 072107/0066 →
RELEASE OF SECURITY INTEREST Recorded Jul 21, 2025
From: HERCULES CAPITAL, INC.
To: SKYDIO, INC.
Reel/Frame 072128/0698 →
SECURITY INTEREST Recorded Dec 5, 2024
From: SKYDIO, INC.
To: ACQUIOM AGENCY SERVICES LLC
Reel/Frame 069516/0452 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Nov 9, 2021
From: SKYDIO, INC.
To: HERCULES CAPITAL, INC., AS COLLATERAL AND ADMINISTRATIVE AGENT
Reel/Frame 058081/0677 →
SECURITY INTEREST Recorded Nov 8, 2021
From: SKYDIO, INC.
To: SILICON VALLEY BANK
Reel/Frame 058053/0768 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 5, 2021
From: HENRY, PETER; ZHU, JACK; RICHMAN, BRIAN; ZHENG, HARRISON; MARTIROSYAN, HAYK; DONAHOE, MATTHEW; BACHRACH, ABRAHAM GALTON; BRY, ADAM
To: SKYDIO, INC.
Reel/Frame 054807/0626 →
Cited By (2)
US 12,337,965 US 12,379,731