IP Library Granted Patent US 11,952,116
Granted Patent B2
US 11,952,116 · App. 17/890,889 · Granted Apr 9, 2024

Roof scan using unmanned aerial vehicle

Inventors: Peter Henry (San Francisco, CA); Jack Zhu (San Mateo, CA); Brian Richman (San Francisco, CA); Harrison Zheng (Palo Alto, CA); Hayk Martirosyan (San Francisco, CA); Matthew Donahoe (Redwood City, CA); Abraham Galton Bachrach (Emerald Hills, CA); Adam Bry (Redwood City, CA)
Assignee: Skydio, Inc.
B64C39/024B64D47/08G01S13/89G01S17/89G05D1/0016G05D1/0038G05D1/106G06F3/04817G06F3/04845G06F3/04847G06T17/05G06T17/10G06T19/006G06V20/13G06V20/17G06V20/176G06V20/647G08G5/003G08G5/04H04N5/272B64U10/13B64U2101/30B64U2201/00B64U2201/20G05D2201/0207H04N13/204
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,952,116
App. No.
17/890,889
Granted
Apr 9, 2024
Kind
B2
Abstract

Described herein are systems for roof scan using an unmanned aerial vehicle. For example, some methods include capturing, using an unmanned aerial vehicle, an overview image of a roof of a building from above the roof; presenting a suggested bounding polygon overlaid on the overview image to a user; determining a bounding polygon based on the suggested bounding polygon and user edits; based on the bounding polygon, determining a flight path including a sequence of poses of the unmanned aerial vehicle with respective fields of view at a fixed height that collectively cover the bounding polygon; fly the unmanned aerial vehicle to a sequence of scan poses with horizontal positions matching respective poses of the flight path and vertical positions determined to maintain a consistent distance above the roof; and scanning the roof from the sequence of scan poses to generate a three-dimensional map of the roof.

Claims (39)

1. A method, comprising:

presenting, within a graphical user interface (GUI) output for display at a user device, a graphical representation of a suggested bounding polygon overlaid on an overview image of a roof to enable a movement of vertices of the suggested bounding polygon within the GUI;

determining a bounding polygon based on movements of one or more of the vertices of the suggested bounding polygon within the GUI; and

determining, based on the bounding polygon, a flight path including a sequence of poses representing positions and orientations for an unmanned aerial vehicle to assume as the unmanned aerial vehicle performs an inspection of the roof.

2. The method of claim 1 , wherein the sequence of poses are at a fixed height relative to the bounding polygon.

3. The method of claim 1 , wherein the overview image is produced based on output of an initial scan of the roof.

4. The method of claim 3 , wherein the initial scan of the roof is performed using one or more of an image sensor of the unmanned aerial vehicle or a distance sensor of the unmanned aerial vehicle.

5. The method of claim 1 , comprising:

updating, while the unmanned aerial vehicle navigates according to the flight path, the GUI to show a current position of the unmanned aerial vehicle overlaid on the overview image.

6. The method of claim 1 , comprising:

generating a three-dimensional map of the roof based on output obtained during a scan of the roof according to the flight path.

7. The method of claim 6 , wherein the output used to generate the three-dimensional map includes one or more of image sensor data or distance sensor data.

8. An apparatus, comprising:

a memory; and

a processor configured to execute instructions stored in the memory to:

determine a bounding polygon based on movements, within a graphical user interface, of one or more vertices of a graphical representation of a suggested bounding polygon overlaid on an overview image of a roof; and

determine, based on the bounding polygon, a flight path including a sequence of poses representing positions and orientations for an unmanned aerial vehicle to assume as the unmanned aerial vehicle performs an inspection of the roof.

9. The apparatus of claim 8 , wherein the sequence of poses have different fields of view that collectively cover the bounding polygon.

10. The apparatus of claim 9 , wherein the movements represent user edits encoded at a user device in communication with the unmanned aerial vehicle.

11. The apparatus of claim 9 , wherein the processor is configured to execute the instructions to:

adjust a pose of the sequence of poses to avoid an obstacle detected based on one or more images captured by the unmanned aerial vehicle.

12. The apparatus of claim 9 , wherein the sequence of poses have horizontal positions matching respective poses of the flight path and vertical positions determined to maintain a consistent distance above the roof.

13. The apparatus of claim 8 , wherein the processor is configured to execute the instructions to:

generate a three-dimensional map of the roof based on an output of the inspection.

14. The apparatus of claim 8 , wherein the overview image is captured from a pose above the roof.

15. A system, comprising:

a user device configured to output, for display, a graphical user interface (GUI) including a graphical representation of a suggested bounding polygon overlaid on an overview image of a roof to enable a movement of vertices of the suggested bounding polygon within the GUI; and

an unmanned aerial vehicle configured to navigate, for an inspection of the roof, a flight path including a sequence of poses determined based on a bounding polygon, wherein the bounding polygon is determined based on movements of one or more of the vertices of the suggested bounding polygon within the GUI at the user device, and wherein the sequence of poses represents positions and orientations for the unmanned aerial vehicle to assume as the unmanned aerial vehicle performs the inspection of the roof.

16. The system of claim 15 , wherein one or more scan parameters are presented for selection at the user device, and wherein the flight path is determined using selected ones of the one or more scan parameters.

17. The system of claim 16 , wherein the one or more scan parameters correspond to one or more of a grid size, a height above a surface of the roof for the unmanned aerial vehicle to maintain while navigating according to the flight path, or a top flight speed for the unmanned aerial vehicle to use while navigating according to the flight path.

18. The system of claim 15 , wherein the unmanned aerial vehicle is configured to:

store a scan state indicating a next pose of the flight path;

fly to a landing spot to land;

after landing, take off from the landing spot; and

after taking off, fly, based on the scan state, to assume the next pose and continue scanning the roof.

19. The system of claim 15 , wherein the unmanned aerial vehicle is configured to:

detect, while navigating according to the flight path, an obstacle based on one or more images captured using the unmanned aerial vehicle; and

adjust a pose of the flight path to avoid the obstacle.

20. The system of claim 15 , wherein the GUI depicts a current position of the unmanned aerial vehicle.

Assignments (2)
SECURITY INTEREST Recorded Dec 5, 2024
From: SKYDIO, INC.
To: ACQUIOM AGENCY SERVICES LLC
Reel/Frame 069516/0452 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 19, 2022
From: HENRY, PETER; ZHU, JACK; RICHMAN, BRIAN; ZHENG, HARRISON; MARTIROSYAN, HAYK; DONAHOE, MATTHEW; BACHRACH, ABRAHAM GALTON; BRY, ADAM
To: SKYDIO, INC.
Reel/Frame 061281/0794 →
Continuity (3)
Continuation 16987336 · Aug 6, 2020
Provisional Application 62926787 · Oct 28, 2019
Related Publication 20220406193A1 · Dec 22, 2022