IP Library Granted Patent US 10,181,081
Granted Patent B2
US 10,181,081 · App. 16/049,056 · Granted Jan 15, 2019

Unmanned aircraft structure evaluation system and method

Inventors: Stephen L. Schultz (West Henrietta, NY); John Monaco (Penfield, NY)
Assignee: Pictometry International Corp.
G06K9/00637B60R1/00B64C39/024B64D47/08G01S19/39G05D1/0094G06F17/3028G06F17/30256G06F17/30268G06K9/0063G06T11/60G08G5/0039G08G5/0069G08G5/0086G08G5/045H04N5/44591B60R2300/8093B64C2201/123B64C2201/127B64C2201/141B64C2201/146G06Q40/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,181,081
App. No.
16/049,056
Granted
Jan 15, 2019
Kind
B2
Abstract

A computerized system, comprising: a computer system having an input unit, a display unit, one or more processors and one or more non-transitory computer readable medium, the one or more processors executing image display and analysis software to cause the one or more processors to: receive an identification of a structure from the input device, the structure having multiple sides, an outline, and a height; obtain characteristics of a camera mounted onto an unmanned aircraft; generate unmanned aircraft information including: flight path information configured to direct the unmanned aircraft to fly a flight path around the structure that is laterally and vertically offset from the structure, the lateral and vertical offset being dependent upon the height of the structure, an orientation of the camera relative to the unmanned aircraft, and the characteristics of the camera; and, store the unmanned aircraft information on the one or more non-transitory computer readable medium.

Claims (62)

1. A computerized method performed by an unmanned aerial vehicle (UAV) comprising one or more processors, the method comprising:

receiving, by the UAV, a flight plan comprising a plurality of flight capture points for a structure of interest having an outline, wherein the plurality of flight capture points each comprise a point having a geospatial reference, a first group of flight capture points adjacent to a first side of the outline, and a second group of flight capture points adjacent to a second side of the outline;

conducting an evaluation of the structure of interest according to the flight plan, the evaluation comprising:

directing the UAV to fly the flight path having the plurality of flight capture points, and capture images while the UAV is at the flight capture points, the flight capture points being a distance from an exterior surface of the structure of interest such that the images includes at least a particular level of detail, the UAV having a collision detection and avoidance system, and

detecting an obstruction in the flight path with the collision detection and avoidance system;

the collision detection and avoidance system steering the aircraft to bypass the obstruction, and aiming the camera at a target capture point on the exterior surface of the structure of interest to capture an image and ensure complete coverage of the structure of interest;

navigating the UAV to another inspection location of the plurality of inspection locations; and

navigating the UAV to a landing location.

2. The method of claim 1 , wherein steering the UAV to bypass the obstruction includes steering the UAV towards the structure of interest.

3. The method of claim 1 , wherein conducting the evaluation further comprises triggering the camera to capture the images based upon a schedule.

4. The method of claim 3 , wherein aiming the camera at the target capture point includes directing a yaw of the UAV.

5. The method of claim 4 , wherein directing the yaw of the UAV includes measuring a current yaw of the UAV using a position and orientation system that is part of the UAV.

6. The method of claim 1 , wherein the flight plan is generated by determining an effective image resolution of the images to be captured by the camera, characteristics of the camera, and a path length that achieves the effective image resolution with the characteristics of the camera.

7. The method of claim 1 , wherein the structure of interest includes a perimeter, and wherein the flight path follows the perimeter of the structure of interest.

8. The method of claim 1 , wherein conducting an evaluation further comprises allowing an operator to steer the UAV around the obstacle.

9. The method of claim 8 , further comprising transmitting a signal to the operator in the form of an audible alarm, prior to allowing the operator to steer the UAV around the obstacle.

10. A non-transitory computer storage medium comprising instructions that when executed by one or more processors included in an Unmanned Aerial Vehicle (UAV) having a camera to cause the UAV to perform operations comprising:

receiving, by the UAV, a flight plan comprising a plurality of flight capture points about a roof, wherein the plurality of flight capture points each comprise a waypoint having a geospatial reference, the flight capture points being spaced apart to obtain overlapping images of the roof from the camera, the images having an effective image resolution, the flight plan comprising instructions to aim the camera at a target capture point on the roof as the UAV is at one of the flight capture points;

conducting an evaluation of the roof through autonomous flight by:

directing the UAV to fly to a first one of the flight capture points of the flight plan during a first portion of the flight of the UAV, the first one of the flight capture points being spaced from the roof a path length to capture a first image having the effective image resolution,

directing the UAV to fly towards the roof during a second portion of the flight of the UAV to obtain a second image having a finer resolution than the effective image resolution,

the distance from the UAV to a selected target capture point on the roof during the second portion of the flight being less than the path length from the one of the flight capture points to the roof; and

capturing, while the UAV is spaced a distance from the roof less than the path length, the second image describing the surface of the roof, the second image having the finer resolution;

directing the UAV to fly to another one of the flight capture points after the second portion of the flight; and

navigating the UAV to a landing location.

11. The non-transitory computer storage medium of claim 10 , wherein the UAV includes a collision detection and avoidance system, and wherein the collision detection and avoidance system detects an obstacle lying along the flight path, and causes the UAV to fly closer to the roof during the second portion of the flight.

12. The non-transitory computer storage medium of claim 11 , wherein the collision detection and avoidance system is imaging based.

13. The non-transitory computer storage medium of claim 11 , wherein the collision detection and avoidance system is active sensor based.

14. The non-transitory computer storage medium of claim 11 , wherein the collision detection and avoidance system steers the UAV so as to ensure complete coverage of the roof with images captured by the camera.

15. A computerized system, comprising:

a computer system having an input unit, a display unit, one or more processors and one or more non-transitory computer readable medium, the one or more processors executing image display and analysis software to cause the one or more processors to:

receive an identification of a structure of interest from the input device, the structure of interest having multiple sides, an outline, and a height;

obtain characteristics of a camera mounted to a computer controlled gimbal mount of an unmanned aircraft to capture images of the structure of interest;

obtain geographic locations of obstructions in potential flight paths pertaining to the structure of interest; and

generate unmanned aircraft information including:

flight path information, camera control information, and gimbal control information, the flight path information configured to direct the unmanned aircraft to fly a flight path around the structure of interest that is laterally and/or vertically offset from an exterior surface of the structure of interest, and avoid the geographic locations of obstructions, the lateral and vertical offset being dependent upon the height of the structure, an orientation of the camera relative to the unmanned aircraft, the characteristics of the camera, and geographic locations of obstructions, the camera control information configured to control the camera to capture aerial images from the camera on the unmanned aircraft while the unmanned aircraft is flying the flight path, the gimbal control information configured to control the computer controlled gimbal mount to align the camera with the structure of interest during capture of the aerial images;

receive aerial images from the unmanned aircraft; and

generate a structure report for evaluating the structure of interest using one or more aerial images obtained from the camera of the unmanned aircraft.

16. The computerized system of claim 15 , wherein the lateral and vertical offsets are determined to provide an effective image resolution due to the characteristics of the camera.

17. The computerized system of claim 15 , wherein the one or more processors executing image display and analysis software to cause the one or more processors to obtain geographic locations of obstructions in potential flight paths pertaining to the structure of interest by detecting the obstructions through the use of imagery selected from a group of imagery consisting of nadir images, ortho-rectified images or oblique geo-referenced images.

18. The computerized system of claim 15 , wherein the one or more processors executing image display and analysis software cause the one or more processors to permit an operator to validate the flight path prior to receiving aerial images from the unmanned aircraft and generating the structure report.

19. The computerized system of claim 18 , wherein the one or more processors executing image display and analysis software cause the one or more processors to permit an operator to alter the flight path prior to receiving aerial images from the unmanned aircraft and generating the structure report.

20. The computerized system of claim 15 , wherein the one or more processors executing image display and analysis software cause the one or more processors to permit an operator to manually manipulate the flight path of the unmanned aircraft during flight of the unmanned aircraft.

21. The computerized system of claim 15 , wherein the one or more processors executing image display and analysis software cause the one or more processors to obtain information pertaining to a footprint of the structure of interest, the footprint being a two-dimensional boundary of the structure of interest, and use the footprint to generate the unmanned aircraft information.

22. A computerized system, comprising:

a computer system having an input unit, a communication port, a display unit, one or more processors and one or more non-transitory computer readable medium, the one or more processors executing image display and analysis software to cause the one or more processors to:

receive an identification of a structure of interest from the input device, the structure of interest having multiple sides, an outline, and a height;

obtain characteristics of a camera mounted to a computer controlled gimbal mount of an unmanned aircraft to capture images of the structure of interest;

obtain geographic locations of obstructions in potential flight paths pertaining to the structure of interest; and

generate unmanned aircraft information including:

flight path information, camera control information, and gimbal control information, the flight path information configured to direct the unmanned aircraft to fly a flight path around the structure of interest that is laterally and/or vertically offset from an exterior surface of the structure of interest, and avoid the geographic locations of obstructions, the flight path being dependent upon the height of the structure, an orientation of the camera relative to the unmanned aircraft, the characteristics of the camera, and geographic locations of obstructions, the camera control information configured to control the camera to capture aerial images from the camera on the unmanned aircraft while the unmanned aircraft is flying the flight path, the gimbal control information configured to control the computer controlled gimbal mount to align the camera with the structure of interest during capture of the aerial images;

permit an operator to manually manipulate the flight path of the unmanned aircraft during flight of the unmanned aircraft;

receive aerial images from the unmanned aircraft; and

generate a structure report for evaluating the structure of interest using one or more aerial images obtained from the camera of the unmanned aircraft.

23. The computerized system of claim 22 , wherein the flight path is determined to provide an effective image resolution due to the characteristics of the camera.

24. The computerized system of claim 22 , wherein the one or more processors executing image display and analysis software to cause the one or more processors to obtain geographic locations of obstructions in potential flight paths pertaining to the structure of interest by detecting the obstructions through the use of imagery selected from a group of imagery consisting of nadir images, ortho-rectified images or oblique geo-referenced images.

25. The computerized system of claim 22 , wherein the one or more processors executing image display and analysis software cause the one or more processors to permit an operator to validate the flight path prior to receiving aerial images from the unmanned aircraft and generating the structure report.

26. The computerized system of claim 22 , wherein the one or more processors executing image display and analysis software cause the one or more processors to permit an operator to alter and validate the flight path prior to receiving aerial images from the unmanned aircraft and generating the structure report.

27. The computerized system of claim 22 , wherein the one or more processors executing image display and analysis software cause the one or more processors to obtain information pertaining to a footprint of the structure of interest, the footprint being a two-dimensional boundary of the structure of interest, and use the footprint to generate the unmanned aircraft information.

28. The computerized system of claim 22 , wherein the gimbal control information configured to control the computer controlled gimbal mount controls a yaw and a declination of the camera relative to the structure of interest to align the camera with the structure of interest during capture of the aerial images.

29. The computerized system of claim 22 , wherein the gimbal control information configured to control the computer controlled gimbal mount controls a declination of the camera relative to the structure of interest, and the flight path information controls a yaw of the unmanned aircraft to align the camera with the structure of interest during capture of the aerial images.

30. The computerized system of claim 22 , wherein the input unit includes a remote control, and wherein the operator is permitted to manually manipulate the flight path of the unmanned aircraft during flight of the unmanned aircraft using the remote control.

Assignments (4)
RELEASE OF FIRST LIEN SECURITY INTEREST IN PATENTS Recorded Apr 9, 2025
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: PICTOMETRY INTERNATIONAL CORP.; EAGLE VIEW TECHNOLOGIES, INC.; OMNIEARTH, INC.
Reel/Frame 070786/0022 →
FIRST LIEN SECURITY AGREEMENT Recorded Mar 28, 2025
From: EAGLE VIEW TECHNOLOGIES, INC.; PICTOMETRY INTERNATIONAL CORP.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 070671/0078 →
FIRST LIEN PATENT SECURITY AGREEMENT SUPPLEMENT Recorded Feb 3, 2025
From: EAGLE VIEW TECHNOLOGIES, INC.; OMNIEARTH, INC.; PICTOMETRY INTERNATIONAL CORP.
To: MORGAN STANLEY SENIOR FUNDING, INC. AS COLLATERAL AGENT
Reel/Frame 070096/0485 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 30, 2018
From: SCHULTZ, STEPHEN L.; MONACO, JOHN
To: PICTOMETRY INTERNATIONAL CORP.
Reel/Frame 046502/0462 →
Continuity (5)
Continuation 15802950 · Nov 3, 2017
Continuation 15475978 · Mar 31, 2017
Continuation 14591556 · Jan 7, 2015
Provisional Application 61926137 · Jan 10, 2014
Related Publication 20180336409A1 · Nov 22, 2018
Cited By (2)
US 12,429,322 US 12,498,208