IP Library Granted Patent US 10,032,267
Granted Patent B2
US 10,032,267 · App. 15/177,875 · Granted Jul 24, 2018

Automating the assessment of damage to infrastructure assets

Inventors: Robert Strebel (Endwell, NY); Reed Durand (Owego, NY); Ryan W. Andrew (Binghamton, NY); Joseph R. Grosso (Endicott, NY); Timothy Douglas (Lisle, NY); Victoria Rose Tagarelli (Endicott, NY); Richard T. Evans (Endicott, NY); David E. Murphy (Owego, NY); Bing C. Li (Vestal, NY); Joseph T. Morrison (Apalachin, NY); Stephen Ray Shafer (Vestal, NY); Benjamin P. Tongue (Vestal, NY); Khader S. Fashho (Endicott, NY)
Assignee: LOCKHEED MARTIN CORPORATION
G06T7/001G01S13/90G01S17/89G06K9/6202G06K9/6279G06T2207/10012G06T2207/10032G06T2207/30184
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,032,267
App. No.
15/177,875
Filed
Jun 9, 2016
Granted
Jul 24, 2018
Kind
B2
Examiner
DANG, DUY M
Art Unit
2667
USPC
382/154
Abstract

A computer-implemented method includes: receiving, by a computing device, sensor data associated with a geographic location; processing, by the computing device, the sensor data to identify an infrastructure asset within the geographic location; determining, by the computing device, a condition of the infrastructure asset based on processing the sensor data; and storing or outputting, by the computing device, information regarding the condition of the infrastructure asset.

Claims (79)

1. A computer-implemented method comprising:

receiving, by a computing device, sensor data associated with a geographic location;

processing, by the computing device, the sensor data to identify an infrastructure asset within the geographic location, wherein processing the sensor data comprises:

forming a first three-dimensional (3D) image of an object from the sensor data; and

determining that the object represents the infrastructure asset based on comparing predefined attributes of the infrastructure asset with attributes of the object, wherein the infrastructure asset is located in either an excepted location or an unexpected location;

determining, by the computing device, a condition of the infrastructure asset based on processing the sensor data;

storing or outputting, by the computing device, information regarding the condition of the infrastructure asset;

down selecting the sensor data based on information identifying a location of the infrastructure asset;

processing the down-selected sensor data; and

forming a second 3D image of the object based on the down-selected sensor data.

2. The method of claim 1 , further comprising classifying the infrastructure asset based on one or more classification rules and the attributes of the object representing the infrastructure asset.

3. The method of claim 1 , wherein determining the condition of the infrastructure asset comprises comparing the attributes of the object representing the infrastructure asset with predefined attributes of an undamaged infrastructure asset.

4. The method of claim 1 , further comprising:

determining a flight path for collection of the sensor data based on a density of infrastructure assets within one or more geographic locations,

wherein the sensor data is received from a set of sensors implemented in an at least one of:

an aerial vehicle that aerially surveys the geographic location;

a ground-based vehicle;

a surface or subsurface marine vehicle; and

an unmanned vehicle.

5. The method of claim 4 , wherein the sensor data includes at least one of:

Light Detection and Ranging (LiDAR) data;

Synthetic Aperture Radar (SAR) data;

Electro-optical (EO) data;

infrared imaging data;

ultraviolet imaging data;

radio frequency (RF) data;

navigation data;

electromagnetic data;

chemical gas snigger data; and

camera data.

6. The method of claim 1 , further comprising inferring a location of the infrastructure asset.

7. The method of claim 6 , wherein the inferring the location of the infrastructure asset includes:

creating search areas around expected line intersections;

detecting lines for the search regions and connected to previously detected infrastructure assets; and

determining the inferred location of the infrastructure asset based on the detected lines connected to previously detected infrastructure assets or other derived or inferred information.

8. The method of claim 1 , wherein storing or outputting information regarding the condition of the infrastructure asset includes generating a report identifying a location of the infrastructure asset and the condition of the infrastructure asset.

9. The method of claim 8 , wherein the report includes at least one of:

a map or satellite view of the geographic area; and

a table identifying the infrastructure asset and the condition of the infrastructure asset.

10. A computer program product for automatically assessing damage to infrastructure, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:

receive sensor data collected by a set of sensors implemented in a vehicle;

identify, based on the sensor data, an infrastructure asset within a geographic location surveyed by the vehicle, wherein identifying the infrastructure asset comprises:

forming a first three-dimensional (3D) image of an object from the sensor data; and

determining that the object represents the infrastructure asset based on a degree in which predefined attributes of the infrastructure asset match attributes of the object;

determine an extent of damage to the infrastructure asset based on processing the sensor data;

store or output a report indicating the extent of damage to the infrastructure asset

down select the sensor data based on information identifying a location of the infrastructure asset;

process the down-selected sensor data; and

form a second 3D image of the object based on the down-selected sensor data.

11. The computer program product of claim 10 , wherein identify the infrastructure asset further comprises inferring a location of the infrastructure asset based on at least one of:

an expected location of the infrastructure asset;

a location of surrounding infrastructure assets; and

an inferred location of surrounding infrastructure assets.

12. The computer program product of claim 10 , wherein determining the extent of damage to the infrastructure asset comprises comparing the attributes of the object representing the infrastructure asset with predefined attributes of an undamaged infrastructure asset.

13. The computer program product of claim 10 , wherein the infrastructure asset include at least one of:

a utility pole;

an electrical distribution power line;

a communication line;

a guy wire;

a gas lines;

a water line; and

a communications tower.

14. The computer program product of claim 10 , wherein the program instructions further cause the computing device to:

identify a shoreline based on the sensor data, time of collection of the sensor data, or known or provided calculated flood stage change; and

identify infrastructure assets within the shoreline.

15. A system comprising:

a CPU, a computer readable memory and a computer readable storage medium associated with a computing device;

program instructions to receive information regarding the extent of damage to an infrastructure asset, wherein the infrastructure asset is identified by:

forming a first three-dimensional (3D) image of an object from the sensor data; and

determining that the object represents the infrastructure asset based on a degree in which predefined attributes of the infrastructure asset match attributes of the object,

wherein the information regarding the extent of damage to the infrastructure asset is automatically determined via processing sensor data implemented in an aerial vehicle that surveys a geographic area defining an area affected by an event that causes damage to infrastructure;

program instructions to display the information regarding the extent of damage to the infrastructure asset,

program instructions to down select the sensor data based on information identifying a location of the infrastructure asset;

program instructions to process the down-selected sensor data; and

program instructions to form a second 3D image of the object based on the down-selected sensor data

wherein the program instructions are stored on the computer readable storage medium for execution by the CPU via the computer readable memory.

16. The system of claim 15 , wherein displaying the information regarding the extent of the damage of the infrastructure asset includes displaying at least one of:

a map or satellite view of the geographic area; and

a table identifying the infrastructure asset and the extent of damage to the infrastructure asset.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 9, 2016
From: STREBEL, ROBERT; DURAND, REED; ANDREW, RYAN W.; GROSSO, JOSEPH R.; DOUGLAS, TIMOTHY; TAGARELLI, VICTORIA ROSE; EVANS, RICHARD T.; MURPHY, DAVID E.; LI, BING C.; MORRISON, JOSEPH T.; SHAFER, STEPHEN RAY; TONGUE, BENJAMIN P.; FASHHO, KHADER S.
To: LOCKHEED MARTIN CORPORATION
Reel/Frame 038946/0108 →
Continuity (1)
Related Publication 20170358068A1 · Dec 14, 2017
Cited By (7)
US 12,443,902 US 12,505,077 US 12,530,727 US 12,541,682 US 12,572,892 US 12,586,135 US 12,723,993