IP Library Granted Patent US 9,633,258
Granted Patent B2
US 9,633,258 · App. 14/702,835 · Granted Apr 25, 2017

Hazard detection for asset management

Inventors: James R. Culp (Towson, MD); Frank D. Fenhagen, IV (Baltimore, MD); Arun Hampapur (Norwalk, CT); Xuan Liu (Yorktown Heights, NY); Sharathchandra U. Pankanti (Darien, CT)
Assignee: International Business Machines Corporation
G06K9/00476G06K9/0063G06K9/52
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,633,258
App. No.
14/702,835
Granted
Apr 25, 2017
Kind
B2
Abstract

An approach that detects locations of hazardous conditions within an infrastructure is provided. This approach uses satellite imagery, GIS data, automatic image processing, and predictive modeling to determine the location of the hazards automatically, thus optimizing infrastructure management. Specifically, a hazard detection tool provides this capability. The hazard detection tool comprises a detection component configured to: receive visual media containing asset location data about a set of physical assets, and hazard location data about potential hazards within a vicinity of each of the set of physical assets. The detection component further receives graphical information system (GIS) data containing asset location data about each of the set of physical assets. The hazard detection tool further comprises an analysis component configured to: analyze the visual media to determine if a hazardous condition exists for each of the set of physical assets; and apply the GIS data to the visual media to determine a location of hazardous conditions within the infrastructure.

Claims (47)

1. A method for detecting locations of hazardous conditions within an infrastructure, comprising:

analyzing visual media data containing asset location data about each of a plurality of physical assets;

processing a set of the visual media data using a texture-based edge-finding operation and an intensity gradient edge-locating operation to detect a boundary position of each physical asset of the plurality of physical assets and a boundary position of a set of hazardous conditions;

locating the boundary of each physical asset and the boundary of each hazardous condition on a satellite image;

identifying, based on the locating, a proximity of the boundary of each hazardous condition to the boundary of each physical asset;

detecting damage to an asset of the plurality of physical assets based on a change in appearance indicating damage to that asset;

assessing, based on the identified proximity and the detected damage, a threat each hazardous condition poses to each physical asset;

generating, from the analyzed visual media data, a plurality of visual overlay layers displayed atop the satellite image, the visual overlay layers including a street address display, a display of one or more of the hazardous conditions, and an infrastructure display of a physical asset of the plurality of physical assets associated with the one or more of the hazardous conditions; and

generating a prioritized maintenance plan based on a ranked inventory of each of the hazardous conditions, the threat each hazardous condition poses to each physical asset, and the visual media data by determining a location of relevant road intersections and traffic control points based on the street address of the one or more hazardous conditions, and relaying the location of the relevant road intersections and traffic control points to maintenance personnel.

2. The method according to claim 1 , further comprising receiving visual media containing asset location data about a plurality of physical assets, and hazard location data about potential hazards within a vicinity of each of the plurality of physical assets.

3. The method according to claim 1 , further comprising ranking each of the hazardous conditions based on severity.

4. The method according to claim 1 , the analyzing the visual media data to determine a hazardous condition for each of the plurality of physical assets comprising identifying a hazardous condition in the case that an object in vicinity to one or more of the physical assets has a recent change in appearance.

5. The method according to claim 4 , the identifying the hazardous condition comprising performing automatic image processing.

6. The method according to claim 5 , the automatic image processing comprising at least one of the following: background modeling, object detection, spatial intensity field gradient analysis, line detection, and texture detection.

7. The method according to claim 1 , the analyzing the visual media data to determine a hazardous condition for each of the plurality of physical assets further comprising determining a physical proximity of a hazardous condition to one or more of the physical assets.

8. A system for detecting locations of hazardous conditions within an infrastructure comprising:

at least one processing unit;

memory operably associated with the at least one processing unit; and

a hazard detection tool storable in memory and executable by the at least one processing unit, the hazard detection tool containing computer instructions, which when executed, cause the system to:

analyze visual media data containing asset location data about each of a plurality of physical assets;

process a set of the visual media data using a texture-based edge-finding operation and an intensity gradient edge-locating operation to detect a boundary position of each physical asset of the plurality of physical assets and a boundary position of a set of hazardous conditions;

locate the boundary of each physical asset and the boundary of each hazardous condition on a satellite image;

identify, based on the located boundaries, a proximity of the boundary of each hazardous condition to the boundary of each physical asset;

detect damage to an asset of the plurality of physical assets based on a change in appearance indicating damage to that asset;

assess, based on the identified proximity and the detected damage, a threat each hazardous condition poses to each physical asset;

generate, from the analyzed visual media data, a plurality of visual overlay layers displayed atop the satellite image, the visual overlay layers including a street address display, a display of one or more of the hazardous conditions, and an infrastructure display of a physical asset of the plurality of physical assets associated with the one or more of the hazardous conditions; and

generate a prioritized maintenance plan based on a ranked inventory of each of the hazardous conditions, the threat each hazardous condition poses to each physical asset, and the visual media data by determining a location of relevant road intersections and traffic control points based on the street address of the one or more hazardous conditions, and relaying the location of the relevant road intersections and traffic control points to maintenance personnel.

9. The system according to claim 8 , the computer instructions further causing the system to receive visual media containing asset location data about a plurality of physical assets, and hazard location data about potential hazards within a vicinity of each of the plurality of physical assets.

10. The system according to claim 8 , the computer instructions further causing the system to rank each of the hazardous conditions based on severity.

11. The system according to claim 8 , the computer instructions causing the system to analyze the visual media data to determine a hazardous condition for each of the plurality of physical assets further causing the system to identify a hazardous condition in the case that an object in vicinity to one or more of the physical assets has a recent change in appearance.

12. The system according to claim 11 , the computer instructions causing the system to identify the hazardous condition further causing the system to perform automatic image processing.

13. The system according to claim 12 , the automatic image processing comprising at least one of the following: background modeling, object detection, spatial intensity field gradient analysis, line detection, and texture detection.

14. The system according to claim 8 , the computer instructions further causing the system to determine a physical proximity of a hazardous condition to one or more of the physical assets.

15. A computer program product for detecting locations of hazardous conditions within an infrastructure, the computer program product comprising a computer readable storage device, and program instructions stored on the computer readable storage device, to:

analyze visual media data containing asset location data about each of a plurality of physical assets;

process a set of the visual media data using a texture-based edge-finding operation and an intensity gradient edge-locating operation to detect a boundary position of each physical asset of the plurality of physical assets and a boundary position of a set of hazardous conditions;

locate the boundary of each physical asset and the boundary of each hazardous condition on a satellite image;

identifying, based on the located boundaries, a proximity of the boundary of each hazardous condition to the boundary of each physical asset;

detect damage to an asset of the plurality of physical assets based on a change in appearance indicating damage to that asset;

assess, based on the identified proximity and the detected damage, a threat each hazardous condition poses to each physical asset;

generate, from the analyzed visual media data, a plurality of visual overlay layers displayed atop the satellite image, the visual overlay layers including a street address display, a display of one or more of the hazardous conditions, and an infrastructure display of a physical asset of the plurality of physical assets associated with the one or more of the hazardous conditions; and

generate a prioritized maintenance plan based on a ranked inventory of each of the hazardous conditions and the visual media data by determining a location of relevant road intersections, the threat each hazardous condition poses to each physical asset, and traffic control points based on the street address of the one or more hazardous conditions, and relaying the location of the relevant road intersections and traffic control points to maintenance personnel.

16. The computer program product of claim 15 , further comprising program instructions to receive visual media containing asset location data about a plurality of physical assets, and hazard location data about potential hazards within a vicinity of each of the plurality of physical assets.

17. The computer program product of claim 15 , further comprising program instructions to rank each of the hazardous conditions based on severity.

18. The computer program product of claim 15 , further comprising program instructions to analyze the visual media data to determine a hazardous condition for each of the plurality of physical assets further causing the system to identify a hazardous condition in the case that an object in vicinity to one or more of the physical assets has a recent change in appearance.

19. The computer program product of claim 18 , further comprising program instructions to perform automatic image processing.

20. The computer program product of claim 19 , the automatic image processing comprising at least one of the following: background modeling, object detection, spatial intensity field gradient analysis, line detection, and texture detection.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 6, 2017
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: UTOPUS INSIGHTS, INC.
Reel/Frame 042700/0530 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 4, 2015
From: CULP, JAMES R.; FENHAGEN, FRANK D., IV; HAMPAPUR, ARUN; LIU, XUAN; PANKANTI, SHARATHCHANDRA U.
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 035553/0520 →
Continuity (3)
Continuation 14504691 · Oct 2, 2014
Continuation 12955944 · Nov 30, 2010
Related Publication 20150235082A1 · Aug 20, 2015