IP Library Granted Patent US 9,805,456
Granted Patent B1
US 9,805,456 · App. 15/165,457 · Granted Oct 31, 2017

Method and system for assessing damage to infrastructure

Inventors: Brian N. Harvey (Bloomington, IL); Nathan L. Tofte (Downs, IL); Roger D. Schmidgall (Normal, IL); Michael Jeffrey Aviles (Bloomington, IL); Kyle Pott (Bloomington, IN); Rosemarie Geier Grant (Ellsworth, IL); Eric Haefli (Bloomington, IL); Michael Shawn Jacob (Le Roy, IL)
Assignee: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANY
G06T7/0004G06K9/4604G06K9/52G06K9/6267G06T7/0085G06T7/60G06T15/005G06T2200/04G06T2207/10032G06T2207/30108
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,805,456
App. No.
15/165,457
Granted
Oct 31, 2017
Kind
B1
Abstract

A method and system may assess the damage to infrastructure using aerial images captured from an unmanned aerial vehicle (UAV), a manned aerial vehicle (MAV) or from a satellite device. Specifically, an item of infrastructure may be identified for assessing damage. The UAV, MAV, or satellite device may then capture aerial images within an area which surrounds the identified infrastructure item. Subsequently, the aerial images may be analyzed to determine a condition and the extent and/or severity of the damage to the infrastructure item. Furthermore, the aerial images along with indications of the extent of the damage may be displayed on a computing device, where one indication of the severity of the damage surrounds another indication of the severity of the damage on the display.

Claims (58)

1. A method for assessing damage to infrastructure using aerial images, the method executed by one or more processors programmed to perform the method, the method comprising:

identifying, by the one or more processors, an item of infrastructure for assessing damage;

receiving, at the one or more processors, one or more aerial images from an aerial image capturing device, wherein the one or more aerial images display the item of infrastructure;

for each of a plurality of portions of the item of infrastructure:

determining, by the one or more processors, a damage severity level of the portion of the item of infrastructure based on the one or more aerial images; and

assigning, by the one or more processors, a damage severity level indicator to the portion of the item of infrastructure based on the damage severity level; and

displaying, by the one or more processors, the item of infrastructure and the plurality of damage severity level indicators corresponding to each of the plurality of portions of the item of infrastructure on a user interface, wherein a first damage severity level indicator surrounds a second damage severity level indicator on the user interface;

wherein in response to identifying the item of infrastructure, the one or more processors cause the aerial image capturing device to capture the plurality of aerial images which display the item of infrastructure.

2. The method of claim 1 , wherein the aerial image capturing device is an unmanned aerial vehicle or a satellite.

3. The method of claim 2 , further comprising:

directing, by the one or more processors, the unmanned aerial vehicle to one or more locations for capturing the plurality of aerial images.

4. The method of claim 1 , further comprising:

determining, by the one or more processors, a plurality of conditions, each condition corresponding to a different portion of the item of infrastructure;

wherein each damage severity level is determined based on the condition of the respective portion of the item of infrastructure.

5. The method of claim 1 , wherein receiving one or more aerial images comprises:

determining, by the one or more processors, a location of the item of infrastructure, including a set of location boundaries; and

receiving, at the one or more processors, aerial images captured from a plurality of locations with the set of location boundaries.

6. The method of claim 1 , further comprising:

aggregating, by the one or more processors, the one or more aerial images to generate a three dimensional display of the item of infrastructure on the user interface.

7. The method of claim 1 , wherein assigning a damage severity level indicator comprises:

determining, by the one or more processors, a plurality of damage severity level categories, wherein each damage severity level category is associated with a different damage severity level indicator; and

assigning, by the one or more processors, the damage severity level indicator to the portion of the item of infrastructure based on a damage severity level category which corresponds to the damage severity level of the portion of the item of infrastructure.

8. The method of claim 1 , further comprising:

determining, by the one or more processors, a cost estimate for repairing or replacing the infrastructure item based on at least one of: a type of the item of infrastructure, a quality of the item of infrastructure, a size of the item of infrastructure, or the damage severity level for the portion of the item of infrastructure.

9. The method of claim 8 , further comprising:

determining, by the one or more processors, whether the portion of the item of infrastructure needs to be replaced based on the type of the item of infrastructure.

10. The method of claim 1 , wherein the item of infrastructure includes at least one of: a highway, a road, a bridge, a tunnel, a sewer treatment plant, a water treatment plant, a reservoir, an aqueduct, an electric power grid, a communications tower, a sidewalk, a paved walkway, a rail line, a waterway, a port facility, or a public transportation system.

11. A system for assessing damage to infrastructure using aerial images, the system comprising:

one or more processors;

a user interface;

a communication network; and

a non-transitory computer-readable memory coupled to the one or more processors, the user interface, and the communication network, and storing thereon instructions that, when executed by the one or more processors, cause the system to:

identify an item of infrastructure for assessing damage;

receive one or more aerial images from an aerial image capturing device, wherein the one or more aerial images display the item of infrastructure;

for each of a plurality of portions of the item of infrastructure:

determine a damage severity level of the portion of the item of infrastructure based on the one or more aerial images; and

assign a damage severity level indicator to the portion of the item of infrastructure based on the damage severity level; and

display the item of infrastructure and the plurality of damage severity level indicators corresponding to each of the plurality of portions of the item of infrastructure on the user interface, wherein a first damage severity level indicator surrounds a second damage severity level indicator on the user interface;

wherein in response to identifying the item of infrastructure, the one or more processors cause the aerial image capturing device to capture the plurality of aerial images which display the item of infrastructure.

12. The system of claim 11 , wherein the aerial image capturing device is an unmanned aerial vehicle or a satellite.

13. The system of claim 12 , wherein the instructions further cause the system to:

direct the unmanned aerial vehicle to one or more locations for capturing the plurality of aerial images.

14. The system of claim 11 , wherein the instructions further cause the system to:

determine a plurality of conditions, each condition corresponding to a different portion of the item of infrastructure;

wherein each damage severity level is determined based on the condition of the respective portion of the item of infrastructure.

15. The system of claim 11 , wherein to receive one or more aerial images, the instructions cause the system to:

determine a location of the item of infrastructure, including a set of location boundaries; and

receive aerial images captured from a plurality of locations within the set of location boundaries.

16. The system of claim 11 , wherein the instructions further cause the system to:

aggregate the one or more aerial images to generate a three dimensional display of the item of infrastructure on the user interface.

17. The system of claim 11 , wherein to assign a damage severity level indicator, the instructions cause the system to:

determine a plurality of damage severity level categories, wherein each damage severity level category is associated with a different damage severity level indicator; and

assign the damage severity level indicator to the portion of the item of infrastructure based on a damage severity level category which corresponds to the damage severity level of the portion of the item of infrastructure.

18. The system of claim 11 , wherein the instructions further cause the system to:

determine a cost estimate for repairing or replacing the infrastructure item based on at least one of: a type of the item of infrastructure, a quality of the item of infrastructure, a size of the item of infrastructure, or the damage severity level for the portion of the item of infrastructure.

19. The system of claim 18 , wherein the instructions further cause the system to:

determine whether the portion of the item of infrastructure needs to be replaced based on the type of the item of infrastructure.

20. The system of claim 11 , wherein the item of infrastructure includes at least one of: a highway, a road, a bridge, a tunnel, a sewer treatment plant, a water treatment plant, a reservoir, an aqueduct, an electric power grid, a communications tower, a sidewalk, a paved walkway, a rail line, a waterway, a port facility, or a public transportation system.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 18, 2025
From: STATE FARM MUTUAL AUTOMOBILE INSURANCE CO.
To: NEARMAP US, INC.
Reel/Frame 070548/0732 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 26, 2016
From: HARVEY, BRIAN N.; TOFTE, NATHAN L.; SCHMIDGALL, ROGER D.; AVILES, MICHAEL JEFFREY; POTT, KYLE; GRANT, ROSEMARIE GEIER; HAEFLI, ERIC; JACOB, MICHAEL SHAWN
To: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANY
Reel/Frame 038729/0017 →
Continuity (2)
Continuation 14808502 · Jul 24, 2015
Continuation 14510784 · Oct 9, 2014