IP Library Granted Patent US 9,390,489
Granted Patent B1
US 9,390,489 · App. 14/808,502 · Granted Jul 12, 2016

Method and system for assessing damage to infrastructure

Inventors: Brian N. Harvey (Bloomington, IL); Nathan L. Tofte (Downs, IL); Roger D. Schmidgall (Normal, IL); Michael Jeffrey Aviles (Bloomington, IL); Kyle Pott (Bloomington, IL); Rosemarie Geier Grant (Ellsworth, IL); Eric Haefli (Bloomington, IL); Michael Shawn Jacob (Le Roy, IL)
Assignee: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANY
G06T7/0008G06K9/00637G06K9/00651G06T2200/04G06T2200/24G06T2207/10024G06T2207/10032G06T2207/30184
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,390,489
App. No.
14/808,502
Granted
Jul 12, 2016
Kind
B1
Abstract

A method and system may assess the damage to infrastructure using aerial images captured from an unmanned aerial vehicle (UAV), a manned aerial vehicle (MAV) or from a satellite device. Specifically, an item of infrastructure may be identified for assessing damage. The UAV, MAV, or satellite device may then capture aerial images within an area which surrounds the identified infrastructure item. Subsequently, the aerial images may be analyzed to determine a condition and the extent and/or severity of the damage to the infrastructure item. Furthermore, the aerial images along with indications of the extent of the damage may be displayed on a computing device.

Claims (58)

1. A method for assessing damage to infrastructure using aerial images, the method executed by one or more processors programmed to perform the method, the method comprising:

identifying, by the one or more processors, an item of infrastructure for assessing damage;

receiving, at the one or more processors, one or more aerial images from an aerial image capturing device, wherein the one or more aerial images display the item of infrastructure;

for each of a plurality of portions of the item of infrastructure:

determining, by the one or more processors, a type of the portion of the item of infrastructure;

determining, by the one or more processors, a damage severity level of the portion of the item of infrastructure based on the one or more aerial images at least in part by determining, according to a set of rules, whether the portion of the item of infrastructure needs to be replaced based on the type of the portion of the item of infrastructure, wherein when the portion of the item of infrastructure is a first type, the set of rules indicate that the portion needs to be replaced based on a condition of the portion and when the portion of the item of infrastructure is a second type, the set of rules indicate that the portion can be repaired based on the condition;

assigning, by the one or more processors, a damage severity level indicator to the portion of the item of infrastructure based on the damage severity level; and

displaying, by the one or more processors, the item of infrastructure and the plurality of damage severity level indicators corresponding to each of the plurality of portions of the item of infrastructure on a user interface;

wherein in response to identifying the item of infrastructure, the one or more processors cause the aerial image capturing device to capture the plurality of aerial images which display the item of infrastructure.

2. The method of claim 1 , wherein the plurality of aerial images are received via an unmanned aerial vehicle or a satellite.

3. The method of claim 2 , further comprising:

directing, by the one or more processors, the unmanned aerial vehicle to one or more locations for capturing the plurality of aerial images.

4. The method of claim 1 , wherein at least two of the plurality of damage severity level indicators are displayed adjacent to each other in a width direction overlaying the item of infrastructure.

5. The method of claim 1 , further comprising:

determining, by the one or more processors, a plurality of conditions, each condition corresponding to a different portion of the item of infrastructure;

wherein each damage severity level is determined based on the condition of the respective portion of the item of infrastructure.

6. The method of claim 1 , wherein receiving one or more aerial images comprises:

determining, by the one or more processors, a location of the item of infrastructure, including a set of location boundaries; and

receiving, at the one or more processors, aerial images captured from a plurality of locations with the set of location boundaries.

7. The method of claim 1 , further comprising:

aggregating, by the one or more processors, the one or more aerial images to generate a three dimensional display of the item of infrastructure on the user interface.

8. The method of claim 1 , wherein assigning a damage severity level indicator comprises:

determining, by the one or more processors, a plurality of damage severity level categories, wherein each damage severity level category is associated with a different damage severity level indicator; and

assigning, by the one or more processors, the damage severity level indicator to the portion of the item of infrastructure based on a damage severity level category which corresponds to the damage severity level of the portion of the item of infrastructure.

9. The method of claim 1 , further comprising:

determining, by the one or more processors, a cost estimate for repairing or replacing the infrastructure item based on at least one of: a type of the item of infrastructure, a quality of the item of infrastructure, a size of the item of infrastructure, or the damage severity level for the portion of the item of infrastructure.

10. The method of claim 1 , wherein the item of infrastructure includes at least one of: a highway, a road, a bridge, a tunnel, a sewer treatment plant, a water treatment plant, a reservoir, an aqueduct, an electric power grid, a communications tower, a sidewalk, a paved walkway, a rail line, a waterway, a port facility, or a public transportation system.

11. A system for assessing damage to infrastructure using aerial images, the system comprising:

one or more processors;

a user interface;

a communication network; and

a non-transitory computer-readable memory coupled to the one or more processors, the user interface, and the communication network, and storing thereon instructions that, when executed by the one or more processors, cause the system to:

identify an item of infrastructure for assessing damage;

receive one or more aerial images from an aerial image capturing device, wherein the one or more aerial images display the item of infrastructure;

for each of a plurality of portions of the item of infrastructure:

determine a type of the portion of the item of infrastructure;

determine a damage severity level of the portion of the item of infrastructure based on the one or more aerial images at least in part by determining, according to a set of rules, whether the portion of the item of infrastructure needs to be replaced based on the type of the portion of the item of infrastructure, wherein when the portion of the item of infrastructure is a first type, the set of rules indicate that the portion needs to be replaced based on a condition of the portion and when the portion of the item of infrastructure is a second type, the set of rules indicate that the portion can be repaired based on the condition;

assign a damage severity level indicator to the portion of the item of infrastructure based on the damage severity level; and

display the item of infrastructure and the plurality of damage severity level indicators corresponding to each of the plurality of portions of the item of infrastructure on the user interface;

wherein in response to identifying the item of infrastructure, the one or more processors cause the aerial image capturing device to capture the plurality of aerial images which display the item of infrastructure.

12. The system of claim 11 , wherein the plurality of aerial images are received via an unmanned aerial vehicle or a satellite.

13. The system of claim 12 , wherein the instructions further cause the system to:

direct the unmanned aerial vehicle to one or more locations for capturing the plurality of aerial images.

14. The system of claim 11 , wherein at least two of the plurality of damage severity level indicators are displayed adjacent to each other in a width direction overlaying the item of infrastructure.

15. The system of claim 11 , wherein the instructions further cause the system to:

determine a plurality of conditions, each condition corresponding to a different portion of the item of infrastructure;

wherein each damage severity level is determined based on the condition of the respective portion of the item of infrastructure.

16. The system of claim 11 , wherein to receive one or more aerial images, the instructions cause the system to:

determine a location of the item of infrastructure, including a set of location boundaries; and

receive aerial images captured from a plurality of locations within the set of location boundaries.

17. The system of claim 11 , wherein the instructions further cause the system to:

aggregate the one or more aerial images to generate a three dimensional display of the item of infrastructure on the user interface.

18. The system of claim 11 , wherein to assign a damage severity level indicator, the instructions cause the system to:

determine a plurality of damage severity level categories, wherein each damage severity level category is associated with a different damage severity level indicator; and

assign the damage severity level indicator to the portion of the item of infrastructure based on a damage severity level category which corresponds to the damage severity level of the portion of the item of infrastructure.

19. The system of claim 11 , wherein the instructions further cause the system to:

determine a cost estimate for repairing or replacing the infrastructure item based on at least one of: a type of the item of infrastructure, a quality of the item of infrastructure, a size of the item of infrastructure, or the damage severity level for the portion of the item of infrastructure.

20. The system of claim 11 , wherein the item of infrastructure includes at least one of: a highway, a road, a bridge, a tunnel, a sewer treatment plant, a water treatment plant, a reservoir, an aqueduct, an electric power grid, a communications tower, a sidewalk, a paved walkway, a rail line, a waterway, a port facility, or a public transportation system.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 18, 2025
From: STATE FARM MUTUAL AUTOMOBILE INSURANCE CO.
To: NEARMAP US, INC.
Reel/Frame 070548/0732 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 24, 2015
From: HARVEY, BRIAN N.; TOFTE, NATHAN L.; SCHMIDGALL, ROGER D.; AVILES, MICHAEL JEFFREY; POTT, KYLE; GRANT, ROSEMARIE GEIER; HAEFLI, ERIC; JACOB, MICHAEL SHAWN
To: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANY
Reel/Frame 036172/0711 →
Continuity (1)
Continuation 14510784 · Oct 9, 2014