IP Library Granted Patent US 11,670,079
Granted Patent B1
US 11,670,079 · App. 17/199,203 · Granted Jun 6, 2023

Technologies for using image data analysis to assess and classify hail damage

Inventors: Marigona Bokshi-Drotar (McKinney, TX); Jing Wan (Allen, TX); Sandra Kane (Garland, TX); Yuntao Li (Champaign, IL)
Assignee: State Farm Mutual Automobile Insurance Company
G06V20/176G06Q40/08G06Q50/16G06T7/0002G06V10/25G06V10/26G06V10/54G06V10/56G06V10/82G06T2207/20084G06V20/17
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,670,079
App. No.
17/199,203
Granted
Jun 6, 2023
Kind
B1
Abstract

Systems and methods for analyzing image data to assess property damage are disclosed. According to certain aspects, a server may analyze segmented digital image data of a roof of a property using a convolutional neural network (CNN). The server may extract a set of features from a set of regions output by the CNN. Additionally, the server may analyze the set of features using an additional image model to generate a set of outputs indicative of a confidence level that actual hail damage is depicted in the set of regions.

Claims (62)

1. A computer-implemented method of analyzing image data to automatically assess hail damage to a property, the method comprising:

accessing digital image data depicting a roof of the property;

segmenting, by a processor, the digital image data into a set of digital images depicting portions of the roof;

identifying, by the processor and using a convolutional neural network (CNN), regions of potential hail damage depicted in the set of digital images;

identifying, by the processor, features indicative of the potential hail damage and illustrated within the respective regions based at least in part on a numerical proximity of aspect ratios of the features to one, wherein identifying the features comprises:

determining sections of a digital image from the set of digital images, the sections including a first section within a region of potential hail damage and a second section outside of the region of potential hail damage; and

identifying a texture feature within the sections by performing a Gray-Level Co-Occurrence Matrices (GLCM) analysis on the sections to output a set of statistical properties comprising one or more of contrast, entropy, energy, or homogeneity for the first section and the second section; and

generating, by the processor, using a classification model, and based on the features, an output indicating a presence of hail damage associated with the roof, wherein the output is used to automatically determine an estimated damage amount to the roof of the property.

2. The computer-implemented method of claim 1 , further comprising:

training the CNN using training data comprising training images and training labels, wherein the trained CNN is configured to:

classify the set of digital images based on at least one of the training images and the training labels, and

generate images depicting the regions of potential hail damage.

3. The computer-implemented method of claim 2 , wherein the training images include at least a first image that depicts hail damage and a second image that depicts non-hail damage, and the training labels include data identifying a portion of the first image as a region depicting hail damage and identifying a remaining portion of the first image as depicting non-hail damage, the remaining portion excludes the region depicting hail damage.

4. The computer-implemented method of claim 1 , wherein identifying the features further comprises:

identifying, by the processor within the sections, one or more of a color feature or a shape feature.

5. The computer-implemented method of claim 4 , wherein identifying the color feature comprises:

generating, by the processor, histograms that represent colors depicted within the sections of the digital image; wherein the histograms are associated with statistics including one or more of a color mean value, a color skewness value, and a color variation value.

6. The computer-implemented method of claim 4 , wherein identifying the shape feature comprises:

determining, by the processor, one or more of an area or a contour curvature of the first section and the second section of the digital image.

7. The computer-implemented method of claim 1 , wherein the output comprises a set of binary outputs indicating whether hail damage is present in the features.

8. The computer-implemented method of claim 1 , wherein generating the output using the classification model comprises:

inputting, by the processor, the features into the classification model; and

assigning a confidence level to the output based on a likelihood of the features indicating the presence of hail damage in the set of digital images.

9. A system for analyzing image data to automatically assess hail damage to a property, comprising:

a memory configured to store non-transitory computer executable instructions; and

a processor interfacing with the memory, and configured to execute the non-transitory computer executable instructions to cause the processor to:

access digital image data depicting a roof of the property;

segment the digital image data into a set of digital images depicting portions of the roof;

identify, using a convolutional neural network (CNN), regions of potential hail damage depicted in the set of digital images;

identify features indicative of the potential hail damage and illustrated within the respective regions based at least in part on a numerical proximity of aspect ratios of the features to one, including by:

determining sections of a digital image from the set of digital images, the sections including a first section within a region of potential hail damage and a second section outside of the region of potential hail damage; and

identifying a texture feature within the sections by performing a Gray-Level Co-Occurrence Matrices (GLCM) analysis on the sections to output a set of statistical properties comprising one or more of contrast, entropy, energy, or homogeneity for the first section and the second section; and

generate, using a classification model, and based on the features, an output indicating a presence of hail damage associated with the roof, wherein the output is used to automatically determine an estimated damage amount to the roof of the property.

10. The system of claim 9 , wherein the processor is further configured to:

train the CNN using a set of training data comprising a set of training images and a set of training labels; and

store the trained CNN in the memory.

11. The system of claim 9 , wherein to generate the output using the classification model, the processor is configured to:

input the features into the classification model to generate a set of binary outputs respectively indicating whether hail damage is present in the features.

12. The system of claim 9 , wherein identifying the features is further based at least in part on determining that the features are associated with a color variation that meets or exceeds a threshold value.

13. The system of claim 9 , wherein to identify features indicative of the potential hail damage, the processor is configured to:

identify, from the respective regions, a set of shape features, wherein a shape feature of the set of shape features includes an area and a contour curvature.

14. The system of claim 9 , wherein to generate the output using the classification model, the processor is configured to:

input the features into the classification model; and

generate an associated confidence level based on a likelihood of the features indicating the presence of hail damage in the set of digital images.

15. A non-transitory computer-readable storage medium configured to store instructions, the instructions when executed by a processor causing the processor to perform operations comprising:

accessing digital image data depicting a roof of a property;

segmenting the digital image data into a set of digital images depicting portions of the roof;

identifying, using a convolutional neural network (CNN), regions of anomalies depicted in the set of digital images, the regions of anomalies indicating potential hail damage;

identifying features indicative of the potential hail damage and illustrated within the respective regions based at least in part on a numerical proximity of aspect ratios of the features to one, wherein identifying the features comprises:

determining sections of a digital image from the set of digital images, the sections including a first section within a region of potential hail damage and a second section outside of the region of potential hail damage; and

identifying a texture feature within the sections by performing a Gray-Level Co-Occurrence Matrices (GLCM) analysis on the sections to output a set of statistical properties comprising one or more of contrast, entropy, energy, or homogeneity for the first section and the second section; and

generating, using a classification model and based on the features, an output indicating a presence of hail damage associated with the roof, wherein the output is used to automatically determine an estimated damage amount to the roof of the property.

16. The non-transitory computer-readable storage medium of claim 15 , wherein segmenting the digital image data into the set of digital images comprises:

segmenting the digital image data into the set of digital images using a sliding window technique.

17. The non-transitory computer-readable storage medium of claim 15 , wherein generating the output using the classification model comprises:

analyzing the features using the classification model to generate a set of binary outputs respectively indicating whether hail damage is present in the features.

18. The non-transitory computer-readable storage medium of claim 15 , wherein identifying the features comprises:

identifying, within individual sections of the sections, at least one of a set of color features or a set of shape features.

19. The non-transitory computer-readable storage medium of claim 15 , wherein generating the output using the classification model comprises:

inputting the features into the classification model; and

generating an associated confidence level based on a likelihood of the features indicating actual hail damage.

20. The non-transitory computer-readable storage medium of claim 15 , wherein identifying the features within sections of the set of digital images is further based at least in part on determining that the features are associated with a color variation that meets or exceeds a threshold value.

Assignments (3)
SECURITY INTEREST Recorded Oct 20, 2025
From: ROOFR INC.
To: STIFEL BANK
Reel/Frame 072598/0354 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 21, 2025
From: STATE FARM MUTUAL AUTOMOBILE INSURANCE CO.
To: ROOFR INC.
Reel/Frame 072083/0039 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 11, 2021
From: BOKSHI-DROTAR, MARIGONA; WAN, JING; KANE, SANDRA; LI, YUNTAO
To: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANY
Reel/Frame 055568/0090 →
Continuity (1)
Continuation 16175126 · Oct 30, 2018