IP Library › Granted Patent US 12,299,811
Granted Patent B2
US 12,299,811 · App. 17/404,815 · Granted May 13, 2025

Photo deformation techniques for vehicle repair analysis

Inventor: William J. Leise (Normal, IL)
Assignee: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANY
G06T17/00G06Q10/20G06T3/4046G06T7/0002G06T7/593G06V10/82H04N23/64G06Q40/08G06T2207/10012G06T2207/10024G06T2207/10028G06T2207/20081G06T2207/30252G06T2215/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,299,811
App. No.
17/404,815
Granted
May 13, 2025
Kind
B2
Abstract

A method and system may use photo deformation techniques for vehicle repair analysis to determine a repair time for repairing a damaged vehicle part. A user's client device may generate a three-dimensional (3D) image or model of a damaged vehicle part by capturing several two-dimensional images of the damaged vehicle part. One or several characteristics of the damaged vehicle part may be extracted from the 3D model and the characteristics may be compared to characteristics for previously damaged vehicle part, where the actual repair times were measured. A repair time for the damaged vehicle part may be determined based on the comparison and displayed on the client device.

Claims (37)

1. A client device for using photo deformation techniques for vehicle repair analysis, the client device comprising:

a user interface;

one or more image sensors;

one or more processors communicatively coupled to the user interface and the one or more image sensors;

a non-transitory computer-readable memory coupled to the one or more processors, and storing thereon instructions that, when executed by the one or more processors, cause the client device to:

capture, via the one or more image sensors, three-dimensional image data depicting a damaged vehicle part from a vehicle involved in a vehicle crash;

generate a three-dimensional model of the damaged vehicle part based on the three-dimensional image data; and

analyze the three-dimensional model to identify a depth of a dent to the damaged vehicle part based on a difference in depth values within a portion of the three-dimensional model corresponding to the dent.

2. The client device of claim 1 , wherein the instructions further cause the client device to:

analyze the three-dimensional model to identify a plurality of damaged part characteristics for the damaged vehicle part including the depth of the dent to the damaged vehicle part.

3. The client device of claim 2 , wherein the damaged part characteristics further include at least one of: a type of the damaged vehicle part, a dent circumference of the dent to the damaged vehicle part, a dent radius, or a shape of the dent.

4. The client device of claim 2 , wherein the instructions further cause the client device to:

transmit the plurality of damaged part characteristics to a server device, wherein the server device compares the plurality of damaged part characteristics to a set of training data to determine a repair time for repairing the damaged vehicle part.

5. The client device of claim 1 , wherein the one or more image sensors includes a laser scanning device removably attached to the client device.

6. The client device of claim 1 , wherein to capture three-dimensional image data, the instructions cause the client device to capture a plurality of two-dimensional images each depicting the same damaged vehicle part at a plurality of positions and angles from the damaged vehicle part.

7. The client device of claim 1 , wherein to capture three-dimensional image data depicting a damaged vehicle part from the vehicle, the instructions cause the client device to:

display an image capture screen instructing the user to capture an image of the damaged vehicle part within a set of boundaries displayed on the image capture screen;

capture, via the one or more image sensors, an image of the damaged vehicle part within the set of boundaries;

display a plurality of additional image capture screens instructing the user to capture a plurality of additional images of the damaged vehicle part within a plurality of additional sets of boundaries, wherein each additional set of boundaries is at a different orientation relative to the damaged vehicle part;

capture, via the one or more image sensors, the plurality of additional images of the damaged vehicle part within the plurality of additional sets of boundaries; and

combine the captured image and plurality of additional images to generate the three-dimensional image data.

8. A method for using photo deformation techniques for vehicle repair analysis, the method executed by one or more processors programmed to perform the method, the method comprising:

capturing, by the one or more processors via one or more image sensors communicatively coupled to the one or more processors, three-dimensional image data depicting a damaged vehicle part from a vehicle involved in a vehicle crash;

generating, by the one or more processors, a three-dimensional model of the damaged vehicle part based on the three-dimensional image data; and

analyzing, by the one or more processors, the three-dimensional model to identify a depth of a dent to the damaged vehicle part based on a difference in depth values within a portion of the three-dimensional model corresponding to the dent.

9. The method of claim 8 , further comprising:

analyzing, by the one or more processors, the three-dimensional model to identify a plurality of damaged part characteristics for the damaged vehicle part including the depth of the dent to the damaged vehicle part.

10. The method of claim 9 , further comprising:

transmitting, by the one or more processors, the plurality of damaged part characteristics to a server device, wherein the server device compares the plurality of damaged part characteristics to a set of training data to determine a repair time for repairing the damaged vehicle part.

11. The method of claim 8 , wherein the one or more image sensors includes a laser scanning device removably attached to the client device.

12. The method of claim 8 , wherein capturing the three-dimensional image data includes capturing, via the one or more image sensors, a plurality of two-dimensional images each depicting the same damaged vehicle part at a plurality of positions and angles from the damaged vehicle part.

13. The method of claim 8 , wherein capturing the three-dimensional image data depicting a damaged vehicle part from the vehicle includes:

displaying, by the one or more processors, an image capture screen instructing the user to capture an image of the damaged vehicle part within a set of boundaries displayed on the image capture screen;

capturing, by the one or more processors via the one or more image sensors, an image of the damaged vehicle part within the set of boundaries;

displaying, by the one or more processors, a plurality of additional image capture screens instructing the user to capture a plurality of additional images of the damaged vehicle part within a plurality of additional sets of boundaries, wherein each additional set of boundaries is at a different orientation relative to the damaged vehicle part;

capturing, by the one or more processors via the one or more image sensors, the plurality of additional images of the damaged vehicle part within the plurality of additional sets of boundaries; and

combining, by the one or more processors, the captured image and the plurality of additional images to generate the three-dimensional image data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 18, 2021
From: LEISE, WILLIAM J.
To: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANY
Reel/Frame 057209/0487 →
Continuity (4)
Continuation 16848334 · Apr 14, 2020
Continuation 15865442 · Jan 9, 2018
Provisional Application 62444110 · Jan 9, 2017
Related Publication 20210375032A1 · Dec 2, 2021
References Cited (25)
US 8972100B2 · Mullen et al. · 2015 [cited by applicant]
US 8977425B1 · Mullen et al. · 2015 [cited by applicant]
US 9208526B1 · Leise · 2015 [cited by applicant]
US 9361735B1 · Leise · 2016 [cited by applicant]
US 9466085B2 · Mullen et al. · 2016 [cited by applicant]
US 9495667B1 · Leise · 2016 [cited by applicant]
US 9607339B1 · Mullen et al. · 2017 [cited by applicant]
US 9646345B1 · Leise · 2017 [cited by applicant]
US 9799010B1 · Leise · 2017 [cited by applicant]
US 9898784B1 · Leise · 2018 [cited by applicant]
US 9904928B1 · Leise · 2018 [cited by applicant]
US 10013718B1 · Leise · 2018 [cited by applicant]
US 10360635B2 · Hanson et al. · 2019 [cited by applicant]
US 20110276396A1 · Rathod · 2011 [cited by applicant]
GB 2554361A · 2018 [cited by applicant]
WO WO2017055878A1 · 2017 [cited by applicant]
Yunxi Wu, Yuehong Zhou and Xiaolong Peng, “Measurement and repair of structural body damage to cars in collisions,” 2011 Second International Conference on Mechanic Automation and Control Engineering, Inner Mongolia, Ch… [cited by examiner]
Srimal Jayawardena and Di Yang and Marcus Hutter, “3D Model Assisted Image Segmentation”, arXiv:1202.1943v1 [cs.CV] Feb. 9, 2012. [cited by examiner]
Min Sun, Bing-Xin Xu, Gary Bradski, and Silvio Savarese. “Depth-encoded hough voting for joint object detection and shape recovery”. In ECCV, Crete, Greece, Sep. 2010. [cited by examiner]
U.S. Appl. No. 14/627,170, filed Feb. 20, 2015. [cited by applicant]
U.S. Appl. No. 14/732,326, filed Jun. 5, 2015. [cited by applicant]
Office Action for U.S. Appl. No. 15/865,442, dated Oct. 9, 2019. [cited by applicant]
Wang, J., Yang, Y., Mao, J., Huang, Z., Huang, C., & Xu, W. (2016). CNN-RNN: A unified framework for multi-label image classification. arXiv 1604.04573 (Year: 2016). [cited by applicant]
Kira, “The Best 3D Scanners of 2015”, retrieved from <url:<http://www.3ders.org/articles/20151209-best-3d-scanners-2015.htail>> Year: 2015). [cited by applicant]
Office Action for U.S. Appl. No. 16/848,334, dated Mar. 17, 2021. [cited by applicant]
Cited By (1)
US 12,536,509