Automatic assessment of damage and repair costs in vehicles
A system and method are provided for automatically estimating a repair cost for a vehicle. A method includes: receiving, at a server computing device over an electronic network, one or more images of a damaged vehicle from a client computing device; performing image processing operations on each of the one or more images to detect external damage to a first set of parts of the vehicle; inferring internal damage to a second set of parts of the vehicle based on the detected external damage; and, calculating an estimated repair cost for the vehicle based on the detected external damage and inferred internal damage based on accessing a parts database that includes repair and labor costs for each part in the first and second sets of parts.
1. A method for automatically estimating a repair cost for a vehicle, comprising:
receiving, at a server computing device over an electronic network, one or more images of a damaged vehicle from a client computing device;
removing artifacts from a first image of the one or more images;
aligning the first image to an undamaged version of the vehicle;
segmenting the first image into vehicle parts; and
detecting damage to a set of parts of the vehicle by comparing portions of each vehicle part in the first image to corresponding portions of each vehicle part in the undamaged version of the vehicle, wherein detecting damage to the set of parts includes:
comparing each of edge distribution, texture comparison, and spatial correlation of portions of each vehicle part in the first image to corresponding portions of each vehicle part in the undamaged version of the vehicle, and
determining whether each of the edge distribution, the texture comparison, and the spatial correlation exceed respective threshold difference values, wherein damage is detected in a portion of a vehicle part in the first image if at least two of the edge distribution, the texture comparison, and the spatial correlation exceed the respective threshold difference value;
training a first Convolutional Neural Network (CNN) to detect a pose of a vehicle in an input image;
training a plurality of CNNs to detect damage to a respective plurality of external vehicle parts;
executing the first CNN to detect the pose of the vehicle in the first image;
executing the plurality of CNNs to determine which external vehicle parts are damaged in the first image; and
executing a Markov Random Field (MRF) algorithm to infer damage to internal parts of the vehicle from the determined damaged external vehicle parts; and
calculating an estimated repair cost for the vehicle based on the detected damage based on accessing a parts database that includes repair costs.
2. The method of claim 1 , wherein said parts database that includes repair costs includes estimates for parts and labor for individual parts.
3. The method of claim 1 , wherein removing artifacts from the first image includes:
removing background material from the first image; and
removing specular reflection due to incident light on the vehicle shown in the first image.
4. A system for estimating a repair cost for a vehicle, comprising:
a client computing device configured to:
display one or more prompts on a display device of the client computing device to capture a portion of the vehicle that has sustained damage,
capture an image of the vehicle based on displaying an outline of the portion of the vehicle, and
transmit the image of the vehicle to a server computing device; and
the server computing device configured to:
receive the image over an electronic communications network from the client computing device,
remove artifacts from the image;
align the image to an undamaged version of the vehicle;
segment the image into vehicle parts; and
detect damage to a set of parts of the vehicle by comparing portions of each vehicle part in the image to corresponding portions of each vehicle part in the undamaged version of the vehicle, wherein detecting damage to the set of parts includes:
comparing each of edge distribution, texture comparison, and spatial correlation of portions of each vehicle part in the image to corresponding portions of each vehicle part in the undamaged version of the vehicle, and
determining whether each of the edge distribution, the texture comparison, and the spatial correlation exceed respective threshold difference values, wherein damage is detected in a vehicle part in the image if at least two of the edge distribution, the texture comparison, and the spatial correlation exceed the respective threshold difference value;
training a first Convolutional Neural Network (CNN) to detect a pose of a vehicle in an input image;
training a plurality of CNNs to detect damage to a respective plurality of external vehicle parts;
executing the first CNN to detect the pose of the vehicle in the image received by the server computing device from the client computing device;
executing the plurality of CNNs to determine which external vehicle parts are damaged in the image received by the server computing device from the client computing device; and
executing a Markov Random Field (MRF) algorithm to infer damage to internal parts of the vehicle from the determined damaged external vehicle parts; and
calculate an estimated repair cost for the vehicle based on accessing a parts database that includes repair and labor costs for each part in one or more damaged parts.
5. The system of claim 4 , wherein the client computing device is further configured to:
receive, in a first user interface screen displayed on the display device, a selection to initiate a new vehicle claim;
display, in a second user interface screen displayed on the display device, graphical elements for selection of a prompting interface for capture of images of damage to the vehicle;
receive selection of a graphical element corresponding to a prompting interface;
display, in a third user interface screen, a three-dimensional (3D) model of the vehicle;
receive a selection of a damaged part on the 3D model; and
display an outline of the selected part to be captured with a camera of the client computing device;
cause the camera of the client computing device to capture the image of the vehicle based on displaying the outline of the selected part; and
cause the image of the vehicle to be transmitted to the server computing device.
6. The system of claim 5 , wherein the client computing device is further configured to:
display a parts list;
receive a selection of a part from the parts list; and
display an outline of the part from the parts list to be captured with the camera of the client computing device.
7. The system of claim 5 , wherein the client computing device is further configured to:
display two or more vehicle views; and
for each of the two or more vehicle views, display an outline of the view of the vehicle to be captured with the camera of the client computing device.
8. The system of claim 5 , wherein the client computing device is operated by an owner or driver of the vehicle.
9. The system of claim 5 , wherein the client computing device is operated by an insurance adjuster.
10. The system of claim 5 , wherein the client computing device is operated by an employee of an auto repair shop.
11. The system of claim 4 , wherein removing artifacts from the image includes:
removing background material from the image; and
removing specular reflection due to incident light on the vehicle shown in the image.
12. The system of claim 4 , wherein displaying the one or more prompts on the display device of the client computing device comprises:
displaying a 3D model of the vehicle;
receiving a selection of a damaged part on the 3D model; and
displaying an outline of the selected part to be captured with a camera of the client computing device.
13. The system of claim 4 , wherein the client computing device comprises a mobile phone configured to execute a mobile application to capture the image of the vehicle and transmit the image to the server computing device over the electronic communications network.