Systems and methods for using machine learning for vehicle damage detection and repair cost estimation
Systems and methods for estimating the repair cost of one or more instances of vehicle damage pictured in a digital image are disclosed herein. These systems and methods may first use a damage detection neural network (NN) model to determine location(s), type(s), intensit(ies), and corresponding repair part(s) for pictured damage. Then, a repair cost estimation NN model may be given a damage type, a damage intensity, and the repair part(s) needed to determine a repair cost estimation. The training of each of the damage detection NN model and the repair cost estimation NN model is described. The manner of outputting results data corresponding to the systems and methods disclosed herein is also described.
1. A computer-implemented method of training a vehicle damage detection neural network model, comprising:
collecting a set of annotated digital images from a database, each annotated digital image comprising one or more annotations each corresponding to vehicle damage pictured in the annotated digital image, each annotation comprising:
a location of the vehicle damage pictured in the annotated digital image;
a repair part corresponding to the vehicle damage pictured in the annotated digital image;
a type of the vehicle damage pictured in the annotated digital image; and
an intensity of the vehicle damage pictured in the annotated digital image;
converting the set of annotated digital images to annotated greyscale digital images;
bifurcating the set of annotated greyscale digital images into a training subset and a validation subset;
training the vehicle damage detection neural network model using the training subset, wherein after said training the vehicle damage detection neural network model is trained to, without the use of a reference image of an undamaged vehicle, for a preprocessed user provided digital image picturing vehicle damage:
determine a location of the vehicle damage pictured in the user provided digital image;
determine a repair part corresponding to the vehicle damage pictured in the user provided digital image;
determine a type of the vehicle damage pictured in the user provided digital image; and
determine an intensity of the vehicle damage pictured in the user provided digital image; and
determining an accuracy of the vehicle damage detection neural network model using the validation subset based, at least in part, on losses determined based on the one or more annotations.
2. The computer-implemented method of claim 1 , wherein the type of the vehicle damage pictured in the annotated digital image comprises one or more of a dent, a scratch, a smash, and a break.
3. The computer-implemented method of claim 1 , wherein the intensity of the vehicle damage pictured in the annotated digital image comprises one or more of minor and severe.
4. The computer-implemented method of claim 1 , further comprising normalizing each annotated digital image of the set of annotated digital images prior to bifurcating the set of annotated digital images.
5. The computer-implemented method of claim 1 , further comprising augmenting the set of annotated greyscale digital images by modifying one or more of the annotated greyscale digital images and adding the one or more modified annotated greyscale digital images to the set of annotated greyscale digital images.
6. The computer-implemented method of claim 1 , wherein determining the accuracy of the vehicle damage detection neural network model using the validation subset comprises determining an overall loss based on the losses corresponding to each annotated greyscale digital image of the validation subset applied to the damage detection neural network model.
7. The computer-implemented method of claim 1 , wherein the vehicle damage detection neural network model comprises a mask RCNN.
8. A computer-implemented method of training a repair cost estimation neural network model, comprising:
collecting a set of historical data for damaged vehicles, each data point in the set comprising a damage type, a damage intensity, a used repair part, and a repair cost;
bifurcating the set of historical data into a training subset and a validation subset;
converting images picturing vehicle damage to greyscale digital images, each data point in the set corresponding to a respective greyscale digital image;
training the repair cost estimation neural network model using the training subset and the grayscale digital images, wherein after said training the repair cost estimation neural network model is trained to determine an estimated cost to repair vehicle damage based on a received digital image picturing the vehicle damage or on a received damage type, a received damage intensity, and a received repair part to make a repair; and
determining an accuracy of the repair cost estimation neural network model using the validation subset based, at least in part, on losses determined based on the one or more annotations.
9. The computer-implemented method of claim 8 , wherein the damage type of each data point in the set of historical data comprises one or more of a dent, a scratch, a smash, and a break.
10. The computer-implemented method of claim 8 , wherein the damage intensity of each data point in the set of historical data comprises one or more of minor and severe.
11. The computer-implemented method of claim 8 , further comprising standardizing the set of historical data by scaling to unit variance.
12. The computer-implemented method of claim 8 , wherein the repair cost estimation neural network model comprises an LSTM RNN.
13. The computer-implemented method of claim 8 , further comprising defining a learning rate for training the repair cost estimation neural network model.
14. The computer-implemented method of claim 8 , wherein each item in the set of historical data further comprises a geographic indication; and wherein after said training the repair cost estimation neural network model can further determine the estimated cost to repair damage based on a received geographic indication.
15. A computer-implemented method for providing vehicle repair information to a user device, comprising:
receiving, from the user device, a user provided digital image of a vehicle that pictures vehicle damage of the vehicle;
generating a preprocessed user provided digital image from the user provided digital image;
providing, to a damage detection neural network model trained with a training subset of grayscale images picturing vehicle damage and verified for accuracy with a validation subset of the grayscale images picturing the vehicle damage based on losses determined based, at least partially, on one or more annotations corresponding to the grayscale images, the preprocessed user provided digital image, wherein the damage detection neural network model is trained to, without a use of a reference image of an undamaged vehicle, determine a location of the vehicle damage, a repair part corresponding to the vehicle damage, a type of the vehicle damage, and an intensity of the vehicle damage using the preprocessed user provided digital image;
receiving, from the damage detection neural network model, the location of the vehicle damage, the repair part corresponding to the vehicle damage, the type of the vehicle damage, the intensity of the vehicle damage;
providing, to a repair cost estimation neural network model, the repair part corresponding to the vehicle damage, the type of the vehicle damage, and the intensity of the vehicle damage, wherein the repair cost estimation neural network model is trained to determine an estimated cost to repair the vehicle damage based on the repair part corresponding to the vehicle damage, the type of the vehicle damage, and the intensity of the vehicle damage;
receiving, from the repair cost estimation neural network model, the estimated cost to repair the vehicle damage; and
providing, to the user device, results data comprising an estimated cost to repair the vehicle damage and segmentation data overlaid on the user provided digital image indicating the location of the vehicle damage.
16. The computer-implemented method of claim 15 , wherein generating a preprocessed user provided digital image from the user provided digital image comprises normalizing the user provided digital image.
17. The computer-implemented method of claim 16 , wherein generating a preprocessed user provided digital image from the user provided digital image further comprises grey scaling the user provided digital image.
18. The computer-implemented method of claim 15 , wherein the results data further comprises the type of the vehicle damage.
19. The computer-implemented method of claim 15 , wherein the results data further comprises the intensity of the vehicle damage.
20. The computer-implemented method of claim 15 , wherein the results data further comprises the repair part associated with the vehicle damage.