Systems and methods for repairing a damaged vehicle using image processing
An image processing system and method obtains one or more source images in which a damaged vehicle is represented, and performs one or more image processing techniques on the obtained images to determine, predict, estimate, and/or detect damage that has occurred at various locations on the vehicle. Specifically, the system/method generates damage parameter values based on the image-processed images, where each value corresponds to damage at a particular location on the vehicle. The image processing techniques may include generating a composite image of the damaged vehicle, aligning and/or isolating the image, applying convolutional neural network techniques to the image to generate the damage parameter values, etc. Based on the generated values, the image processing system/method determines, predicts, estimates, and/or selects a set of parts that are needed to repair the vehicle and, in some embodiments, orders the parts needed to repair the vehicle.
1. A method performed by an image processing system, comprising:
obtaining a source image that includes an image of a damaged vehicle;
applying, by a computing device, an image processor component of the image processing system to the source image, including:
selecting a particular base object model corresponding to the damaged vehicle from a plurality of base object models, each base object model of the plurality of base object models depicting (i) a respective vehicle, of a plurality of vehicles, in an undamaged condition and (ii) a respective plurality of surface area segments of the respective vehicle in the undamaged condition, the respective vehicle in the undamaged condition corresponding to the particular base object model being a particular undamaged vehicle;
selecting, based on the selected particular base object model, a plurality of different convolutional filters, each different convolutional filter corresponding to a different landmark of the particular base object model;
applying the plurality of different convolutional filters to the source image to determine a depiction of a plurality of surface segments of the damaged vehicle, the depicted plurality of surface area segments of the damaged vehicle corresponding to the depicted respective plurality of surface area segments of the particular undamaged vehicle;
for each depicted surface area segment of the damaged vehicle, selecting a different plurality of convolutional neural networks (CNNs), each CNN included in the selected different plurality of CNNs having been trained on images of a respective surface area segment of the particular base object model to detect a different type of damage to the respective surface area segment of the particular base object model, each different type of damage included in a set of damage types including dents, scratches, creases, bends, cracks, scuffs, scrapes, missing parts, and folds;
analyzing the image of the damaged vehicle using the particular base object model and the selected different plurality of CNNs associated with the particular base object model to determine a set of changes to the damaged vehicle, including:
detecting, by utilizing one or more CNNs of the selected different plurality of CNNs associated with the particular base object model, one or more changes to the respective surface area segments corresponding to the one or more CNNs; and
generating, based on the detected one or more changes to the respective surface area segments, a plurality of values corresponding to damage to the damaged vehicle, each value included in the plurality of values respectively corresponding to respective damage at a respective location on a corresponding surface area segment of the damaged vehicle;
based on the generated plurality of values corresponding to the damage to the damaged vehicle, identifying, by the computing device, one or more parts of the damaged vehicle needing repair or replacement;
generating an indication of the identified one or more parts of the damaged vehicle needing repair or replacement; and
providing the indication of the identified one or more parts to at least one of a user interface or another computing device.
2. The method of claim 1 , further comprising ordering the one or more parts.
3. The method of claim 1 , wherein applying the image processor component to the source image that includes the image of the damaged vehicle further comprises generating a composite, three-dimensional image of the damaged vehicle from a plurality of two-dimensional images, the plurality of two-dimensional images including the source image.
4. The method of claim 3 , further comprising aligning the composite, three-dimensional image of the damaged vehicle with a three-dimensional model of an undamaged vehicle, the three-dimensional model corresponding to the particular base object model.
5. The method of claim 4 , wherein aligning the composite, three-dimensional image of the damaged vehicle with the three-dimensional model of the undamaged vehicle comprises isolating the image of the damaged vehicle from the source image.
6. The method of claim 1 , wherein identifying the one or more parts of the damaged vehicle needing repair or replacement based on the generated plurality of values comprises filtering a set of vehicle parts of the damaged vehicle and selecting the one or more parts from the filtered set of vehicle parts based on at least some of the generated plurality of values.
7. The method of claim 1 , wherein identifying the one or more parts of the damaged vehicle needing repair or replacement based on the generated plurality of values comprises predicting, using a predictive parts model, the one or more parts of the damaged vehicle needing repair or replacement, including providing at least one value of the generated plurality of values as an input to the predictive parts model and receiving an indication of the one or more parts of the damaged vehicle needing repair or replacement from an output of the predictive parts model.
8. The method of claim 1 , wherein identifying the one or more parts of the damaged vehicle needing repair or replacement is based on two or more values included in the generated plurality of values, the two or more values corresponding to adjacent surface area segments of the particular base object model.
9. The method of claim 1 , wherein identifying the one or more parts of the damaged vehicle needing repair or replacement comprises selecting either (i) one or more individual parts, or (ii) an integral unit including the one or more individual parts based on at least one of an availability of the one or more individual parts or an availability of the integral unit.
10. The method of claim 1 , wherein identifying the one or more parts of the damaged vehicle needing repair or replacement comprises selecting either (i) one or more individual parts, or (ii) an integral unit including the one or more individual parts based on at least one of a cost of the one or more individual parts or a cost of the integral unit.
11. The method of claim 1 , wherein:
the selected different plurality of CNNs further includes at least one CNN trained on images of the respective surface area segment of the particular base object model to determine a presence or a degree of damage at the respective surface area segment; and
the each value included in the generated plurality of values is indicative of at least one of: a determined degree of severity of damage at the respective location, a type of damage at the respective location, or a likelihood of an occurrence of damage at the respective location.
12. The method of claim 1 , further comprising determining at least one of a cost of or a time interval for repairing the damaged vehicle based on the one or more parts, and providing an indication of the at least one of cost of or the time interval for repairing the damaged vehicle to at least one of the user interface or the another computing device.
13. The method of claim 12 , further comprising:
determining an insurance cost corresponding to the damaged vehicle, the insurance cost comprising at least one of a cost of an insurance claim corresponding to the damaged vehicle, an out-of-pocket cost for an insured party, or a cost of a supplement to the insurance claim corresponding to damaged vehicle; and
providing an indication of the insurance cost to at least one of the user interface or the another computing device.
14. The method of claim 13 , wherein determining the insurance cost corresponding to the damaged vehicle comprises:
providing, by the computing device, at least one value of the generated plurality of values to a prediction model that has been generated based on historical vehicle claim data of a plurality of vehicles; and
receiving, from the prediction model, a prediction of the insurance cost corresponding to the damaged vehicle.
15. The method of claim 1 , wherein at least one of:
the method is integrally performed by the image processing system exclusive of any user input; or
obtaining the source image comprises obtaining a source image captured by a mobile device.
16. The method of claim 1 , wherein:
the image processing system is a back-end system including the image processor component, the computing device, and one or more memories storing respective indications of respective vehicle parts included in the set of vehicle parts of the vehicle; and
the at least one of the user interface or the another computing device is included in another system that is communicatively connected to the back-end system.
17. The method of claim 16 , wherein the image processor component generates the plurality of values corresponding to the damage to the vehicle based on historical data stored at the back-end system, the historical data including imaging data, parts data, and repair data.
18. The method of claim 1 , wherein providing the indication of the identified one or more parts to the at least one of the user interface or the another computing device comprises providing respective indications of respective confidence levels of at least some of the identified one or more parts in conjunction with providing the indication of the identified one or more parts to the at least one of the user interface or the another computing device.
19. The method of claim 1 , wherein a respective size of the each surface area segment of the particular base object model varies in accordance with a respective gradient of curvature of a respective location included in the each surface area segment of the particular base object model.
20. The method of claim 1 , wherein detecting, by utilizing the one or more CNNs of the selected different plurality of CNNs associated with the particular base object model includes:
applying each CNN of the one or more CNNs to a respective portion of the image of the damaged vehicle corresponding to the respective surface segment area corresponding to the each CNN of the one or more CNNs, including overlaying a two-dimensional area surrounding the respective portion of the image of the damaged vehicle with the each CNN of the one or more CNNs.