Deep learning-based system and method for automatically determining degree of damage to each area of vehicle
The present invention relates to a deep-learning based system and method of automatically determining a degree of damage to each area of a vehicle, which is capable of quickly calculating a consistent and reliable quote for vehicle repair by analyzing an image of a vehicle in an accident by using a deep learning-based Mark R-CNN framework and then extracting a component image corresponding to a damaged part, and automatically determining the degree of damage in the extracted component image based on a pre-trained model.
1. A system for automatically determining a degree of damage for each vehicle area based on deep learning, the system comprising:
a model generating unit which compares a plurality of vehicle photographed images obtained by photographing a state of the vehicle before an accident repair and a plurality of vehicle photographed images obtained by photographing a state of the vehicle after the accident repair, masks each component with a different color, learns subdivided data for the vehicle component for a bumper, a door, a fender, a trunk, and a hold based on the masked area, inspects and re-labels damage degree labelling data based on a result of a comparison between the damage degree labelling data for each damage type for the vehicle photographed image and a reference value, and learns data obtained by determining the degree of damage for each damage type of a plurality of damage area photographed images to generate a model;
an image pre-processing unit which performs correction-processing on a vehicle photographed image obtained from a user terminal based on the model generated through the model generating unit;
a component recognizing unit which recognizes and subdivides the correction-processed vehicle photographed image for each vehicle component based on the model generated through the model generating unit;
a damaged area image extracting unit which extracts an image of a damaged vehicle component in the vehicle photographed image; and
a damage degree determining unit which determines a degree of damage of the corresponding damaged area according to a predetermined state based on the extracted damaged area image and the model generated through the model generating unit.
2. The system of claim 1 , wherein the image pre-processing unit performs correction-processing of augmenting an image by rotating or reversing the vehicle photographed image, or performs correction-processing of removing light reflection on the vehicle photographed image.
3. The system of claim 1 , wherein the component recognizing unit recognizes and subdivides vehicle components for a bumper, a door, a fender, a trunk, and a hood in the vehicle photographed image by using a Mask R-CNN framework.
4. The system of claim 1 , wherein the damage degree determining unit determines whether the degree of damage of the damaged area corresponds to any one of a normal state, a scratch state, a small-damage plate work required state, a medium-damage plate work required state, a large-damage plate work required state, and an exchange state by using an Inception V4 network structure of the CNN framework.
5. The system of claim 3 , wherein the component recognizing unit masks an area for the bumper, the door, the fender, the trunk, and the hood by using the Mask R-CNN framework, and masks a wider area than an outer line of each area so as to cover a boundary of adjacent areas.
6. The system of claim 1 , further comprising:
an estimated repair quote providing unit which calculates an estimated repair quote based on the degree of damage for the damaged area image and provides the user terminal with the calculated estimated repair quote.
7. A method of automatically determining a degree of damage for each vehicle area based on deep learning, the method comprising:
comparing, by a model generating unit, a plurality of vehicle photographed images obtained by photographing a state of the vehicle before an accident repair and a plurality of vehicle photographed images obtained by photographing a state of the vehicle after the accident repair, masking each component with a different color, learning subdivided data for the vehicle component for a bumper, a door, a fender, a trunk, and a hold based on the masked area, inspecting and re-labelling damage degree labelling data based on a result of a comparison between the damage degree labelling data for each damage type for the vehicle photographed image and a reference value, and learning data obtained by determining the degree of damage for each damage type of a plurality of damage area photographed images to generate a model;
performing, by an image pre-processing unit, correction-processing on a vehicle photographed image obtained from a user terminal based on the model generated through the model generating unit;
recognizing and subdividing, by a component recognizing unit, the correction-processed vehicle photographed image for each vehicle component based on the model generated through the model generating unit;
extracting, by a damaged area image extracting unit, an image of a damaged vehicle component in the vehicle photographed image; and
determining, by a damage degree determining unit, a degree of damage of the corresponding damaged area according to a predetermined state based on the extracted damaged area image and the model generated through the model generating unit.
8. The method of claim 7 , further comprising:
calculating, by an estimated repair quote providing unit, an estimated repair quote based on the degree of damage for the damaged area image and providing the user terminal with the calculated estimated repair quote.