Method and system for providing vehicle exterior damage determination service
A processor implemented method including outputting guide information to guide a capture of an image of a predetermined area via a camera of a mobile terminal including the processor, inputting a first exterior image of a vehicle, the first exterior image being captured based on the output guide information and a second exterior image of the vehicle stored in advance to a processor including a deep learning model, matching the first exterior image and the second exterior image with each other to acquire a matched image, masking a detected area from the predetermined image within the matched image as a masked area, and determining whether an exterior of the vehicle has been damaged and a type of damage based on the masked area.
1 . A processor-implemented method, the method comprising:
outputting guide information to guide an image to be captured of a predetermined area via a camera of a mobile terminal including a processor;
inputting a first exterior image of a vehicle, the first exterior image being captured based on the output guide information and a second exterior image of the vehicle stored in advance to a processor including a deep learning model;
matching the first exterior image and the second exterior image with each other to acquire a matched image;
masking a detected area from the predetermined image within the matched image as a masked area; and
determining whether an exterior of the vehicle has been damaged and a type of damage based on the masked area,
wherein the determining of whether the exterior of the vehicle has been damaged and the type of damage further comprises:
generating a probability map from the matched image via image segmentation;
determining whether the exterior of the vehicle has been damaged via the generated probability map; and
performing masking in pixel units for each type of damage to extract the masking result as damage characteristics information.
2 . The method of claim 1 , wherein the guide information includes one or more of information on whether the predetermined area is contained in a first area captured by the camera and information on whether the predetermined area is recognized and stored as the first exterior image.
3 . The method of claim 1 , wherein the first exterior image input to the processor includes an RGB image and a depth image.
4 . The method of claim 3 , wherein the camera includes a time of flight (ToF) camera configured to capture the depth image.
5 . The method of claim 1 , wherein the predetermined area is one of a second area with a high possibility of damage to the exterior of the vehicle and a third area where the exterior of the vehicle has been impacted acquired via a sensor of the vehicle.
6 . The method of claim 5 , wherein the second area includes a fourth area where a distance to an external object measured during travel of the vehicle is less than or equal to a reference distance.
7 . The method of claim 5 , wherein the sensor of the vehicle comprises one or more of a radar sensor, a bumper sensor, and a camera sensor.
8 . The method of claim 1 , wherein the determination of whether the exterior of the vehicle has been damaged and the type of damage include states of normal, scratched, dented, cracked, and opened.
9 . The method of claim 1 , wherein the deep learning model is further trained based on the damage characteristics information.
10 . The method of claim 1 , further comprising:
outputting the determination of whether the exterior of the vehicle has been damaged and the determined type of damage via an output device of the mobile terminal.
11 . The method of claim 10 , wherein the outputting comprises transmitting the determination of whether the exterior of the vehicle has been damaged and the determined type of damage via a wireless communication transceiver.
12 . An electronic system, the system comprising:
a camera mounted on a mobile terminal;
an output device configured to output guide information to guide a capture of a predetermined area via the camera; and
a processor configured to:
input a first exterior image of a vehicle captured based on the output guide information and a second exterior image of the vehicle stored in advance to a deep learning model;
match the first exterior image and the second exterior image to acquire a matched image;
masking a detected area based on a match between the predetermined area and the matched image as a masked area; and
determine whether an exterior of the vehicle has been damaged and a type of damage based on the masked area,
wherein the determining of whether the exterior of the vehicle has been damaged and the type of damage further comprises:
generating a probability map from the matched image via image segmentation;
determining whether the exterior of the vehicle has been damaged via the generated probability map; and
performing masking in pixel units for each type of damage to extract the masking result as damage characteristics information.
13 . The system of claim 12 , wherein the guide information includes one or more of information on whether the predetermined area is contained in a first area captured by the camera and information on whether the predetermined area is recognized and stored as the exterior image of the vehicle.
14 . The system of claim 12 , wherein the first exterior image input to the processor includes an RGB image and a depth image, and
wherein the camera includes a time of flight (ToF) camera configured to capture the depth image.
15 . The system of claim 12 , wherein the predetermined area is one of a second area with a high possibility of damage to the exterior of the vehicle and a third area where the exterior of the vehicle has been impacted acquired via a sensor of the vehicle.
16 . The system of claim 15 , wherein the second area includes a fourth area where a distance to an external object measured during travel of the vehicle is less than or equal to a reference distance.
17 . The system of claim 12 , wherein the determination of whether the exterior of the vehicle has been damaged and the type of damage include states of normal, scratched, dented, cracked, and opened.
18 . The system of claim 12 , wherein the deep learning model is trained based on the damage characteristics information.
19 . The system of claim 12 , further comprising:
a central management server configured to transmit the determination of whether the exterior of the vehicle has been damaged and the determined type of damage via a wireless communication transceiver.
20 . The system of claim 12 , wherein the output device is configured to output the determination of whether the exterior of the vehicle has been damaged and the determined type of damage.
21 . A processor-implemented method, the method comprising:
outputting guide information to direct a motion of a camera of a mobile terminal including the processor to capture an of a predetermined area via;
training a machine learning model on a first exterior image of a vehicle, the first exterior image being captured based on the output guide information and a second exterior image of the vehicle, the second image being of an undamaged version of the vehicle;
masking an area from the first image that matches the second image; and
determining whether an exterior of the vehicle has been damaged and a type of damage based on the masked area, wherein the determining of whether the exterior of the vehicle has been damaged and the type of damage further comprises:
generating a probability map from the matched image via image segmentation;
determining whether the exterior of the vehicle has been damaged via the generated probability map; and
performing masking in pixel units for each type of damage to extract the masking result as damage characteristics information.
22 . The method of claim 21 , wherein the guide information is output via one of a voice command or display images.