Systems, methods and programs for generating damage print in a vehicle
The disclosure relates to systems, methods and computer readable media for providing network-based identification, generation and management of a unique damage (finger) print of vehicle(s) by geodetic mapping of stable key points onto a ground truth 3D model of the vehicle, and vehicle parts—identified from the raw images using supervised and unsupervised machine learning. Specifically, the disclosure relates to System and methods for the generation of unique damage print on a vehicle that is obtained from captured images of the damaged vehicle, photogrammetrically localized to a specific vehicle part, and the computer programs enabling the method, the damage print configured to be used, for example, in fraud detection in insurance claims.
1. A computerized networked system for providing fraud alarm comprising:
a. a user access terminal;
b. a vehicle;
c. a vehicle database;
d. a damage print database; and
e. a backend management server (BMS), comprising a network communication module configured to communicate with the communication network, and a central processing module (CPM), in communication with the communication module, the vehicle database and the damage print database, the CPM further comprises at least one processor in communication with a non-transitory memory storage device, storing thereon a processor readable media with a set of executable instructions, configured, when executed to cause the at least one processor to:
i. receive a plurality of images, each image associated with the vehicle damage from the user, each image taken at a different distance and angle relative to the damage;
ii. Identify at least one damaged part in the vehicle;
iii. using the plurality of images, generate a plurality of image pairs;
iv. for each image pair, estimate the area of vehicle covered in each image in the pair;
v. using fragment matching, match a first image to the second image in the pair for all image pairs determined to have at least one image in the image pair, having a sufficient vehicle area coverage, wherein the image that has sufficient vehicle area coverage is a pair's main image and the second image in the pair is a pair's fragment image;
vi. create a matched pairs group;
vii. add the matched image pairs to the matched pairs group;
viii. using at leat one image in at least one image pair, identify the vehicle;
ix. retrieve a canonical 3D model rendering of the identified vehicle from the vehicle database;
x. georeference the main image in the matched image pair, for each image pairs in the matched pairs group to the retrieved canonical 3D model;
xi. isolate the damaged parts from the 3D canonical model;
xii. compare the images to the isolated damaged parts obtained from the 3D canonical model;
xiii. detect a plurality of points associated with surface anomalies on the images;
xiv. georeference the fragment image in the matched image pair, for each image parts in the matched pairs group;
xv. using photogrammetry, georeference the damage on the identified damaged vehicle part;
xvi. register the georeferenced damage to the 3D canonical model of the identified vehicle;
xvii. generate a unique damage fingerprint of the damaged part; and
xviii. if the damage fingerprint generated matches a unique damage print stored on the damage print database, generate a fraud alarm; else
xix. store the generated unique damage print in the damage prints database.
2. The system of claim 1 , wherein to identify the vehicle using at least one image in at least one image pair, the set of executable instruction, is further configured, when executed, to cause the at least one processor to: identify at least one of: make, model, trim, year and principal color.
3. The system of claim 1 , wherein of the set of executable instructions is further configured, prior to using at least one image in at least one image pair, identify the vehicle, to cause the at least one processor to:
a. determine the presence of a vehicle in the image; and
b. if there is no vehicle in the image, generate an alert; else
c. identify the vehicle.
4. The system of claim 1 , wherein to georeference, for each of the main image in the matched image pair, and each fragment image in the matched image pair, the set of executable instruction, is further configured, when executed, to cause the at least one processor to:
a. extract a plurality of stable key points of the anomalies on the at least one image;
b. reference each stable key point on a two-dimensional (2D) coordinate system;
c. using the canonical 3D model rendering, calculate geodetic coordinates of each stable key point; and
d. geospatially position each stable key point onto the at least one damaged part.
5. The system of claim 4 , wherein to register the georeferenced damage to a 3D model of the identified vehicle, the set of executable instruction, is further configured, when executed, to cause the at least one processor to:
a. for each image captured, using the canonical 3D identified vehicle model, obtain at least one of exterior orientation parameter, and interior orientation parameter of the at least one damaged part's image;
b. using the at least one of exterior orientation parameter, and interior orientation parameter, perform ortho-rectification;
c. detect and position the plurality of the stable key points in the orthophoto coordinate system;
d. using the canonical 3D model rendering, calculate geodetic coordinates of each stable key point; and
e. geospatially position each stable key point onto the at least one damaged part.
6. The system of claim 5 , wherein the seto of executable instructions are further configured, when executed by the at least one processor to:
a. generate a list of damaged parts; and
b. store the list of damaged parts and the unique damage print associated with that damaged part in the damage print database.
7. The system of claim 4 , wherein to extract the plurality of stable key points of the anomalies on the at least one image, the set of executable instructions is further configured, when executed by the at least one processor to:
a. extract a scale invariant feature transform (SIFT) feature of the at least one image; and
b. using difference of Gaussian (DoG) scale-space over at least two scales, extract the plurality of stable key points by, thereby obtaining the convolution of the original image.
8. The system of claim 7 , wherein the set of executable instructions is further configured, when executed by the at least one processor to eliminate edge points.