Image processing systems for detecting vehicle damage
An image processing system analyzes each of a set of vehicle images to determine if there is any damage to the vehicle depicted in each or any of the images. The image processing system uses a characterization engine in the form of a neural network based image model to process each of the pixels of each of the selected tagged images to determine the particular pixels of the image (or of the object depicted within the image) that depict the presence of damage to the object, and the likelihood of the pixels depicting damage. The characterization engine or image model may be developed or trained using a training engine that analyzes a plurality of images of different vehicles damaged in various different manners which have been annotated, on a pixel by pixel basis, to indicate which pixels of each image represent damaged areas of the objects and which have also been annotated, on an image basis, to indicate the view and/or zoom level of the image.
1 . A method of detecting damage to a vehicle, comprising:
obtaining a plurality of electronic images of a plurality of vehicles having damage;
marking each electronic image of the plurality of electronic images, via a computer processing device, to indicate, for each pixel of a first set of pixels of the each electronic image of the plurality of electronic images illustrating damage to a respective vehicle depicted within the each electronic image, a respective damage type of a plurality of damage types and a respective segment of a plurality of predetermined vehicle segments, each vehicle segment corresponding to a different part of an undamaged vehicle, to create a set of marked vehicle images;
developing, using a computer processing device, a regression model from the plurality of marked vehicle images, the regression model trained to detect vehicle damage in an image on a pixel by pixel basis;
obtaining at least one electronic image of a further vehicle;
applying, using a computer processing device, the regression model to the at least one electronic image of the further vehicle to determine, for each pixel of a second set of pixels of the at least one electronic image of the further vehicle, a measure of a probability of damage to the further vehicle at locations of the vehicle associated with the each pixel of the second set of pixels, the applying of the regression model to the at least one electronic image of the further vehicle including providing, to the regression model, at least one of: a first input indicative of a respective view of each electronic image of the at least one electronic image, or a second input indicative of a respective zoom level of the each electronic image of the at least one electronic image, including determining a predicted damage type and a predicted vehicle segment at the locations of the vehicle associated with the each pixel of the second set of pixels; and
producing, using a computer processing device, a heat map of the further vehicle illustrating areas of detected damage to the further vehicle.
2 . The method of claim 1 , wherein the regression model is a neural network model.
3 . The method of claim 1 , wherein the regression model is a convolutional neural network model.
4 . The method of claim 3 , wherein developing the regression model includes selecting a size and a shape for a pixel area surrounding a pixel being processed to use as inputs to the regression model.
5 . The method of claim 1 , wherein developing the regression model includes using a plurality of electronic images of a plurality of different vehicles including different damage.
6 . The method of claim 1 , wherein developing the regression model includes using a plurality of electronic images of a plurality of different vehicles including different types of damage.
7 . The method of claim 1 , wherein developing the regression model includes using a plurality of electronic images of a plurality of different vehicles including damage at different locations on the different vehicles.
8 . The method of claim 1 , wherein developing the regression model includes using a plurality of electronic images of a plurality of different vehicles of different years, makes, models or trims, including damage.
9 . The method of claim 1 , further including using the heat map to determine one or more repair activities to be applied to the further vehicle to correct damage to the further vehicle.
10 . The method of claim 1 , further comprising marking the each electronic image of the plurality of electronic images with at least one of: an indication of a respective view of the respective vehicle within the each electronic image of the plurality of electronic images, or an indication of a level of zoom of the each electronic image of the plurality of electronic images.
11 . An image processing system for use in detecting damage to a vehicle, comprising:
one or more tangible, non-transitory memories that store a plurality of electronic images of a plurality of vehicles including damage, wherein each electronic image of the plurality of electronic images is marked to indicate, for each pixel of a first set of pixels of the each electronic image of the plurality of electronic images at which damage to a respective vehicle depicted in the each electronic image is present, a respective damage type of a plurality of damage types and a respective segment of a plurality of predetermined vehicle segments, each vehicle segment corresponding to a different part of an undamaged vehicle;
a model generation routine, for execution on a computer processor, that creates a regression model from the plurality of marked vehicle images stored in the one or more tangible, non-transitory memories, the regression model trained to detect vehicle damage in an image on a pixel by pixel basis;
at least one electronic image of a further vehicle, the at least one electronic image of the further vehicle stored on the one or more tangible, non-transitory memories;
a damage detection routine, for execution on a computer processor, that analyzes the at least one electronic image of the further vehicle using the regression model to determine, for each pixel of a second set of pixels of the at least one electronic image of the further vehicle, a measure of a probability of damage to the further vehicle at locations of the vehicle associated with the each pixel of the second set of pixels, the using of the regression model including providing, to the regression model, at least one of: a first input indicative of a respective view of the further vehicle depicted within each electronic image of the at least one electronic image, or a second input indicative of a respective zoom level of the each electronic image of the at least one electronic image, the damage detection routine further determining a predicted damage type and a predicted vehicle segment at the locations of the vehicle associated with the each pixel of the second set of pixels; and
a heat map routine, for execution on a computer processor, that develops a heat map of the further vehicle illustrating locations of the further vehicle at which damage has been detected.
12 . The image processing system of claim 11 , wherein the regression model is a neural network model.
13 . The image processing system of claim 11 , wherein the model generation routine develops the regression model using a plurality of electronic images of a plurality of different vehicles including different types of damage.
14 . The image processing system of claim 11 , wherein the model generation routine develops the regression model using a plurality of electronic images of a plurality of different vehicles including damage at different locations on the different vehicles.
15 . The image processing system of claim 10 , wherein:
the each electronic image of the at least one electronic image of the further vehicle includes at least one of: first information about the respective view of the further vehicle depicted within the each electronic image of the further vehicle, or second information about the respective zoom level of the each electronic image of the further vehicle; and
at least one of: the first input to the regression model is based on the first information or the second input to the regression model is based on the second information.
16 . An image processing system for detecting damage to a vehicle, comprising:
a memory that stores one or more electronic images of the vehicle;
a characterization engine having an image model, wherein the characterization engine executes on a computer processor to process each electronic image of the one or more electronic images of the vehicle using the image model to detect, on a pixel by pixel basis, if there is damage to the vehicle at a corresponding location of the vehicle depicted in the each electronic image of the one or more electronic images of the vehicle, wherein the image model is a regression model trained on a plurality of marked vehicle images in which each pixel, of the plurality of marked vehicle images, corresponding to damage to a respective vehicle depicted within each vehicle image of the marked vehicle images is marked with an indication of a respective damage type of a plurality of damage types and a respective segment of a plurality of predetermined vehicle segments, each vehicle segment corresponding to a different part of an undamaged vehicle, and wherein the using of the image model includes providing, to the image model, at least one of: a first input indicative of a respective view of the vehicle depicted within the each electronic image of the one or more electronic images of the vehicle, or a second input indicative of a respective zoom level of the each electronic image, and wherein using the characterization engine further includes determining a predicted damage type and a predicted vehicle segment at the corresponding location of the vehicle depicted in the each electronic image of the one or more electronic images of the vehicle; and
one or more tangible, non-transitory memories that store an image of the vehicle indicating, for each pixel of a set of pixels of the image, if damage exists on the vehicle at the each pixel.
17 . The image processing system of claim 16 , wherein the characterization engine is a neural network engine and wherein the image model includes neural network weights developed using the plurality of marked vehicle images.
18 . The image processing system of claim 16 , wherein the characterization engine applies, at each pixel of a multiplicity of pixels within the one or more images of the vehicle, an input template that specifies a set of pixels around a pixel being analyzed to use as inputs to the image model.
19 . The image processing system of claim 16 , wherein at least one of:
the each electronic image of the one or more electronic images of the vehicle includes information about the respective view of the vehicle depicted within the each electronic image of the vehicle, and the first input to the regression model is based on the information about the respective view; or
the each electronic image includes information about the respective zoom level of the each electronic image, and the second input to the regression model is based on the information about the respective zoom level.
20 . The image processing system of claim 16 , wherein the characterization engine determines, using the image model, a probability of damage at each pixel of a set of pixels in the one or more of the electronic images of the vehicle and uses the determined probability of damage at the each pixel of the set of pixels to determine if damage to the vehicle exists at the each pixel of the set of pixels.