Deep convolutional neural networks for crack detection from image data
View Patent ↗A method includes detecting at least one region of interest in a frame of image data. One or more patches of interest are detected in the frame of image data based on detecting the at least one region of interest. A model including a deep convolutional neural network is applied to the one or more patches of interest. Post-processing of a result of applying the model is performed to produce a post-processing result for the one or more patches of interest. A visual indication of a classification of defects in a structure is output based on the result of the post-processing.
1. A method comprising:
detecting at least one region of interest in a frame of image data;
detecting one or more patches of interest in the frame of image data based on detecting the at least one region of interest;
applying a model comprising a deep convolutional neural network to the one or more patches of interest;
performing post-processing of a result of applying the model to produce a post-processing result for the one or more patches of interest; and
outputting a visual indication of a classification of defects in a structure based on the result of the post-processing, wherein the classification distinguishes between normal edges of the structure and cracks of the structure.
2. The method of claim 1 , wherein detecting the one or more patches of interest comprises applying a threshold on a percentage of pixels with edges in a given patch.
3. The method of claim 1 , wherein the post-processing comprises aggregating classifications from each of the one or more patches and smoothing the classifications to identify dominant classifications.
4. The method of claim 1 , wherein the visual indication comprises a classification heat map overlaid upon the image data to highlight location and severity of the defects.
5. The method of claim 1 , wherein the method is performed in part using cloud computing resources.
6. The method of claim 1 , wherein the image data is received from a boroscope camera.
7. The method of claim 1 , wherein the model is trained using a plurality of image frames comprising a plurality of defects labeled on a patch or pixel basis.
8. The method of claim 1 , wherein the image data comprises at least one channel per frame.
9. The method of claim 1 , wherein the deep convolutional neural network comprises a plurality of pairs of convolution layers and pooling layers, and at least one of the convolution layers comprises a plurality of kernels with a pixel stride and pixel edge padding.
10. The method of claim 9 , wherein the deep convolutional neural network comprises three of the pairs of convolution layers and pooling layers, and a third pooling layer of the pairs is connected to a soft-max layer configured to provide a defect classification value for each of the one or more patches of interest in the frame.
11. A system comprising:
a camera or a database of images; and
a processing system operable to:
detect at least one region of interest in a frame of image data from the camera or the database of images;
detect one or more patches of interest in the frame of image data based on detecting the at least one region of interest;
apply a model comprising a deep convolutional neural network to the one or more patches of interest;
perform post-processing of a result of applying the model to produce a post-processing result for the one or more patches of interest; and
output a visual indication of a classification of defects in a structure based on the result of the post-processing, wherein the classification distinguishes between normal edges of the structure and cracks of the structure.
12. The system of claim 11 , wherein detection of the one or more patches of interest comprises application of a threshold on a percentage of pixels with edges in a given patch.
13. The system of claim 11 , wherein the post-processing comprises aggregation of classifications from each of the one or more patches and smoothing the classifications to identify dominant classifications.
14. The system of claim 11 , wherein the visual indication comprises a classification heat map overlaid upon the image data to highlight location and severity of the defects.
15. The system of claim 11 , wherein the processing system interfaces with cloud computing resources to perform a portion of the processing.
16. The system of claim 11 , wherein the camera is a boroscope camera.
17. The system of claim 11 , wherein the model is trained using a plurality of image frames comprising a plurality of defects labeled on a patch or pixel basis.
18. The system of claim 11 , wherein the image data comprises at least one channel per frame.
19. The system of claim 11 , wherein the deep convolutional neural network comprises a plurality of pairs of convolution layers and pooling layers, and at least one of the convolution layers comprises a plurality of kernels with a pixel stride and pixel edge padding.
20. The system of claim 19 , wherein the deep convolutional neural network comprises three of the pairs of convolution layers and pooling layers, and a third pooling layer of the pairs is connected to a soft-max layer configured to provide a defect classification value for each of the one or more patches of interest in the frame.