In-situ inspection method based on digital data model of weld
A method inspects weld quality in-situ. The method obtains a plurality of sequenced images of an in-progress welding process and generates a multi-dimensional data input based on the plurality of sequenced images and/or one or more weld process control parameters. The parameters may include: (i) shield gas flow rate, temperature, and pressure; (ii) voltage, amperage, wire feed rate and temperature (if applicable); (iii) part preheat/inter-pass temperature; and (iv) part and weld torch relative velocity). The method generates defect probability and analytics information by applying one or more computer vision techniques on the multi-dimensional data input. The analytics information includes predictive insights on quality features of the in-progress welding process. The method then generates a 3-D visualization of one or more as-welded regions, based on the analytics information, and the plurality of sequenced images. The 3-D visualization displays the quality features for virtual inspection and/or for determining weld quality.
1. A method for in-situ inspection of weld quality, the method comprising:
obtaining a plurality of sequenced images of an in-progress welding process, wherein the plurality of sequenced images captures temporal patterns related to characteristics of a weld pool for an entire weld event for the welding process forming an as-welded region;
generating a multi-dimensional data input based on the plurality of sequenced images and one or more weld process control parameters, wherein generating the multi-dimensional data input includes (i) detecting features and patterns from the plurality of sequenced images, (ii) assembling data across the entire weld event, (iii) assigning significance to the patterns, (iv) integrating the one or more weld process control parameters, and (v) extracting features of statistical significance, wherein the patterns include one or more of: a shape of a weld pool, a size of the weld pool, a contour of the weld pool, a depth of the weld pool, a thermal gradient of the weld pool, a change over time of the weld pool, and a uniformity of the weld pool, wherein assembling data across the entire weld event comprises applying temporal cross-correlations to align the plurality of sequenced images to geometry, and wherein generating the multi-dimensional data input comprises converting temporal image trends to stationary signals by taking a temporal derivative of the plurality of sequenced images;
generating defect probability and analytics information by applying one or more computer vision techniques on the multi-dimensional data input, wherein the analytics information includes predictive insights on quality features of the in-progress welding process, wherein the quality features include a type of weld defect and a location of the weld defect, wherein the one or more computer vision techniques includes applying a convolutional neural network with 3D convolutions on sequential, lagged batches of the plurality of sequenced images, wherein the 3D convolutions consider pixel position, intensity, and color/spectral band; and
generating a 3-D visualization of the as-welded region, based on the analytics information and the plurality of sequenced images, wherein the 3-D visualization displays the quality features for virtual inspection of the as-welded region.
2. The method of claim 1 , wherein the one or more computer vision techniques includes one or more trained machine learning algorithms trained to identify anomalies or defects in an in-progress welding process based on image sequences.
3. The method of claim 2 , wherein the one or more trained machine learning algorithms includes one or more trained unsupervised anomaly detection algorithms trained, on images of welds that passed a quality criterion, to identify defective welds based on image sequences.
4. The method of claim 3 , wherein the one or more trained machine learning algorithms includes one or more trained supervised anomaly detection algorithms trained, on images of classified defective welds that failed a quality criterion, to identify defective welds based on image sequences.
5. The method of claim 2 , wherein the one or more trained machine learning algorithms includes one or more trained supervised anomaly detection algorithms trained, on images of classified defective welds that failed a quality criterion, to identify defective welds based on image sequences.
6. The method of claim 2 , wherein the multi-dimensional data input comprises a multi-dimensional array representing pixel intensity and color, and the one or more trained machine learning algorithms include a convolutional neural network (CNN) trained to identify boundaries of weld pool shape, speed, spatter, rate of change, and/or welding parameters, for determining weld qualities, defects, and/or one or more characterizations of the in-progress weld process, based on the multi-dimensional array.
7. The method of claim 6 , wherein the convolutional neural network identifies boundaries of weld pool shape by identifying contours using thresholding or edge detection methods.
8. The method of claim 1 , further comprising:
in accordance with a determination that the quality features of the in-progress welding process does not satisfy a predetermined quality criterion:
causing the in-progress welding process to cease; and
generating a warning of one or more events of the in-progress welding process based on the analytics information.
9. The method of claim 1 , wherein weld process control parameters include one or more of:
shield gas flow rate, temperature, and pressure;
voltage, amperage, wire feed rate and optionally temperature;
part preheat/inter-pass temperature; and
part and weld torch relative velocity.
10. The method of claim 1 , wherein the one or more computer vision techniques includes a temporal convolutional neural network that is trained with sliding window and gated activation functions to learn anomalies across time series data points.
11. The method of claim 1 , wherein generating the 3-D visualization of the one or more as-welded regions comprises annotating the 3-D visualization with a quality assessment of a weld defect.
12. The method of claim 11 , wherein the annotated quality assessment includes one or more of size of the weld defect, shape of the weld defect, extent of the weld defect, depth of the weld defect, and type of the weld defect.
13. The method of claim 1 , wherein generating the 3-D visualization of the one or more as-welded regions comprises providing a visualization of normalized gradient of weights to indicate defect characteristics.
14. The method of claim 13 , wherein the weights are indicated in time along the plurality of sequenced images, to locate the weld defect in time.
15. The method of claim 13 , wherein the weights indicate part of an image that is different in terms of intensity, shape, or spectral hue.
16. The method of claim 1 , wherein the one or more computer vision techniques includes one or more machine learning algorithms trained using laser profile progression by projecting laser light onto a post-weld surface to obtain a 3-D profile of the weld surface and voids to infer weld quality.
17. The method of claim 16 , wherein the laser profile progression is performed at an angle to a welding plane.
18. The method of claim 16 , wherein the laser profile progression is performed perpendicular to a welding plane.
19. The method of claim 1 , wherein the virtual inspection of the as-welded region is used for determining weld quality of the as-welded region.
20. The method of claim 1 , wherein the patterns further include one or more of: an intensity pattern within the weld pool, a spatter of the weld pool, and an alignment of the weld pool.