Review on Machine Learning Based Welding Quality Improvement
Article information
Abstract
Artificial intelligence technology is rapidly developing with the improvement of computer performance and the development of various algorithms, and research using artificial intelligence technology is being actively conducted in the field of manufacturing technology. In the field of welding, research on arc welding quality prediction using artificial neural network algorithms (ANN) was mainly conducted in the early stages. Since then, in the case of arc welding quality prediction using a deep neural network (DNN) algorithm, research has been conducted to increase the accuracy by increasing the hidden layer in the ANN algorithm. Recently, many studies have been conducted in the form of predicting and classifying the quality of arc welding based on the convolutional neural network (CNN) algorithm, which is one of the deep learning algorithms. Therefore, in this paper, we review representative algorithms such as ANN, DNN, and CNN applied to the welding field, and introduce studies that have successfully performed bead width prediction, welding quality prediction, and quality classification.
1 Introduction
Artificial intelligence (AI) can be defined as imitating human thinking and calculation methods and creating value after thinking and judging on its own using a neural network that allows computers to solve problems. This is an early form of artificial neural network proposed by Frank Rosenblatt in 1957, which derives a single result from multiple inputs, and is similar to how neurons constituting a real brain transmit signals [1,2]. Machine learning and deep learning fall under these large categories of artificial intelligence. Machine learning, which Samuel et al. first used in 1959 [3], refers to an algorithm that automatically learns without the user specifying all the rules by using a matrix for prediction or classification purposes with the development of artificial intelligence. For example, artificial intelligence technology makes it possible to classify blurry images and predict results. Machine learning has been studied since the 1970s, but development has been slow for a while due to limitations in computer performance and a lack of technology. Since the late 1990s, remarkable advances in related technologies have stimulated the field of research.
In addition, deep learning is being studied to supplement the limitations of existing machine learning [4]. This is a field of machine learning, which is a method of learning data by stacking two or more hidden layers of artificial neural networks, and is being used in many fields with the recent development of graphics processing units (GPU). Since Yann Lecun [5] created the LeNet-5 neural network model that recognizes handwritten digits in 1998, many deep learning models have appeared and are being compared and analyzed in various forms. For example, human image classification evaluation results are known to be 95% accurate, but we are starting to demonstrate relevance, winning the ImageNet 2013 competition with 97.2% using the VGG-16 model.[6]
For this reason, artificial intelligence-related technologies are being applied in various ways in the field of computer vision. It is also actively expanding into traditional manufacturing technologies such as welding and machining. In particular, in the field of manufacturing technology, there are a lot of unskilled workers due to high wages and aging, so active research is being conducted in the direction of developing technologies for unskilled workers to easily find the optimal conditions for the process or to prevent mistakes during the process.
In the field of welding, research on arc welding quality prediction using artificial neural network algorithms was mainly conducted in the early stages. The reason is that the ANN algorithm was developed first, and the weight used in the model was low, so it is assumed that the approach was easy. Since then, in the case of arc welding quality prediction using a deep neural network algorithm, research has been conducted to increase the accuracy by increasing the hidden layer in the ANN algorithm. Recently, many studies have been conducted in the form of predicting and classifying the quality of arc welding based on the convolutional neural network algorithm, which is one of the deep learning algorithms. Fig. 1 shows the difference between ANN and DNN).
Based on this, this paper reviews three major algorithms, such as ANN, DNN, and CNN, which are applied to the welding field, and introduces studies that have successfully performed bead width prediction, welding quality prediction, and quality classification.
2 Defect Classification for Welding Quality
Welding is a method of joining structural members. Compared to mechanical joining methods, welding has been adopted as a joining method of structure members for a long time because of its simplicity and excellent airtightness [8]. However, during welding, the structure itself is rapidly heated locally and then cooled again, making it susceptible to deformation and residual stress remaining, which can lead to poor welding or defects. Therefore, identifying and resolving welding defects significantly impacts welding performance and life [9].
The types of welding defects can be largely divided into dimensional defects, structural defects, and performance defects [10]. Dimensional defects are defects caused by changes in volume and stress in the process of solidifying metal after welding, such as extreme steel bars, concave beads, undercuts, and poor penetration. In the case of over-welding, other internal defects may occur, and in the case of undercutting and insufficient penetration, notches may occur around the welded area, resulting in serious damage or reduced fatigue life.
Structural defects are the most easily encountered defects that occur during welding in an atmosphere or inert gas environment [11]. Among them, pores generated by gases while welding is fatal to material properties, and high-temperature cracks due to solidification of high-temperature rapid welding and low-temperature cracks due to residual gas of high-strength steel can harm the stability of the entire structure.
These defects can be prevented by selecting the correct current, voltage, welding material, electrode position, angle, etc., but too low a welding current and transition speed cause over-welding and must be fully considered [12].
In addition to defects such as dimensional and structural defects, it also includes performance defects such as lack of notch toughness, lack of tensile strength, and excessive hardness. These defects must be taken into account as they may affect the basic performance of the material.
Therefore, in this paper, previous studies that improved welding quality were reviewed by examining machine learning-based neural networks that partially predict welding defects that occur under unsuitable welding conditions and compare their performance.
3 Machine Learning Algorithms for Welding Quality
3.1 Artificial Neural Networks
An Artificial Neural Network (ANN) is a weighted directed graph first devised by Warren McCulloch [13] in 1943. ANNs are composed of multiple layers and contain numerous nodes that mimic biological neurons in the human brain, such as Axons, Synapses, and Dendrites. It has evolved from the simplest artificial neural network proposed by Frank Rosenblatt in 1957 to a deep neural network (DNN) and Convolution neural network (CNN) [14].
Choudhury et al. [15] proposed microstructure evaluation of welding, parametric analysis, and ANN modeling. The results of the ANN-teach-learning-based optimization (ANN-TLBO) integrated modeling approach to optimize welding strength are as follows. a) Compared to multiple regression analysis (MRA), the ANN model confirmed 91.53% accuracy. b) As a result of parametric analysis, welding current was the most dominant factor, followed by welding speed. High heat input at low welding speed increased the penetration, improving the maximum tensile strength of welding. The proposed integrated ANN-TLBO optimization approach was optimal welding parameters I = 120 A, V = 180 mm/min, GFR (gas flow rate) = 121/min, and N = 2.24 mm, with a maximum tensile strength of 701.43 MPa.
Jacques et al. [16] proposed an ANN-based model to predict welding shapes and dimensions in laser-contact welding of galvanized steel. The model was constructed based on the fusion data provided by 3D finite element analysis. The possible relationship between welding parameters such as laser power, welding speed, diameter, and gap and the geometric properties of welding, especially penetration depth and bead width, was analyzed, and sensitivity to welding conditions was evaluated using relevant statistical tools. As a result, it was verified that the model has excellent performance and that the shape and dimensions of laser welding can be effectively predicted with an average prediction error of less than 10%.
Li et al. [17] developed an ANN model to predict bead shapes under welding conditions in which welding speed changes continuously. Experiments on several structures to optimize the hyperparameters of the model showed that the structure with 33-6-2 layers was the most suitable, and the result of the bead shape prediction model using it showed little deviation between the predicted value and the actual value. The highest error is 0.899 mm and the average bead width error is 0.223 mm. In addition, the highest error in bead reinforcement was 0.534 mm and the average error was 0.09 mm. It explained that this was caused by a lack of training data and an error due to the environment, and added that this precision is relatively acceptable in the welding process.
Thekkuden et al. [18] developed a voltage-based feed-forward backpropagation network for immediate inspection of V-groove joint welding quality by conducting bead-on-plate test welding using a gas metal arc welding process. Among the classification accuracy obtained using a total of nine techniques, the highest accuracy of ANN was 94.7%, and the voltage data corresponding to Bid-on-plate and V-groove welding were selected by Train and Test, respectively. The accuracy of Train, Validation, and Test was 92.3%, 100%, and 100%; the overall classification accuracy achieved was 94.7%.
Kamla Babu et al. [19] performed ANN modeling through Genetic Algorithm (GA) optimization for maximum tensile strength, microhardness, and corrosion resistance of AA2219 alloys. ANN established the relationship by providing L9 orthogonal array experimental input and output response, and optimization using GA was carried out in the experimental results. In the Friction Stir Wedding (FSW) joint results consisting of ANN-GA modeling optimization and feasible solutions, the Batch Back Propagation (BBP) model was measured at a minimum Root Mean Square Deviation (RMSE) value of 0.0089991 when seven hidden Layers were used.
Paven Kumar et al. [20] said that the process parameter is the penetration depth in the Cold Metal Transfer (CMT) pulse welding process using ANN. The effect on the weld pool width and bead height was investigated, and from experimental data, the neural network model developed in this study was used to control the process parameters to achieve the desired bead shape. In addition, it was observed that a correlation coefficient of 0.999 was obtained between the experimental results and the developed Back Propagation Network (BPN) model, and it was confirmed that the error was small for the desired bead shape. This confirmed that the developed neural network model can predict the weld bead dimensions with high accuracy.
He et al. [21] performed a pulsed tungsten inert gas (TIG) welding-brazing process to bond with aluminum-stainless steel dissimilar metals. The main parameters that have a large effect on the bonding characteristics are pulse peak current, base current, pulse on time, frequency, etc., which were verified through pre-experiments. The samples were set according to the central composite design and based on the samples, Response Surface Methodology (RSM) and ANN were used to separately predict the tensile strength of the joints. A mathematical model for joint strength prediction was derived using RSM, in which an algorithm consisting of one input layer with four neurons, one hidden layer with eight neurons, and one output layer with one neuron predicted strength, and compared to RSM, ANN's average state prediction error was about 10%, stable and more accurate.
Hamid Reza Fazli Shahri et al [22] predicted the Heat Affected Zone (HAZ) and temperature distribution of samples using ANN and GA. The HAZ of the material is prone to numerous mechanical defects such as residual stress and microcracks through a thermal process, and the temperature distribution of the sample is considered a key indicator that can determine the HAZ and Fusion Zone (FZ) regions. The model introduced in this paper optimized the three characteristics of backpropagation algorithms: number of neurons, learning volume, and learning speed in hidden layers using GA and developed to address the shortcomings of artificial neural networks. The combined model of GA-ANN was applicable to establish the relationship between HAZ and temperature distribution and three process parameters. As a result, it was confirmed that the experimental data and the prediction result were well-matched. We conclude that this model can predict physical and metallurgical results such as temperature distribution and HAZ for laser curing processes as well as other Gaussian heat source heat treatment processes such as EDM and laser welding.
Anand et al. [23] describe the use of a central composite design matrix to conduct experiments on friction welding of Incoloy 800 H. Tensile strength and burn-off length were measured using RSM and ANN, and the response was optimized and predicted by applying RSM first. After that, independent variables such as Heating Pressure (HP), Heating Time (HT), Upsetting Pressure (UP), and Upsetting Time (UT) were supplied as inputs to ANN, and the output of the network was tensile strength and burn-off length. A quick propagation algorithm was used to create different combinations of input/output patterns that further train multi-layer feed-forward networks and compared the results obtained through the RSM and ANN models to analyze which methods were more specific. Comparing the predictive capabilities of the two methodologies, it was determined that ANN (0.98031) provided a solution close to the actual value compared to RSM (2.167).
Olabi et al. [24] uses the L9 Taguchi approach and artificial neural networks to optimize the CO2 butt welding process in the range of parameters investigated. They proposed Taguchi analysis as a solution to neural networks that only work within the range of parameters. Taguchi analysis allowed us to fill the available experimental data’s space and calculate the optimal penetration depth, fusion area width, and heat-affected area width. As a result, it was explained that the analysis of the data was performed quickly and intuitively, and the optimization process worked reasonably. They described that there was no problem investigating other keyhole laser welding applications.
Chaki [25] compared 65 different artificial neural network architectures to estimate the tensile strength of hybrid CO2 laser metal inert gas (MIG) welded aluminum alloy plates. Sensitivity analysis was performed to study the effect of process parameters on tensile strength using Backpropagation Neural Networks (BPNNs), Raial basis function networks (RBFNs), and 65 ANN architectures. Based on the results, the following conclusions were drawn. (1) Bayesian regularization (BR) training and testing showed that the BPNN 3-1-1 network had the best prediction performance among various architectures with a welding strength prediction MSE of 3.24E-04. The 3-1-1 network trained using BPNN with batesian regularization (BR) once again confirmed the superiority of predictive ability with an average absolute error of 1.7%, a maximum error of 3.2%, and a minimum error of 0.3%. (2) As a result of the sensitivity analysis of the process, it was found that the welding strength increased as the laser power increased and the welding speed decreased as the welding speed increased. Therefore, welding speed and laser output were found to be the factors that had the greatest influence.
Chen et al. [26] applied the automatic detection and classification method of welding defects in the galvanized steel laser welding process. The authors compared ANN with the Support vector machine (SVM) and described that experiments showed that SVM and ANN were able to efficiently detect and classify defects such as voids, top bead depression, and bead separation. In SVM, the prediction rate of 48-dimensional data (98.08%) was higher than that of 16 dimensions (92.73%), and the prediction rate of ANN (97.27%) was higher than that of SVM (92.73%). Although the ANN algorithm performed well in terms of prediction rates because similar results were shown in cross-validation, we explained that the SVM has excellent generalization capabilities and is easy to adjust the parameters.
Sathiya et al. [27] performed laser beam welding to bond AISI 904L Super Austenitic Stainless Steel with three different shielding gases (Argon, Helium, Nitrogen). Laser welded butt joints were performed using a three-level full factorial design with three factors (beam output, moving speed and focal position) and three environments (argon, helium and nitrogen). Through this, welding bead shapes such as penetration depth, bead width, and tensile strength of joints were estimated, and the developed ANN model was used in conjunction with optimization algorithms such as GA to optimize welding parameters. Laser welding using optimized welding parameters showed better quality of joints, and the good agreement between theoretically predicted GA and experimentally obtained tensile strength, penetration depth, and bead width confirmed the applicability of calculation techniques to optimize parameters in the welding process.
Patil and Reddy [28] proposed an independent technique for detecting and classifying multi-type welding bonding defects, including cracks, undercuts, gas pores, pores, slags, wormholes, lack of penetration, and non-failure, in X-ray images through artificial neural networks and support vector machines. To implement this technology, a brightness histogram was built through image processing, boundaries were clarified through edge operation, and major analysis factors were selected to quickly detect and classify welding defects using ANN and SVM, respectively.
3.2 Deep Neural Networks
DNN is a neural network developed to solve problems in the ANN technique and is a learning method with two or more hidden layers in the model. DNN, also called deep learning, is a subfield of machine learning and has been continuously studied since 2010 due to the performance improvements of PCs and the spread of GPUs. As a result of the application of technology, it is actively expanded and applied to industrial sites because it is a learning method that can derive performance similar to or above people's level.
Oh et al. [29] conducted an eradication welding experiment to develop a bead width prediction model for the Gas Tungsten Arc (GTA) welding process, and analyzed the correlation between real-time process variables and bead width prediction models using the data obtained. The results are as follows. A total of 27 GTA welding experiments were conducted for each welding condition according to the complete factor experiment method, and current and voltage data were extracted in real-time using the welding monitoring system during the welding experiment. The welding quality was verified in real-time based on the extracted data, and a 3D scanner was used to check the trend of bead width according to the current-voltage data. To develop a DNN algorithm using a neural network based on the experimental results measured in the welding experiment, it was decided to minimize the error by setting the number of neurons in the hidden layer through a neural network structure. In addition, to verify the reliability of the DNN algorithm, the bead width obtained from the actual welding experiment was compared and analyzed, and the accuracy was evaluated using Predictive Ability of Model (PAM). As a result, it was confirmed that the bead width of the DNN algorithm was 100% predicted, and the exact bead width was predicted when the DNN algorithm was used.
Yang et al. [30] extracted 11 welding defect characteristics and proposed four factors based on the background and strong contrast. We also implement a cross-validation strategy, randomly dividing the entire dataset into five subsets, and then dividing 220 samples in each cross-validation interval into 176 Train data and 44 Test data. After applying it to the DNN algorithm using, the train set and test set were constructed by the Stacked Auto-Encoder (SAE). Finally, for comparison, welding defects were classified into the same dataset using SVM and DNN models. As a result, we confirm that SAE and Data augmentation via microrotation can achieve superior performance over existing models even with small datasets. This demonstrates superior performance with a 3.18% and 4.33% improvement in classification accuracy compared to regular DNN models, reaching 91.36%, and achieves good performance in welding defect classification.
Günther et al. [31] proposed an architecture for laser welding. It goes through the process of data observation, training, and control, which explains the combination of reinforcement learning and deep learning algorithms, and deep learning was used for the first time in laser welding and industrial production processes. The study was developed in line with laser welding and confirmed that DNN can derive meaningful features from laser welding settings, which can be used to review and control current process conditions. Furthermore, experiments with SVM demonstrated that DNN outperforms Principal Components Analysis (PCA). This structure learned how to predict photodiode data, and the learned behavior demonstrated the validity of the DNN function. It was also experimentally verified that adding a sensor does not increase the difficulty of calculation. Research shows that 52.6 seconds is used to simulate a laser welding system, whereas the overall simulation takes 66.3 seconds. Therefore, it is confirmed that single management takes 0.34 ms, which meets the process requirements for real-time process capability (1 ms).
Jin and Rhee [32] proposed a new DNN-based welding monitoring system and developed a LabVIEW software-based signal measurement, analysis, judgment, and control system by synchronizing robots and welders. Welding current and voltage signals were analyzed in the time and frequency domain, respectively, and a total of 24 feature variables were derived, and the proposed DNN model was used as input to train the model. In addition, signal preprocessing software has been developed to extract the corresponding feature variable in real-time. Notable results from the study are as follows. a) The proposed DNN-based gap detection model was trained based on feature variables extracted from welding current and voltage signals, showing a training accuracy of about 94.3%. b) When the verification results were performed with data not included in the training data offline and online, 93.7% and 87.7% accuracy were obtained, respectively. c) Based on the real-time welding gap detection results using this, the welding robot was controlled, and uniform external welding beads were achieved.
Shin et al. [33] developed a system to detect and predict porous defects occurring in the welding area using galvanized high-tension steel plates used in automobile parts in the GMA welding process. Through this, the following results were obtained. a) The welding current and arc voltage signals generated in the GMA welding process were measured in real-time, and characteristic variables were extracted through pretreatment. In addition, by analyzing the correlation between the shape variable and the porosity, a shape variable considered to be a defect signal was selected. b) Artificial intelligence techniques bonded to nonlinear arc welding process prediction were used, and a model was developed to detect and predict porosity by comparing DNN and ANN. c) We evaluated the predictive performance of ANN models and DNN models. The evaluation results show that the predictive performance of the DNN model is 15.2% higher on average than that of the ANN model. d) As a result of field application, all defects in the welding area were detected.
3.3 Convolution Neural Networks
Convolutional Neural Network (CNN) is a technique derived from DNN and is mainly used for image or image processing. For DNNs that use data in essentially one-dimensional form, training time and performance suffer because spatial and local information is lost within the image. Therefore, detection and classification are performed using the main information of the input image. Fig. 2 shows the general architecture of CNN.
Kumaresan et al. [35] proposed object detector-based casting defect detection for aluminum components used in automobiles. The object detector can classify defects and 14 types of defects, and the proposed defect classifier was used in object detectors by adjusting the key elements of existing histograms of oriented gradients (HOG) linear and SVM detectors. Key elements include a) image pyramid configuration, b) ROI extraction via sliding windows, and c) non-maxima suppression. The image begins with welding seam radiography. The sliding window is then run on images of all scales, ROI is detected, then passed through the weld defect classifier model, and the label is predicted after defect identification. Finally, the process ends when Non-maxima suppression deletes objects with low prediction accuracy and then visualizes radiographs. However, it may not match the performance of deep learning-based object detectors such as Fast R-CNN and You only look once (YOLO), because running a sliding window on multi-scale image data can slow the entire process for real-time detection, and end-to-end training is not possible.
Nomura et al. [36] predicted or estimated the welding quality by the CNN model, including changes in the difference between tapers or steps, using a molten pool image obtained by a camera during MAG (metal active gas) welding. The input image is a top image containing a molten pool obtained using a bandpass filter of 980 nm every 0.1 seconds during welding. As a result of the study, a) a burn-through prediction model was built with a classification problem to predict excessive penetration and burn-through in advance, but there was a problem that the prediction accuracy was poor because it was treated as a discontinuous phenomenon. Therefore, as a result of configuring the penetration depth estimation model as a regression problem, more than 95% were confirmed to have a 1 mm error, and more than 87% had a 0.5 mm error. Although this was an excellent estimation result, it was found that the estimation accuracy varies locally with the quality of the training data. On the other hand, when examining the effect of the image size used in the input image, it is confirmed that the estimation accuracy does not change significantly even if the image size is very different. As a result, it was experimentally verified that the calculation time was short enough to be used as a real-time monitoring tool.
Guo et al. [37] proposed a new defect detection method for resistance welding spots, down-sampling the size of the data image, and designating the number of classes with the help of experts. The loss function is cross-entropy and the optimization algorithm is ADADELTA (An Adaptive Learning Rate Method). Added Dropout layer to prevent overfitting. To find the optimal hyperparameters, each layer was experimentally identified, and as a result of comparing 5HL-CNN, 7HL-CNN, and 13HL-CNN, 13HL-CNN showed the best performance and the time per Epoch was not large at 5.58 s and 1.02 s, respectively. Defect detection of resistance welding spots using 13HL-CNN showed 99.01% performance.
4. Conclusions
This paper introduced research papers using ANN, DNN, and CNN in welding and summarized on Table 1. As shown in Table 1, there are several previous studies using machine learning using various welding methods. However, it is difficult to suggest which algorithm is the best because the method used in individual research results, the number of data, and the method of (pre/post) processing data are varied with each research.
The welding process is a major process technology as it is used throughout the industry. The effective and accurate combination of artificial intelligence with these process technologies is considered essential and attracts the attention of many researchers. Most of the artificial intelligence used in the welding process showed better welding results by finding parameters between processes. Although the judgment and parameters of the results varied on paper, it is worth noting that welding technology has developed when artificial intelligence is used in the welding process. Welding processes are being used in numerous places, such as automobiles, aircraft, ships, and architecture, but due to the nature of the process, they are classified as difficult and difficult jobs, so the inflow of new workers is low. Therefore, welding technology applied to the industry continues to expand automation or welding process technology using robots. To this end, various studies are continuously being conducted, and studies using relatively recent techniques, DNN and CNN, as well as traditional ANN technology, will continue to develop in the future.
Acknowledgement(s)
This study was conducted with the support of the Ministry of Trade, Industry, and Energy (MOTIE) and the Korea Energy and Technology Evaluation Institute (KETEP) (No. 20206410100080).
List of Symbols
AI
Artificial Intelligence
ANN
Artificial Neural Network
ANN-TLBO
ANN-Teach-Learning-Based Optimization
BBP
Batch Back Propagation
BPNNs
Backpropagation Neural Networks
CNN
Convolutional Neural Network
DNN
Deep Neural Network
FSW
Friction Stir Wedding
GA
Genetic Algorithm
GMA
Gas Mungsten Arc
GPU
Graphics Processing Units
GTA
Gas Tungsten Arc
HAZ
Heat Affected Zone
MIG
Metal Inert Gas
PAM
Predictive Ability of Model
PCA
Principal Components Analysis
RBFNs
Raial Basis Function Networks
RMSE
Root Mean Square Deviation
RSM
Response Surface Methodology
SVM
Support Vector Machine
TIG
Tungsten Inert Gas
References
Biography
Ik-Su Kim Master Student in the Department of Mechanical Engineering, Ajou University. His research interest is deep learning on novel manufacturing processes. E-mail: hiiksu@ajou.ac.kr
Moon-Gu Lee He received his B.S. degree in precision engineering and his M.S. and Ph.D. degrees in mechanical engineering from the Korea Advanced Institute of Science and Technology, Korea, in 1995, 1997, and 2003, respectively. He later became a Senior Researcher for Samsung Electronics and a Visiting Associate Professor at Harvard Medical School. He is currently a Professor in the Division of Mechanical Engineering, Ajou University, Suwon, Korea. His research interests include analysis, design, and control of high precision positioning systems and biomedical devices. E-mail: moongulee@ajou.ac.kr
Yongho Jeon He received his B.S. degrees in mechanical engineering from Ajou University, South Korea and the Illinois Institute of Technology, USA, in 2003. He received his M.S. and Ph.D. degrees from the University of Wisconsin, Madison, Wisconsin, USA, in 2005 and 2008, respectively. He later became a manufacturing researcher for Hyundai Motors. He is currently a professor in the Department of Mechanical Engineering at Ajou University. His research interests are developing novel manufacturing processes using heat and vibration and the effect of material characteristics. E-mail: princaps@ajou.ac.kr