A custom W-LDED system (Klabs, South Korea) was used in this research. The system employed a continuous wave fiber laser (RFL-C1500, Raycus, China) with a wavelength of 1,080 nm and a maximum power of 1,500 W. The laser module was equipped with an irradiation head, allowing the laser beam to be projected in a straight line 3 mm in length, perpendicular to the printing direction. The operation and movement of the laser module, along with wire feeding, were controlled via Gcode. MGC-308 wire (Chosun Welding, diameter 1 mm, South Korea), commonly used for welding 304 stainless steel, The operation and movement of the laser module, along with wire feeding, were controlled via G-code. MGC-308 wire (Chosun Welding, diameter 1 mm, South Korea), commonly used for welding 304 stainless steel, was fed at a 45° angle from the front relative to the printing direction onto a stainless-steel substrate. The process parameters used in this experiment were a laser power of 1,140 W, a print speed of 20 mm/s, and a wire feed rate of 12 mm/s. The deposition width for bead formation was approximately 3 mm, corresponding to the laser beam diameter, while the height of each deposited layer was maintained at 0.25 mm. After each layer was deposited, the system incrementally moved 0.25 mm along the z-axis direction to prepare for the deposition of subsequent layer. The toolpath for each layer followed a straight trajectory with a length of 50 mm, producing consistent bead geometry. The dimensions and uniformity of the deposited beads were monitored to ensure repeatability under the given conditions.

To capture images of the W-LDED process, an off-axis high dynamic range (HDR) welding camera (XVC-1100e, Xiris, Canada) was installed to move in sync with the laser module. The camera was mounted to the laser module and positioned perpendicular to the printing direction, with a distance of 300 mm between the camera and the deposition area, to monitor the shape of melt pool and bead from a side view, recording at a resolution of 1,280 × 724 and 35 fps. To intuitively observe the visible light emitted from the heated region and distinguish the boundaries of the residual HAZ, the built-in lamp of the camera illuminated the deposited bead to ensure high-quality images even when the laser was not operating. The recorded images were captured frame by frame using Python code and used to create training data.

For the training of the U-Net model, a dataset was assembled, consisting of original images paired with corresponding annotated ground truth images. In this study, regions of interest (ROIs) that encompassed the bead, residual HAZ, and wire were extracted from the original 1,280 × 724 pixel images, and these ROIs were resized to 256 × 256 pixels. Ground truth images were prepared using the LabelMe annotation tool, where pixel-level manual classification was applied to distinguish between the two categories. The resulting annotations were saved in JSON format and later converted into single-channel label images, which served as the ground truth for training. The dataset comprised a total of 335 original and annotated image pairs, including both normal and defective states. From this dataset, 235 images were used for training, 50 for validation, and the remaining 50 for model evaluation. The training process was conducted in a Python 3.10 environment using PyTorch version 2.5.1, with computations performed on an Nvidia GTX 3070 Ti GPU equipped with 8 GB of GDDR6X memory, running on CUDA 11.8.

The U-Net model in this study operates on input images with a resolution of 256 × 256 pixels. The encoder (contracting path) consists of four downsampling layers, each applying a 3 × 3 convolution followed by an exponential linear unit (ELU) activation, and then a 2 × 2 max-pooling operation, halving the spatial dimensions at each step. As the spatial resolution decreases from 256 × 256 to 16 × 16 pixels, the number of feature channels doubles at each level, reaching 256 at the bottleneck. The decoder (expansive path) utilizes 2 × 2 transposed convolutions to upsample the feature maps, progressively restoring the resolution back to 256 × 256 pixels. Skip connections between corresponding layers in the encoder and decoder paths help recover fine-grained spatial information. The final layer employs a 1 × 1 convolution to produce the output segmentation map of size 256 × 256 × 3.

To assess the segmentation performance, several evaluation metrics were employed: IoU, mIoU, Recall, Precision, and F1 Score. These metrics were calculated for the two target classes, residual HAZ and dripping defects. The IoU for each class quantifies the overlap between the predicted and ground truth regions, calculated as the ratio of true positives (TP) to the sum of true positives, false positives (FP), and false negatives (FN):

During training, mIoU on the validation set was used to evaluate the model’s performance, ensuring checkpoints were saved at the highest-performing state for accurate segmentation of residual HAZ and dripping defects.

Recall, measuring the model’s ability to identify true positives, was calculated as:

Precision, evaluating the proportion of correctly predicted positives, was calculated as:

The F1 Score, which balances Recall and Precision, was defined as:

These four performance metrics were used to evaluate the segmentation accuracy on the test data.

Fig. 1(a) shows a conceptual schematic illustration of the proposed in-situ monitoring system, designed to maintain quality during the W-LDED process. A side-view camera captures the melt pool of the current deposition layer. As the laser focuses on and melts the wire feedstock, visible light is emitted from the heated area due to thermal radiation [15]. The visible light emitted during the process, as described by Wien’s displacement law, reflects the relationship between temperature and the peak wavelength of thermal radiation. Steel initially emits red light at relatively low temperatures, transitioning to orange and eventually to white as the temperature increases [16]. Typically, steel emits red light at temperatures around 500°C, while bright regions are associated with temperatures exceeding 1,100°C [17]. The color of thermal radiation may vary depending on the metal’s emissivity and surface condition. The residual HAZ refers to regions with temperatures above 500°C, and its size reflects the cooling rate during the process, with larger zones indicating slower cooling. Since the cooling rate is a critical factor in microstructure formation, achieving uniform cooling is important for ensuring consistent mechanical properties in the final product [18]. Therefore, the size and shape of the residual HAZ serve as indicators of process stability. By comparing these features across images, the system can reliably evaluate process performance.

Additionally, defects like dripping, primarily caused by improper process parameters such as an excessively high laser power, an inadequate wire feed rate, or an overly large distance between the wire tip and the substrate, occur when excessive melting of the wire leads to molten droplets detaching from the tip [19]. These detached droplets, which are typically spherical due to surface tension minimizing their energy state, accumulate on the substrate or surrounding areas, disrupting bead formation and compromising the uniformity of the deposition. Such defects can create irregularities that propagate through subsequent layers and degrade process stability, making it crucial to monitor them closely throughout the operation (Fig. 1(c)) [19]. To achieve this, the system employed a U-Net semantic segmentation model for pixel-wise classification of the residual HAZ and defects in the images, enabling the extraction of specific features, including their geometry, size, and area (Fig. 1(d)).

To develop an effective real-time monitoring system, it is essential to thoroughly characterize the defects occurring in the W-LDED process. The shape and size of these defects are related to the specific requirements for the monitoring system. Fig. 2 shows the printing process and defects caused by heat accumulation during the formation of the first layer and the deposition of subsequent layers. During the printing of the first layer, effective heat conduction to the substrate is achieved, resulting in a shorter length of the residual HAZ (Fig. 2(a)). However, as additional layers are deposited, heat conduction becomes less effective because the underlying material traps heat and limits its dissipation to the substrate. This leads to localized heat accumulation near the deposition area, slowing the cooling rate and enlarging the residual HAZ (Fig. 2(b)). Excessive heat accumulation reduces the cooling rate, prolonging the molten state of the material and leading to instability in the melt pool [20]. This instability amplifies minor irregularities in the previously deposited layers, causing geometric distortions that propagate through subsequent layers. Consequently, this cumulative effect results in uneven surface formation and compromises the overall stability and quality of the deposition process (Fig. 2(c)).

Fig. 3 shows the process of stable bead formation and the occurrence of dripping defects. The liquid bridge forms as the molten wire and melt pool merge under surface tension, facilitating continuous material transfer during the deposition process [21]. The stability of this bridge is critical for producing uniform and well-formed beads, as shown in Figs. 3(a) and 3(b). When the bridge becomes unstable, primarily caused by an excessively large distance between the deposition tool and substrate or an insufficient wire feed rate, it weakens and eventually breaks, causing the molten wire to form defective droplets (Fig 3(c)). Once these droplets exceed a certain size or touch the substrate, they result in dripping defects, potentially causing serious process failures (Fig. 3(d)).

Defects and uneven residual HAZs occurring during the W-LDED process hinder the production of uniform structures. Therefore, it is necessary to develop a system capable of addressing these issues, and a monitoring system utilizing semantic segmentation has been developed. Semantic segmentation is an image-based deep learning technique that segments various objects in an image at the pixel level, providing information about the size and location of each object. To date, various models such as FCN, U-Net, and DeepLab have been developed. Among them, U-Net is widely used across different fields due to its simple architecture, which allows for fast computation in image processing [22].

As the name suggests, U-Net has a U-shaped structure, with the encoder and decoder symmetrically connected (Fig. 4(a)). To train this model, both the original image and the ground truth image are required. For this, a labeling process was conducted as shown in Fig. 4(b), where the boundary information of the residual HAZ and dripping defect was saved and assigned the colors white and red, respectively. Once the training data were prepared, the original image is input into the encoder, where features are extracted by reducing the resolution and increasing the dimensionality through convolution and pooling processes. Then, in the decoder, these extracted features are used to restore the resolution using up-sampling and transposed convolution, generating the final segmentation map. However, low-resolution images may lack sufficient information for precise restoration. To address this, the model applies a skip connection technique, which transfers the information extracted at each stage of the encoder to the corresponding decoder stage at the same resolution, enabling more precise segmentation [23].

Through this process, the output image displays each pixel in the color corresponding to the class with the highest probability. The model weights are then adjusted to maximize the alignment between the output image and the ground truth image, and this process is repeated to create a model with optimized weights. Once the model is well-trained, new process images can be input, and the pixels corresponding to the residual HAZ and dripping defect will be accurately colored. Additionally, data augmentation and hyperparameter tuning were performed to achieve the best-performing model.

It is common to utilize data augmentation to improve the learning performance of U-Net [24]. As shown in Fig. 5(b), data augmentation primarily consists of physical transformations such as brightness adjustment, blurring, rotation, and flipping. These techniques increase the size of the training dataset, allowing for more efficient learning even with limited data. In this study, 285 images, excluding the test set, were augmented fivefold, resulting in 1,425 images for training and validation. Each augmentation technique was applied randomly to enhance dataset diversity. The augmented dataset was divided into training and validation sets at a 5:1 ratio to ensure a sufficient number of samples for evaluation. However, since the training data and real-time process images always have the same orientation, there were concerns that transformations like rotation and flipping might degrade the model’s performance. To test this hypothesis, an experiment compared augmented data with and without rotation and flipping.

The results of the model trained with rotation and flipping are shown in Fig. 5(a). As seen in the first and second images, the model struggled to distinguish between the high-energy flame and the residual HAZ. Similarly, in the case of the dripping defect, there were instances where part of the residual HAZ was misclassified, and only portions of the defect were detected. In contrast, when rotation and flipping were excluded from the data augmentation (Fig. 5(c)), the model performance to differentiate between the flame and residual HAZ improved, as did its accuracy in detecting the dripping defect.

Table 1 highlights the impact of data augmentation on model performance. Without data augmentation, all performance metrics declined, with precision particularly low, indicating an excessive number of false positive predictions. Excluding rotation and flipping from the augmentation process significantly improved most metrics, with precision showing the most substantial increase. This suggests that rotation and flipping increased prediction errors, reducing the model’s accuracy and generalization capability. While U-Net, originally developed for cell segmentation, typically benefits from augmentation techniques like rotation and flipping to recognize objects at various angles, these techniques appear to have hindered performance in this study. As mentioned above, this is likely because the orientation of the training data and process images remained consistent throughout, making such augmentations unnecessary and even detrimental to the accuracy of the model.

To find the optimal performance based on the prepared dataset, hyperparameter tuning was conducted. By adjusting the learning rate and batch size, trained models were obtained, and their mIoU, Recall, Precision, and F1 score values were calculated using test data to compare the results. First, experiments were carried out to determine the appropriate learning rate using a fixed batch size of 4 and the stochastic gradient descent (SGD) optimizer. It is generally accepted that a low learning rate slows down convergence to optimal weights, while a high learning rate increases the risk of overfitting [25]. Experiments were conducted using three learning rates: 0.1, 0.01, and 0.001, yielding results consistent with theoretical expectations. Figs. 6(a) and 6(b) show graphs of the mIoU for the validation data, organized by epoch. IoU values were calculated for each object, and the averages of these values were used to derive the mIoU.

With a learning rate of 0.001, the training process was excessively slow, leading to suboptimal results with an mIoU of 65.4% and a significantly low Precision of 44.5%. Conversely, a learning rate of 0.1 resulted in higher mIoU (79.2%) and F1 Score (87.9%), but its Recall (93.4%) was relatively lower, indicating a tendency to miss some true positives. In contrast, when the learning rate was set to 0.01, the model achieved a well-balanced performance across all metrics, with an mIoU of 77.8, Recall of 98.6, Precision of 74.0, and F1 Score of 87.2%. These results suggest that a learning rate of 0.01 provides the optimal balance between convergence speed and segmentation accuracy, making it the most suitable choice for the given dataset and task.

To evaluate the impact of optimization functions on model performance, batch size tuning experiments were conducted for both SGD and adaptive moment estimation (Adam) optimizers to determine their optimal configurations. SGD, a traditional optimizer, updates model parameters by computing the gradient of the loss function with respect to the parameters and applying the update rule [26]:

where θ_t represents the model parameters at iteration t, η is the learning rate, and ∇L(θ_t) is the gradient of the loss function. While SGD is known for its stability and consistency in simpler datasets, it may require careful learning rate tuning to avoid convergence issues.

Adam, on the other hand, combines the benefits of SGD with momentum and adaptive learning rates. It uses moving averages of the gradient (m_t) and squared gradient (v_t) to update the parameters:

where m_t and v_t are estimates of the first and second moments of the gradient, respectively, and ɛ is a small constant to avoid division by zero. Adam’s dynamic adjustment of learning rates allows for faster convergence, particularly in complex datasets, but may introduce variability in well-defined tasks [27].

Training was conducted under various batch size conditions, and the model’s performance was assessed using four key metrics: mIoU, Recall, Precision, and F1 Score (Table 3). SGD achieved its highest performance with a batch size of 4, yielding an mIoU of 77.8%, Recall of 98.6%, Precision of 74.0%, and an F1 Score of 87.2%. Adam performed best with a batch size of 16, achieving an mIoU of 72.8%, Recall of 91.5%, Precision of 70.5%, and an F1 Score of 84.4%.

The results highlight that SGD consistently outperformed Adam, particularly in mIoU and Recall, which are crucial for accurate segmentation. This superior performance can be attributed to the simplicity of the dataset, where each image contains only one of two distinct classes with significant shape differences. This structure allows SGD to achieve stable and efficient convergence, effectively capturing the underlying data distribution. While Adam’s dynamic learning rate adjustment is beneficial in more complex or noisy datasets, it may introduce unnecessary variability when applied to datasets with well-defined class distinctions and minimal noise. These findings emphasize the effectiveness of SGD for this specific segmentation task and dataset configuration.

As a final step to validate the suitability of U-Net for the proposed monitoring system, it was compared with Unet 3+, a segmentation model that builds upon the original U-Net architecture with advanced feature extraction and multi-scale feature fusion. Unet 3+ employs densely connected convolutional layers and adaptive decoders, which enhance its ability to capture both global context and fine-grained details [28]. These structural improvements enable Unet 3+ to achieve higher segmentation accuracy, particularly in complex regions like the residual HAZ and dripping defects, making it an appealing choice for precision-demanding applications.

Table 4 presents a detailed comparison of the two models in terms of segmentation performance and latency. Unet 3+ demonstrated a higher mIoU of 81.1, surpassing U-Net’s 77.8%. While this improved segmentation accuracy reflects Unet 3+’s advanced feature representation capabilities, it came at the cost of increased computational latency, with Unet 3+ requiring 7.68 ms per image compared to U-Net’s 2.69 ms. Considering the 35 fps used in this study, equivalent to approximately 28.6 milliseconds per frame, the 5 ms latency difference between the two models becomes significant, particularly when accounting for the additional computational load from the shape measurement algorithm and potential future modules for real-time process parameter adjustments. This latency gap could widen further depending on the computing hardware available, emphasizing the importance of model efficiency in real-time systems.

By refining the data augmentation strategy and adjusting the hyperparameters, optimal model performance was achieved, resulting in predicted images such as those shown in Figs. 7 and 8. Fig. 7 presents prediction results for test images containing the residual HAZ, with the predicted images ranked in descending order based on IoU scores. The model achieved an average IoU of 74.6% across 25 test images containing residual HAZ. In semantic segmentation, an IoU exceeding 50% is generally considered indicative of a well-trained model, suggesting that the model developed in this study achieves competitive performance.

As shown in Fig. 7, the model was able to accurately segment even complex areas such as the residual HAZ, which can be attributed to the optimal performance achieved through the selectively refined data augmentation strategy and hyperparameter tuning. Additionally, Fig. 8 shows prediction results for the dripping region, with an IoU of 81.0%. This indicates that the model can consistently detect the dripping area with high accuracy, even in varied shapes of defects. These results suggest that the model is not simply overfitting to specific process image data but maintains strong generalization performance with newly acquired data as well.

Edge detection is one of the most basic image segmentation techniques, identifying boundaries in areas where pixel values change abruptly [29]. While it might seem that this method could be used to detect the boundaries of the residual HAZ or dripping defect, it quickly becomes evident that edge detection alone is insufficient. The flame and residual HAZ that occur during the process share similar color regions, making it difficult to achieve accurate separation by relying solely on pixel value differences. Furthermore, shape-based distinctions, such as those in dripping, cannot be adequately captured with edge detection techniques. In contrast, U-Net, with its encoder-decoder architecture and skip connections, analyzes both global and local features, enabling accurate differentiation between visually similar regions, such as the flame and reflections on the substrate [30]. By integrating pixel-level and contextual analysis, U-Net goes beyond pixel-level analysis to deliver a detailed understanding of the residual HAZ and process conditions. Therefore, to achieve the precise results seen in Figs. 7 and 8, more advanced models like U-Net are required. U-Net, by learning the deeper features of an image, goes beyond pixel-level analysis to provide a more detailed understanding of the residual HAZ and process conditions. Additionally, U-Net is capable of quickly detecting unpredictable dripping defects in a real-time manner. These factors make U-Net-based models far more effective and reliable for process monitoring than the conventional edge detection methods.

By utilizing U-Net, the effective detection of residual HAZ and dripping defects in W-LDED process images has demonstrated the potential of process monitoring technology based on this approach. The ability to accurately capture the boundaries of each object in the process images has enabled more precise analysis. In this study, a pixel-based measurement algorithm was applied to extract detailed information about the shape of each feature from these predicted images.

The length of the residual HAZ varies depending on the laser power and the height of the deposition, making real-time length measurement crucial during the process, as it is a key factor affecting the quality and performance of the process. To address this, a length measurement algorithm based on pixel information was developed in this study. The process image shows that a 1 mm wire diameter consists of 8 pixels, meaning each pixel edge corresponds to 0.125 mm. As shown in Fig. 9(a), the number of pixels between the leftmost and rightmost pixels of the residual HAZ was measured, and this value was multiplied by the pixel edge length of 0.125 mm to calculate the length of the zone.

To eliminate noise caused by elements such as flames, the contour detection function provided by OpenCV was utilized. This function identifies the boundaries of connected regions in an image, enabling precise segmentation of target areas based on pixel characteristics. The algorithm was configured to only consider contours with more than 150 pixels, determined by counting the number of pixels within each contour. This threshold was determined by analyzing the actual residual HAZ sizes in the dataset. Contour detection was applied to all ground truth data, revealing that the smallest residual HAZ contained 178 pixels. To allow for a margin of error and ensure robustness, a threshold of 150 pixels was selected, classifying contours with fewer pixels as noise. This method effectively minimized errors caused by flame-related noise and ensured accurate length measurements.

A similar method was used to measure the area of the dripping defect (Fig. 9(b)). The total number of red pixels in the predicted image is counted, and the area of each pixel is multiplied to calculate the total area of the defect. As with the length measurement method, contour detection was also used to ensure robustness against noise by identifying and only considering regions above a certain size. In the case of the dripping defect, since it is larger than any residual HAZ, this algorithm was directly reflected in the measurement. The prediction of the image and the execution of the morphology measurement algorithm have a combined latency of approximately 5.0±0.8ms per image, ensuring timely feedback. This measurement approach can play an important role in the development of a feedback system. For instance, when a dripping defect occurs, the system can immediately measure its area and provide feedback to prevent the defect from reaching the bed.

An experiment was also conducted to verify whether the developed shape analysis method could accurately measure the length of the residual HAZ. The experiment involved depositing a total of 31 layers of beads under two conditions: one with constant laser power and another with a reduction in laser power every 10 layers. This setup was used to evaluate the ability of the system to detect changes in the length of the residual HAZ during the process. Under the constant laser power condition, the residual HAZ showed a length of approximately 11.9 mm (Fig. 10(a)). As deposition continued to 21 layers, the length of the residual HAZ increased to 14.5 mm, and by 31 layers, it reached 18.4 mm, indicating a gradual increase in length. In contrast, when the laser power was reduced by 100 W every 10 layers, the length of the residual HAZ remained consistent at around 13 mm throughout the process.

In addition, a comparison of the output quality revealed that the uneven beads were formed when the laser power was kept constant (Figs. 11(a)–11(c)). In contrast, when the laser power was gradually reduced, a uniform residual HAZ was formed, and the height of the deposited beads remained consistent. These results demonstrate that adjusting laser power is a key factor in improving the precision and consistency of the process. Furthermore, the cooling time of molten metal would affect the formation of the microstructure, which could influence the mechanical properties. In the field of DED processes, previous studies have reported microstructural changes caused by differences in cooling rates across layer heights [31,32]. It has been demonstrated that rapid cooling in the lower layers results in fine and uniform microstructures, while slower cooling rates lead to coarser microstructures [33]. These microstructural variations are known to directly influence material properties, including mechanical characteristics such as hardness, strength, ductility, and residual stress [34]. Therefore, forming a uniform residual HAZ is crucial for ensuring consistent mechanical performance throughout the product. The monitoring system developed in this study enables real-time detection of variations in the residual HAZ, facilitating the observation of changes under different process parameters. Although this experiment applied a simple method of reducing laser power by 100 W at each stage, future research could leverage this monitoring system to develop an automated control system. A future system will optimize process parameters based on the measured residual HAZ length and dripping defect size. Additionally, to address the limitations of continuously reducing laser power for taller structures, this system could incorporate additional parameters, such as laser scanning speed, into a comprehensive control framework. By dynamically balancing heat input, this approach would ensure consistent bead quality, stable process conditions, and enhanced overall product performance.