Enhancing Inductive IR Thermography by Using FFT-Equalization, Motion Tracking Detection and VDSR Super-resolution Processing
Article information
Abstract
Among non-destructive inspection techniques, infrared thermography stands out as a promising technology that enables real-time inspection of large areas without the need for physical contact. In this study, we employed the dynamic induction thermography method, which is one of the active infrared thermography techniques, to detect defects on the back side of the S275 material specimen. This technique involves creating relative movement between the IR camera and the specimen. We acquired sequence images at different moving speeds using the induction thermography technique, and then used the FFT with Gaussian filtering to solve for non-uniform heat sources. To further enhance the resolution, we applied the VDSR technique, which is based on deep neural networks. The effectiveness of this approach was validated both qualitatively and quantitatively. Finally, we utilized the MOT algorithm to automatically detect defects in the image with the highest thermal contrast, which was captured at a speed of 15 mm/s. In this study, we demonstrate the effectiveness of thermal equalization using the GF-based FFT algorithm, as well as the super-resolution conversion achieved through the VDSR-based deep neural network. Additionally, we present a mechanism for automated slit detection using the MOT algorithm.
1 Introduction
Steel is one of the most essential materials in modern industry. It is a material that is widely used in aerospace, automobiles, ships, power plants, cultural properties, buildings, sports, and medical care. However, discontinuities (defects, cracks, peeling, voids, corrosion, etc.) occur during the service period due to various causes which affect durability and reliability.
It is essential to detect discontinuities on the back or inside before material fracture occurs. These problems can be solved by using non-destructive testing (NDT) techniques. The types of NDT techniques are very diverse, so the infrared thermography (IRT) techniques were applied in this study. The IRT is a technique that measures the thermal response of an inspection object using the infrared (IR) camera [1–3]. The IRT techniques can generally be classified into passive and active and are distinguished by the presence or absence of a heat source [4–6].
In this study, the induction thermography (IT) technique among the active IRT techniques was used to detect the slits on the back side of S275 specimens. The induction thermography (IT) technique generally utilizes a static mode in which all devices are stationary. However, in this study, a dynamic mode in which relative motion occurs between the IR camera and the specimen was applied [7,8]. When the static mode is used, the shape of the coil is recorded in the thermal image and is affected when image processing is performed [9,10]. Therefore, when recording with an IR camera, a sequence image can be acquired based on the scanning line, so only the image of the specimen can be acquired.
In the case of the optical technique, which is the most widely used among IRT techniques, radiation energy must be absorbed on the surface by the flash lamp or laser to generate heat [11,12]. In this case, it is greatly influenced by the absorptivity and emissivity of the surface [13]. However, in the IT technique, heat is generated at internal discontinuities, so uniform heat diffusion occurs.
The thermal characteristics of the defect were analyzed by applying various conditions to the thermal image acquired using the IT technique using the dynamic mode [14–16]. The degree of influence on the thermal contrast of the defective area and the sound area was analyzed, and the thermal trend for each profile was analyzed.
If there is foreign matter on the specimen or the emissivity is not uniform, the heat concentration characteristic can be confirmed. Heat flux, or temperature distribution, is affected by clutter, not by a temperature scale itself, reducing the thermal contrast between the defective area and the sound area. Therefore, fast fourier transform (FFT) with gaussian filtering (GF) was applied to remove thermal-noise and equalize. However, smoothing was also applied around the defect area, resulting in a decrease in resolution. Therefore, resolution was improved using deep learning techniques.
The thermal image is data that records the thermal response generated from the external heat source. Since the image is captured during the process of thermal equilibrium (thermal diffusion) rather than measuring the moment when the external heat source reaches the specimen, it is difficult to clearly identify the shape of the defect. Therefore, it is essential to improve the resolution to clearly detect defects.
Based on the deep learning technique, very-deep super-resolution (VDSR) was used to improve the resolution of thermal images [17–19]. Using the VDSR neural network, it is possible to enhance low-resolution images to high-resolution images. The VDSR is a convolutional neural network architecture designed to perform single-image super-resolution reconstruction. The VDSR neural network is a technique for learning mapping between low-resolution and high-resolution images. Using a residual learning strategy, it learns to estimate residual images. Here, the residual image means the difference between the low-resolution image and the high-resolution reference image that are upscaled using bicubic interpolation to match the size of the reference image.
The motion object tracking (MOT) algorithm was used to detect defects in images to which VDSR was applied. The MOT algorithm converts the image into a binary image based on the blob algorithm and then detects objects with a scale of 1 [20–22]. The detection rate of defects for each frame was analyzed.
This paper is organized as follows: In section 2, the theoretical contents of the joule’s heat generated at the discontinuity point were described using the IT technique. In addition, the process of acquiring sequence images using dynamic mode was presented. In section 3, the performance of the experimental device used in this paper is described, and the dimensions of the S275 specimen are presented. In section 4, after acquiring sequence images using the IT technique, the thermal trend was analyzed. The FFT was performed to remove thermal-noise, the image resolution was improved through the VDSR deep neural network, and qualitative and quantitative evaluations were performed. Next, the MOT algorithm was used to detect defects in the image to which VDSR was applied. Section 5 represents the conclusion of this paper and future research.
2 Detection Theory of Induction Thermography
2.1 Principle of IT
In the IT technique, an eddy current is electrically induced in the conductor, and the current moves inside the surface of the material [23–25]. Due to the ohmic resistance of the material, local joule’s heating is generated. The induced eddy current is determined by the penetration depth. It is called the skin depth Eq. (1) and follows as [16,26]:
Where δ is the penetration depth, σ is the electrical conductivity, μ is the permeability, and f is the excitation frequency. The skin depth equation is used to calculate the lift-off of the copper coil and specimen. The excitation frequency, f, and the penetration depth, δ, have an inverse relationship, so lift-off must be set considering the set excitation frequency. Where can be calculated as:
Where μ0 means the permeability magnetic field constant in a vacuum state, the constant value is 4π × 10−7 [H/m]. μr is the relative magnetic permeability, and means a scale by which the degree of magnetization of the material can be compared. Magnetic flux density, which is the magnetic field induced in a space filled with a material such as an iron core in a coil wound with wire, varies depending on the material.
When the discontinuity exists inside the conductor, it can be explained through heat transfer caused by joule’s heating. The heat distribution Eq. (3) inside the conductor is followed as:
Where ρ is the density, cp is the heat capacity, λ is the thermal conductivity, and q is the heating power. The heat due to joule’s heating, q, can be expressed as:
Where J is the current density, E is the electrical field vector. Fig. 1 shows the principle that when a current flows through the copper coil, the magnetic field is formed around it and the current is induced in the magnetic material.
The principle of the magnetic field generated by the coil of the induction thermography
2.2 Theory of GF-based FFT
The GF-based FFT algorithm was performed to remove thermal-noise located in the center of all thermal images. GF is a smoothing filtering technique used in image processing. GF has the characteristic of being attenuated as it moves away from the center. Therefore, after converting to the frequency domain with FFT, the high-frequency component can be attenuated to solve the pattern of heat concentration in the sound area. The Eq. (5) for GF is as follows.
Where H(u,v) is the frequency domain of GF, (u,v) is coordinate of frequency domain, u0 and v0 are center position of GF, σu and σv are standard deviation of gaussian distribution.
Where F(u,v) is the image with the FFT applied to thermal image F(x,y) and G(u,v) is the image of frequency domain. And then, if the inverse transform is performed on G(u,v) again, it is as follows.
2.3 Principle of VDSR Algorithm
The resolution improvement of the 2D thermal image was performed using the VDSR deep neural network. The VDSR algorithm is a technology that converts low-resolution images into high-resolution images using deep learning. It primarily utilizes residual learning and deep neural network structures to improve image details. A low-resolution image is accepted as input, the residual of the high-resolution image is predicted, and the predicted residual is added to the low-resolution image to finally obtain a high-resolution image.
Fig. 2 shows the principle of learning the VDSR neural network. The process of the VDSR architecture is as follows. First, the low resolution (LR) image is interpolated and then input to the network. After down-sampling the image by the scale factor using bicubic, ILR is generated again with bicubic. Second, the ILR images are learned from Conv and ReLU layers from level 1 to 20. Finally, the ILR image is added to the output, and the HR image is acquired.
The structure of the VDSR deep neural network
The performances of the computer used in this study are as follows. AMD Ryzen 5 7600 (6 core), RAM 32 GB, NVIDIA GeForce RTX 4070 Ti. The pre-trained VDSR model is provided in MATLAB 2023a’s Computer Vision Toolbox and was used in this study to improve the resolution of thermal images. The VDSR was pre-trained with a benchmark data set of 20,000 images.
3 Experimental Setup
3.1 Description of the Laboratory Setup
Fig. 3 shows the configuration of the experimental equipment. The experimental setup consists of the induction generator, cooling water circulator, copper coil, and IR camera. In the induction generator, the current intensity was set to 160 A, and the excitation frequency was set to 40 kHz. The moving speed of the specimen is 5, 7, 9, 11, 13, and 15 mm/s, and it moves along the sliding guide rail at a constant speed. The induction generator is cooled by the circulator’s water. The copper coil was manufactured in the ‘U’ type, and the liftoff distance from the specimen was set to 1 mm. The IR camera was FLIR’s SC645 model (un-cooled, 640 × 480 pixels, 7.5–13 μm, 30 Hz). The distance between the IR camera and the specimen was 700 mm. The images measured by the IR camera were saved using FLIR software, and post-processing of the images was performed using MATLAB software.
Diagram of experimental setup for induction thermography composed of line scanning method
3.2 Steel Specimen
In Fig. 4(a) shows the specimen made of steel, and Fig. 4(b) shows the dimensions. There are a total of 6 artificial slits. Line 1 has the same width but a different depth, and Line 2 has the same depth but a different width. The total thickness of the specimen was 10 mm, and it was manufactured in a size of 150 × 120 mm. The respective depths of defects A, B, and C in Line 1 are 7, 5, and 3 mm. The depths of defects A, B, and C are all 5 mm. Table 1 shows the material properties of steel.
Steel specimen with 6 artificial slits
Thermal-electrical properties of steel specimen
4 Results of Inductive Thermography
4.1 Acquiring 2D Thermal Image and Trend Analysis
To acquire the thermal image of the steel specimen for each moving speed, a sequence image was acquired based on the scanning line. Fig. 5 shows the principle of acquiring sequence images. The process of acquiring a thermal image is as follows. First, a thermal image of the moving specimen is acquired with the IR camera for every frame. Second, convert all thermal images to Excel data. Third, one thermal image is acquired by scaling the vertical pixels for all frames based on the reference axis. If the sequence principle is used, the coil shape is not measured in the thermal image, so there is no interference with the temperature analysis and post-processing.
Fig. 6 shows the thermal image for each moving speed. Precise slit detection is also possible visually (qualitatively). In the case of Line 1, the deeper the defect, the higher the maximum temperature. In the case of Line 2, it can be seen that the wider the slit, the higher the maximum temperature. In addition, the higher the moving speed of the specimen, the higher the temperature can be confirmed.
Thermal-sequence images for each moving speed
Fig. 7 shows the thermal increase trend for each moving speed. The -axis distance was set as a normalized distance. The thermal trend of 1 line in Fig. 6 is difficult to analyze qualitatively. Fig. 7(a) shows the clear trend of temperature increase. The tendency of the temperature to gradually change according to the shape of the slit can be confirmed.
Thermal profile graph of 1 and 2 Lines for normalized distance
4.2 Thermal-noise Removal Using GF-based FFT
Fig. 8 shows the image with the GF-based FFT algorithm applied. When compared to Fig. 6, it can be seen that thermal-noise is relatively significantly reduced in the central area. Here, if the standard deviation value of the GF is further increased, thermal-noise can be removed more effectively. However, locally the boundaries of the defect area become blurred. This reduces the overall image resolution. Therefore, low-resolution images were converted to high resolution using the VDSR algorithm to improve resolution.
Image with FFT applied to remove thermal-noise
4.3 Resolution Improvement Using VDSR
Fig. 9 shows images with improved resolution using the VDSR neural networks. Qualitatively (visually), It can be confirmed that the resolution of the image to which the VDSR is applied is better than that of raw images. In particular, in the VDSR image, it can be seen that the blurring around the defect is noticeably removed. However, the criteria for qualitative comparison are not clear. Therefore, in this study, a quantitative comparison was performed by calculating the peak signal to noise (PSNR) and the naturalness image quality evaluator (NIQE).
Resolution-improved images with the VDSR neural network applied
Table 2 shows quantitatively compared data, and the PSNR and the NIQE of the bicubic and the VDSR images were analyzed. The PSNR is one of the indicators for measuring video quality. The quality of the image is evaluated by evaluating the difference between the reference image and the transformed image. Here, the reference model means a thermal raw image. The Eq. (8) of PSNR is as follows [27].
Comparative analysis of the error rate between the real values and the estimated values of 1 and 2 Lines
Overall, it can be seen that the value of PSNR is superior to the VDSR than bicubic. However, contrast does not occur significantly in the two metrics. This is because the quality of the entire region is calculated, not the ROI, which is a specific region of the two images.
The NIQE is one of the metric techniques for calculating the non-reference image quality score, and the lower the score, the better the perceptual quality. Overall, the value of the VDSR is lower than that of bicubic, indicating that the quality of the image is excellent. The image quality is improved by 124.066% at 5 mm/s, 124.225% at 7 mm/s, 129.693% at 9 mm/s, 135.29% at 11 mm/s, 120.303% at 13 mm/s, and 132.209% at 15 mm/s.
In the case of images obtained using the VDSR, it is considered that they will be used more effectively when there are complex shapes or many unspecified discontinuities. As a subsequent process, automatic slit detection of the image for which resolution enhancement was performed was performed using the VDSR.
4.4 Binarization Applied Morphology Operation
The otsu algorithm was used to binarize images to which the VDSR was applied. The otsu algorithm is a technique that calculates the optimal threshold for classifying gray scale-based images into 0 and 1 by using a histogram. Calculate the threshold that maximizes between-class variance, and the Eq. (9) is as follows.
Where t is the thresholding value, ω0(t) and ω1(t) mean the cumulative probabilities of classes 0 and 1, respectively. m0(t) and m1(t) mean the cumulative average of classes 0 and 1, respectively.
Where argmax is the variable t with maximum value. The otsu algorithm selects the optimal binarization threshold by calculating the maximum value.
After binarization was performed using the otsu algorithm, a morphology operation was used to remove pixel noise. For the morphology calculation, functions provided by MATLAB, such as bwareaopen, strel, and imfill, were used. Fig. 10 shows the binarized image acquired using the otsu algorithm. The faster the moving speed, the higher the thermal contrast between defective area and sound area, so the shape of the slit can be clearly detected. Therefore, automated slit detection was performed by applying the MOT algorithm to images at a moving speed of 15 mm/s.
Binarization of VDSR images using otsu algorithm
4.5 MOT-based Slit Detection
The MOT technique is the algorithm that tracks moving objects based on their motion and is a very important element in computer vision applications. The MOT algorithm can be classified into two techniques. First, detect moving objects in each frame of the video. Second, defect areas correspond to the same object over time. In this study, the first technique was applied.
To detect motion objects, an image background subtraction algorithm was used based on the gaussian mixture model. Afterwards, noise is removed using morphology operations. Finally, binarization processing was performed for the otsu algorithm, and moving objects were detected.
Slits were detected in the image at 15 mm/s, where the greatest thermal contrast occurred. The total frame of the 15 mm/s image acquired by the IR camera was 1,138, and the VDSR was applied to all images. Afterwards, the images were collected, and the MOT technique was applied. Fig. 11 shows the tracked slits for each frame.
Resolution-improved images with the VDSR neural network applied
In Fig. 11, objects with unclear slits were set to be recognized as ‘estimated’. Additionally, the numbers in each frame indicate the order in which objects are tracked. Even though noise was removed through morphology operation in 200 frames, the object occurring in the center failed to be tracked due to the presence of noise. At 400 frames, all slits are measured by the IR camera as they pass through the copper coil, and all slits can be tracked. At frame 700, it can be seen that confusion occurred while tracking the slit due to the coil. This is because the temperature of the coil was relatively high while converting the thermal image to the binary image, so the pixel values of the slit and coil were converted to 1. Finally, it can be seen that most slits are tracked at 1,000 frames.
5 Conclusions
In this study, the correlation equation was acquired by estimating the thermal trend based on the induction thermography technique. In addition, the resolution of the image was improved by using the deep learning-based the VDSR algorithm, and the MOT algorithm was used to detect the slits in the image to which the VDSR algorithm was applied. The conclusions of this study are summarized as follows.
1) After acquiring thermal images for each moving speed, thermal contrast was analyzed through the profile. It can be seen that the faster the moving speed, the higher the thermal contrast between the defective area and the sound area.
2) The problem of localized thermal concentration was solved using the FFT algorithm based on gaussian filtering. In addition, the resolution of the image was improved using the VDSR deep learning. The PSNR value evaluated the quality of the entire area rather than the ROI, so there was no significant difference. However, when the VDSR algorithm was applied, the NIQE value was greatly improved. Typically, it improved to 124.066% at 5 mm/s and 132.209% at 15 mm/s.
3) The otsu algorithm with morphological calculation was used to acquire a binarized image with pixel noise removed. Binarization conversion of the image to which the VDSR algorithm was applied was performed using the otsu algorithm. In addition, pixel noise was removed using the morphology calculation techniques bwareaopen, strel, and imfill functions.
4) The MOT algorithm was used to track the slits in images to which the VDSR was applied. Although all slits could be traced, there were frame sections where blob analysis was affected by the coil, and confusion occurred.
This study presented that images converted to low resolution using the GF algorithm were converted back to high resolution using the VDSR algorithm. In addition, a technique for detecting slits in scanning images was presented using the MOT algorithm.
Inductive IRT technology has difficulty inducing current when there is curvature on the surface. Additionally, there are limitations in detecting internal discontinuities due to the excitation frequency of the skin depth. However, it is emerging as a promising technology in the 4.0 NDT era, as it is possible to inspect a large area in real-time for defects existing on the surface. Through this study, it is possible to acquire big data for transfer learning of deep learning, and it is expected to be used as basic research data capable of detection moving objects.
Notes
Funding
This work was supported by the Technology Development Program (S3249835) funded by the Ministry of SMEs and Startups (MSS, Korea).
Conflict of interest
The author declares no competing interests.
References
Biography
Seungju Lee is a Post-doctoral fellow in Eco-Sustainable Energy Institute of Kongju National University, Korea. He received his PhD from Kongju National University in 2023. His research interests are as follows: 1) Evaluation of optimal thermal performance using optical and electromagnetic IRT techniques 2) Development of AI-based real-time automated object recognition system 3) Development of GUI S/W for field application
Yoonjae Chung is a Post-doctoral fellow in Eco-Sustainable Energy Institute of Kongju National University, Korea. His research field is 1) infrared thermographic Non-destructive testing & evaluation, 2) infrared signal, data and image fusion algorithms for enhancement the defect detectability, 3) Mechanical design and computational numerical analysis of solid-fluid interaction.
Wontae Kim is a professor emeritus in Eco-sustainable Energy Institute of Kongju National University, Korea. He received his Ph. D. on Mechanical Engineering from University of Utah in 1991. His research field is 1) Detection of backside defects in industrial materials using 4.0 NDT-based IRT technique, 2) AI-based Digital signal and image processing, 3) System modeling and computational analysis of solid-fluid interaction, 4) Assessment on human thermology and thermal comfort.