Morphological Dynamics-Based Anomaly Detection Towards In Situ Layer-Wise Certification for Directed Energy Deposition Processes

The metal-based additive manufacturing (AM) technologies have demonstrated their significant potential by producing fully functional parts with reduced production time and costs for low-volume and high-value complex-shaped components. The AM technologies provide the opportunity to manufacture parts of diverse designs in both nonstandard and remote environments, which is not feasible for conventional manufacturing processes. Moreover, AM is also capable of reducing the overall time-to-market of new products by accelerating the in-house testing of the designed prototypes prior to first market introduction [1,2]. The directed energy deposition (DED) process is a widely used metal-based AM process with the capacity of product prototyping, production, and component repair. During the fabrication, the feedstock material is deposited while being melted and subsequently solidified on top of the previous layer to gradually form the final geometry [3]. The high uncertainty in the DED process may lead to quality issues in the final parts, including porosity, cracks, and lack of fusion [4]. These issues will significantly compromise the mechanical properties and reliability of the fabricated AM components, hindering broader adoption of DED technologies in various mission-critical applications. Therefore, there is an urgent need in developing reliable certification methods to accelerate the scale up of DED technologies [4,5].

AM component certification refers to evaluating the conformity of the component during or after the fabrication based on the pre-established standards [6]. In practice, AM component certification activities are essential to assure the trustworthiness and reliability of the AM components, especially for mission-critical applications [7]. To satisfy the needs for certification, various anomaly detection methods can provide critical information for AM component certification, where different inspection techniques can be adopted to identify and characterize the critical flaws (e.g., porosity and crack) in the fabricated component [8,9].

In the DED process, the thermal history is capable of characterizing process dynamics to predict material microstructure and its resulting mechanical properties. Various AM process modeling studies focus on characterizing process thermal history. Some approaches characterize the thermal history using finite element models (FEMs) [10,11]. However, they are usually extremely computationally expensive, highly dependent on the design geometry, and very cumbersome to capture process uncertainty [4]. Additionally, notable advancements of sensor technologies have enabled in situ monitoring and anomaly detection towards component certification for metal-based AM [4,12,13]. Most approaches for in situ AM process monitoring are performed through the feature extraction from thermal images [13–15]. Alternatively, the layer-wise thermal image series can capture the complete spatiotemporal information in a layer-wise manner, which is more robust for anomaly detection of the fabricated parts. However, in most studies, only local features extracted from individual thermal images are considered, and those approaches cannot be directly applied to layer-wise certification, and the key reasons include: (1) the thermal distribution of different layers varies continuously (i.e., mushroom effect), and thus it is difficult to establish one benchmark distribution for all the layers in one build [4]; (2) given a complicated component, the thermal behavior also varies as a function of the printing path, which needs to be accounted for in the anomaly detection method [16]; and (3) the state-of-the-art approaches are usually purely data-driven and thus do not directly quantify the morphological dynamics of melt pools [17,18], which often makes their anomaly detection models and results challenging to interpret.

To fill these gaps, a new in situ layer-wise AM anomaly detection method is proposed by characterizing the morphological dynamics of the melt pools and heat affected zones (HAZs) which are segmented based on specific temperature ranges. In this study, the optimal transport problems are leveraged to formulate the morphological dynamics, and a new Wasserstein distance (WD)-based transition metric system is developed to characterize the temporal variation in the melt pool and HAZ morphologies, respectively. Subsequently, several new layer-wise key process features are extracted and used for layer-wise anomaly detection, facilitated by a supervised machine learning framework.

The technical contributions of this paper are summarized as follows. First, it explicitly establishes a physics-informed formulation of layer-wise melt pool and HAZ morphology dynamics to characterize the stability of thermal history. Second, several novel layer-wise process key features are proposed for the layer-wise anomaly detection of the DED process based on the melt pool and HAZ transition metrics with validated better performance than the state-of-the-art approaches, providing critical information towards AM component certification. The impacts to the research community and industries are twofold. For the AM quality control area, the newly proposed method opens the new venue of physics-informed data-driven anomaly detection for AM processes, which provides critical information for component-wise certification as well as guiding in situ process control practices for AM quality improvement. For AM industrial practices, reliable layer-wise anomaly detection can significantly reduce the needs for post-manufacturing inspection for AM components certification and ultimately accelerate the broader adoption of AM technologies in mission-critical applications.

The remainder of the paper is organized as follows. A literature review on the state-of-the-art metal-based AM process anomaly detection techniques is summarized in Sec. 2. The proposed methodology for developing the new melt pool and HAZ transition metrics and their properties, and layer-wise certification are introduced in Sec. 3. In Sec. 4, case studies of fabricating a thin wall and a cylindrical specimen using the DED process are presented, and the performance of the proposed method is compared with the benchmark methods. Finally, Sec. 5 presents the conclusion and potential future research directions.

This section briefly summarizes the state-of-the-art methodologies for metal-based AM process anomaly detection. The major metrological technologies used for AM anomaly detection are first summarized. Subsequently, the defect characterization and modeling techniques based upon the data collected using the metrological technologies are summarized. Lastly, the research gaps of machine learning-based AM anomaly detection are briefly discussed.

Most data-driven AM anomaly characterization methods heavily rely on data collected from the metrological techniques used. In this section, a comprehensive review and comparison among different metrological techniques used for AM quality assurance and certification is conducted. The respective merits and limitations of the most widely used metrological approaches are briefly summarized in Table 1.

All the metrological techniques can be briefly categorized into two types, in situ and post-manufacturing techniques (as illustrated in Table 1). In cases of in situ sensing technologies, the measurements are taken from the system without interrupting or hampering the AM process. The post-manufacturing inspection technologies, on the other hand, can only be used after the completion of the part fabrication. Therefore, those in situ metrological techniques can be used to develop real-time process monitoring and quality prediction models, and the post-manufacturing approaches can be used to provide the validation for those data-driven models.

The defect characterization methods, including X-ray computed topography (XCT), magnetic resonance imaging (MRI), and ultrasonic testing (UT), are widely used for post-manufacturing anomaly characterization of AM parts [3–5]. The XCT technique is used in metal-based AM to measure porosity occurrences, their spatial distribution, and severity in the build parts [12,25]. The XCT data can be analyzed using the computer aided quality technology to trace the porosity orientation in the printed parts [26]. Moreover, the XCT can be analyzed using machine learning tools for automatic segmentation of porosity from the scanned images of metallic specimens [27]. Compared to other post-manufacturing methods, XCT can provide detailed 3D information even for complex geometries. However, the XCT process usually requires high energy and skilled operators to operate the machine. In addition, the XCT machine is subject to high capital cost, and the inspection process is extremely time consuming [20].

MRI is used to track the overall structure of the AM parts using radio frequency pulses and powerful magnetic field [21]. The MRI systems can produce the cross-sectional images of the structures by capturing magnetic field intensity gradient without ionizing radiation as observed in XCT scans [20,21]. However, in this approach, a complex shape may need to be tested for multiple times to obtain precise structural information, making it time consuming and complicated in data collection [20].

The UT technology is also primarily used to characterize the internal structure of parts [28]. Basically, the ultrasonic phase velocity is used to characterize the defects like pores and voids in AM parts. In some studies, during experimentation, the samples are printed with a porosity of up to 5–15%, and the output denotes that the measured phase velocity is inversely proportional to porosity [22,29]. Moreover, the ultrasonic phase velocity can characterize defects and microstructures of AM parts. For example, changing process parameters, like hatch spacing and mechanical properties, can affect the defect formation, leading to varying pores numbers, sizes, and shapes, which can be captured by the UT technology [22]. As a non-destructive technique, UT measurements can be integrated with other monitoring techniques for developing a comprehensive evaluation system for AM parts. It is very important to maintain standard experimental procedures including voltage gain, pulse repetition frequency, and good surface finish, otherwise few degrees of change can deviate important signal information [29].

In summary, although the post-manufacturing approaches generate good information about the internal structure and surface properties of the AM parts, they are extremely high skill demanding, time consuming, and costly in nature [4,20]. Therefore, it is challenging and usually impractical to extensively apply these post-manufacturing technologies in AM certification. However, the detailed information and superior accuracy of post-manufacturing approaches make them a good choice for providing limited labeling information for supervised and/or semi-supervised learning for in situ AM certification.

In situ monitoring of the AM process can capture the process uncertainty and thus can certify the AM process/part in real time. For in situ AM process monitoring and anomaly detection, multiple data sources of thermal imaging, optical imaging, and 3D scanning can be used to capture the process information during the fabrication of AM parts. In this section, the recent works in anomaly detection are categorized based on the sensing technologies used, namely, thermal imaging, video streams, and 3D point cloud.

The thermal characteristics are identified as one of the most informative process signals for the AM process dynamics. During the fabrication of AM parts, the thermal history can be used for process monitoring, microstructural assessment, and mechanical properties prediction [30]. In general, the AM process thermal history is captured as a high-resolution thermal image stream, which are subsequently used for developing data-driven approaches to model process dynamics for anomaly detection [12,31–33]. In the specific case of metal-based AM, major proportion of the published works are conducted with thermal imaging systems, where the machine learning-based models are developed for in situ anomaly detection. In some studies, along with the thermal history, the morphological features of the melt pool (e.g., depth, size, and temperature distribution) are extracted for anomaly detection [11,12,14]. Furthermore, instantaneous temperatures, melt pool orientations, and aspect ratios are also estimated based on pyrometry observation for defect detection [34]. Besides these, tensor decomposition has been applied to the thermal images for dimension reduction, making it easy to model the process-defect relationship [4]. In addition, deep learning-based models are developed to link the layer-wise images and the ground truth XCT data for autonomous anomaly detection and classification [35,36]. Moreover, unsupervised anomaly detection method based on neural networks is also used for distance-based multivariate anomaly detection [37]. With only a few exceptions of layer-wise modeling proposed by Seifi et al. [4] and Esfahani et al. [16], most of the existing data-driven approaches focus on process monitoring and anomaly detection only considering individual thermal images, which fails to characterize layer-wise process condition and part quality. However, quantifying the morphological variability of the multiple melt pools within a layer will provide a new process knowledge-driven approach for anomaly detection.

Another stream of research utilizes the in situ optical video acquisition technique, which has become more accessible due to the availability of robust machine vision systems integrated with the AM process, enabling highly efficient online anomaly detection and process monitoring [38–40]. Using the video as a spatiotemporal data stream, the integrated spatiotemporal decomposition and regression-based modeling is proposed for in situ anomaly detection and monitoring [41]. In addition, this method can be applied to any image-based process monitoring applications where the foreground events are random and sparse, and the anomaly is spatially and temporally correlated. A recursive estimation procedure for real-time implementation of the algorithm is also proposed to handle the challenges of the high dimensionality of the video-image stream. However, the spatial information of the video streams may not be sufficient to distinguish the anomalous events from other natural foreground [41,42].

Three-dimensional point cloud data are used for geometric deviation and surface defect detection of AM parts [23,43]. The machine learning models can be trained with synthetic 3D point clouds rather than experimental data that can reduce training time and costs associated with many prints for each design [44]. Basically, 3D point clouds are high-density data and used to measure the geometric integrity of the AM products fabricated through a variety of processes. Specifically, dimensional or surface variation induced by miniature process shifts in the AM system can be detected and characterized using point cloud-based approaches [23]. For instance, the laser-scanned 3D point cloud data are used for identifying dimensional variation in AM parts [45]. Although the point clouds data are more capable of depicting structural information of the printed parts compared to low-dimensional signals, translating the 3D point clouds data into a model is very difficult due to the computational complexity and segmentation precision issues [45,46].

This section discusses the research gaps in the state-of-the-art machine learning-based anomaly detection methods for anomaly detection of metal-based AM processes.

The machine learning models aim to capture the process-defect relationships to distinguish between anomaly inducing samples and healthy samples, with various machine learning models being adopted [18,33]. One group of mostly used machine learning approaches are the deep learning related methods, which include convolutional neural network [47–49], long-term recurrent convolutional networks [50], deep belief networks [51], and self-organizing maps [14]. Although these methods demonstrate great success in capturing the patterns to distinguish the anomalies from the healthy samples, they also have some limitations when being applied to layer-wise anomaly detection. First, these models mostly need to be trained using a relatively large training dataset, which can be quite costly for layer-wise anomaly detection with limited model interpretation. Second, it can be difficult for the deep learning methods to incorporate engineering domain knowledge, such as the layer-wise printing trajectory information, in the modeling and anomaly detection.

In addition, various other methods first leverage feature extraction approaches to extract critical process features from the high-dimensional raw process signals, and subsequently apply various machine learning approaches. The low-dimensional features extracted include melt pool geometric features [52], melt pool boundary related morphological characteristics [13], melt pool surface temperature related features [14,53], tensor decomposition-based features [4,12], auto-encoder-based features [54,55], and various summary statistics of the raw signals [56]. Once the features are extracted, various machine learning methods, such as support vector machines (SVMs) [4,13,32,57], discriminative models [32,56], and random forests [58], have been used for anomaly detection. These methods have demonstrated good performance in anomaly detection for individual observations (e.g., single melt pool images). However, they can be quite cumbersome when being applied to layer-wise anomaly detection for the following two main reasons. First, the thermal distribution of different layers in the same build is continuously varying, and thus it is difficult to establish one benchmark distribution for all the layers in one build. Second, given a complicated component, the thermal behavior also varies as a function of the printing path, which needs to be accounted for in the anomaly detection method.

To fill the above-mentioned gaps, the proposed method aims to extract layer-wise process features that can account for the varying thermal distribution due to different layer change as well as constantly changing printing trajectories. A new in situ layer-wise AM anomaly detection method is proposed by characterizing the morphological dynamics of the melt pools and HAZs which are segmented based on specific temperature ranges. Specifically, for characterizing the morphological dynamics of the melt pools and HAZs, the Wasserstein distance-based optimal transport problem is formulated by leveraging the engineering domain knowledge which is concise and easily interpretable. Subsequently, the WD-based morphological transition metric (MTM) features can be extracted for layer-wise anomaly detection, and a support vector machine-based supervised machine learning framework is leveraged for layer-wise anomaly detection.

In this section, the proposed methodology is introduced. The melt pool and HAZ in the thermal images are first segmented, and subsequently the new melt pool/HAZ transition metrics are derived by leveraging the Wasserstein distance to quantify the perturbation in a series of melt pools/HAZs’ morphologies. The resulted transition metrics are used to extract critical layer-wise features for anomaly detection towards AM part certification. Finally, the supervised learning method is leveraged for layer-wise anomaly detection based on the labeling information generated by post-manufacturing inspection methods, such as XCT. The overall steps of the proposed methodology are illustrated in Fig. 1(a), and the data visualization of multiple key steps is shown in Figs. 1(b)–1(e).

The thermal images are captured using a pyrometer camera that consists of a temperature measurement at each pixel within the field of view of the camera. To compress the thermal image data without losing useful information for anomaly detection, the widely used concept of heat affected zones is adopted in this paper together with the melt pool to provide more comprehensive information to characterize the observed temperature distribution. For the sake of clarity, the heat affected zone observed in the thermal image is defined as follows.

A new transition metric system for both melt pools and HAZs is developed by leveraging the Wasserstein distance formulation. The newly developed transition metrics are introduced as follows;

Supervised learning algorithms can be used to establish a distinction between healthy and unhealthy layers based on the layer-wise features extracted. The post-manufacturing XCT scanning can be used to obtain structural quality information of each layer, providing the ground truth information to the data set. For layer-wise anomaly detection, one layer is labeled as abnormal if there is at least one porosity observed. It is worth noting that all different machine learning methods can be used to detect layer-wise anomalies. The SVM technique is adopted in this work because of the flexibility it can offer in learning the decision boundaries by varying kernel functions. The SVM classifier can be trained by finding the hyperplane that best separates all data points of distinct classes using a training data set with labeling information, and the complexity of the hyperplane can be determined through cross validation [63]. In summary, the newly proposed in situ layer-wise anomaly detection algorithm is illustrated in Table 3.

In this section, the proposed in situ layer-wise anomaly detection method is validated based on the data collected from two specimens using the DED process, which is one of the most common techniques to perform powder-based DED.

The machine used for data collection was the LENS 750 machine manufactured by OPTOMEC, and the machine is equipped with the 1 kW Nd:YAG laser [16]. To evaluate the robustness of the proposed method, two specimens with different geometries (i.e., one thin wall and one cylindrical shaped part) were fabricated using the Ti–6Al–4V powder as the feedstock material. The process parameters used for both specimen fabrication are listed in Table 4 [16].

For process data collection, the dual-wavelength coaxial pyrometer (Stratonics, Inc.) captures the temperature distribution from the top view of melt pool. The key benefit of using the pyrometer camera is that the image data collected demonstrate reduced motion blur, as it has a specific exposure period (2.0274 ms) with the nominal image collection rate of approximately 6.4 Hz. A brief sketch of the experimental setup with the sensor positioning is illustrated in Fig. 4.

Each observed thermal image is in the form of a 752 × 480 matrix, which consists of the temperature values at each pixel within the range of 1000–2500 °C. During the thin wall fabrication, the captured melt pools in each layer represent identical orientation, due to its unidirectional printing path. However, for the cylindrical specimen, the layer-wise melt pools demonstrate varying orientations due to the complex printing paths. Furthermore, using the XCT inspection (Skyscan 1172) technology, the porosities in the printed specimens were characterized, and thus the layer-wise process data were labeled using the porosity characterization results [14]. All the layers that include at least one pore with diameter larger than 0.05 mm were labeled as unhealthy [3].

Two benchmark methods of layer-wise AM process anomaly detection are selected to compare with the proposed method. The first benchmark method was the layer-wise anomaly detection model proposed by Seifi et al. [4], which leveraged multi-linear principal component analysis (MPCA). In this approach, the MPCA algorithm was used for dimensionality reduction for each single thermal image, and then the volume of the convex hull formed by the extracted multi-linear principal components from each layer was employed as a layer-wise process feature for anomaly detection. In addition, the maximum norm value of the residual from the MPCA modeling within a layer was used as a secondary layer-wise feature during classification for anomaly detection. The effectiveness of this benchmark method was validated by comparing with the traditional machine learning driven anomaly detection approaches.

More recently, in situ layer-wise certification of direct laser deposition process based on image series analysis was proposed by Esfahani et al. [16]. This method leverages the image registration technique to characterize the thermal history dynamics in the layer-wise thermal images. Moreover, the Gaussian process (GP) models were employed to characterize the unexplained variation component by the image registration operation. The effectiveness of this benchmark method was validated by comparing with the approach proposed by Seifi et al. [4].

In this study, to compare the effectiveness of the proposed and the benchmark methods, both thin wall and cylinder data set were randomly split into the training layer set (80%) and testing layer set (20%), and 200 replications of this evaluation were performed to assess the average performance. Basically, using the training layer set, the Gaussian SVM classification model can be trained when the GTM and MTM features have been defined, which can then be used to predict the labels of newly observed layers. Specifically, the classifier is fed a vector of response labels (0 for healthy, 1 for unhealthy) and a matrix of four predictors: the GTM and MTM extracted from the melt pool and tail region, respectively. The features of GTM and MTM are associated with the segmented melt pools/HAZs of thermal images, and the selection of temperature range of interests to segment the HAZ affects the extracted features and the prediction accuracy. In this case, the melt pool regions were segmented based on the melting point of the feedstock material and above, i.e., $R1=[1636,+∞)$⁠, and the HAZs were segmented considering some specific temperature to the melting point of the feedstock material, i.e., $R2=[T0,1636)$⁠, which can define as the tail region of the melt pool.

The lower bound T₀ of the tail region temperature range can be considered as the tuning parameter in the proposed approach. By tuning the value of T₀, the effect of different temperature ranges of interests on the extracted features in both the thin wall and cylindrical specimen can be depicted as shown in Fig. 5. It can be observed that different values of T₀ lead to different distinguishability between the healthy and unhealthy layers for both specimens, demonstrating the necessity of parameter tuning for the lower bound temperature T₀.

As mentioned above, the tuning parameter affects the extracted features and subsequently the prediction results. Therefore, by tuning the temperature range of the tail region, the performance metrics are listed as shown in Table 5, where $R1$ is always fixed and $R2$ changes as the T₀ value is tuned. From Table 5, it is observed that in the case of thin wall, $T0=1000∘C$ (bolded) provides the highest F-score value. Furthermore, in the case of cylinder, the F-score value shows the best results when $T0=1400∘C$ (bolded).

The performance metrics of the proposed and benchmark methods are summarized in Table 6. Furthermore, a number of sample size values were explored to examine the effects of the sample size used in fitting the GP model for the error matrix in Ref. [16] for a fair comparison. When comparing the F-score, it is observed that the proposed method outperforms both benchmark methods for the thin wall specimen. Moreover, for the cylindrical specimen, the proposed method outperforms Seifi et al.’s [4] in all four metrics and demonstrates comparable (or slightly better) Fscore results with Esfahani et al. [16]. The good performance of the proposed method is mainly due to its joint consideration of the layer-wise melt pools/HAZs morphological dynamics. In addition, given the robustness in the performance of the proposed method in both thin wall and cylindrical specimens, the proposed method can be applied for in situ layer-wise anomaly detection for intricate shaped parts.

There are three possible reasons for the classification errors observed in the case study. First, the discrete data sampling of the pyrometer camera may potentially lead to a lot of missing data in the thermal data collection, resulting in missing defect inducing process information. Second, the characterization of XCT scanning may be subject to noise and error, leading to occasional labeling errors in the data set. Third, the proposed in situ layer-wise anomaly detection approach ignores the impacts of re-melting between consecutive layers and adjacent tracks, which may eliminate some porosities formed in the preceding layer/track during subsequent layer/track deposition.

The average computational efficiency of the proposed method is illustrated in Fig. 6, where the average computation time and the average build time for each layer are compared for both specimens. The processor used to implement the proposed method was Intel^® Core^™ Processor i7–7700 CPU at 3.60 GHz. It is worth noting that the average computation for the proposed method takes significantly shorter time than its corresponding layer-wise build time for both thin wall and cylinder specimen. As a result, the proposed method can facilitate layer-wise AM certification in real time, potentially enabling real-time process control for quality improvement.

The quality issues are the major barriers for wider and sustainable industrial adoption of the AM technology, and there is a need of reliable techniques to detect quality issues (i.e., porosity, mini cracks, lack of fusion, etc.) in AM processes. Different approaches like FEMs are used to characterize the anomaly based on thermomechanical relationship, but they cannot realistically capture process uncertainty. These limitations are addressed by the data-driven approaches where usually the individual thermal images of the melt pools are used for local anomaly prediction. Few approaches are proposed for layer-wise anomaly detection, and the extracted features cannot explicitly characterize melt pool/HAZ morphological dynamics.

This paper proposes a novel formulation of melt pools and HAZs morphological dynamics through which the new melt pool/HAZ transition metrics are derived. Furthermore, novel layer-wise process key features are derived from the characterization of melt pools/HAZs morphological dynamics for AM layer-wise anomaly detection. Specifically, morphological transition between consecutive melt pools/HAZs along with the global temperature shift are extracted as the novel features. Based on the extracted layer-wise features, the SVM classification approach is used for in situ layer-wise anomaly detection. The specimens of a thin wall and a cylinder were fabricated using the DED process to validate the proposed methodology. The anomaly detection accuracy of the proposed model is reasonably high and outperforms both benchmark methods, which are both state-of-the-art AM layer-wise anomaly detection methods.

For future research, a couple of interesting topics remain open. First, the inter-layer and inter-track dependence during fabrication can be characterized to quantify the re-melting effect between the adjacent layers/tracks. Second, a part certification framework can be established to quantify the size and severity of the defects and their impact to the part functionality. Finally, layer-wise process control algorithms can be used in conjunction with the in situ monitoring scheme to adjust process parameters for quality improvement of the AM parts.

This work was partially sponsored by the National Science Foundation (No. CMMI-2046515).

There are no conflicts of interest.

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.