Energy efficiency prediction model in fused deposition modeling based on Bayesian optimized random forest

INTRODUCTION

Additive manufacturing (AM), often referred to as 3D printing, is a technique for fabricating solid parts by layering materials based on a 3D computer-aided design (CAD) model. Unlike traditional subtractive manufacturing (cutting and machining), AM is a “bottom-up” material accumulation method^[1]. Fused deposition modeling (FDM) 3D printing primarily uses filament materials such as polylactic acid (PLA), acrylonitrile butadiene styrene (ABS), and fiber-reinforced composites. The filament is heated and melted, extruded through a nozzle, and then deposited layer by layer onto a preheated substrate, ultimately forming parts with specific shapes and structures. Depending on the characteristics of the part’s shape and structure, supports may be added as necessary. Based on its process principles, FDM 3D printing demonstrates high material utilization, aligning with the requirements of green manufacturing. However, this technique still faces challenges such as long printing times and high energy consumption^[2]. Due to the unique process characteristics of FDM, numerous printing parameters such as printing speed, infill speed, nozzle temperature, bed temperature, and layer height exist. It is intuitively understood that there is a certain relationship between these parameters and energy consumption. To obtain a more comprehensive understanding of how printing parameters influence energy consumption, researchers have conducted a series of studies using various qualitative and quantitative analysis methods.

In qualitative analysis, the impact of printing parameters and their interactions on energy consumption is studied using experimental design and statistical analysis methods. Griffiths et al. employed the design of experiments (DOE) method to investigate the effects of part orientation, number of perimeters, infill rate, and layer thickness on energy consumption^[3]. Their results indicated that layer thickness was the most influential factor in determining energy consumption, and the interactions between the studied parameters also had a notable impact on energy consumption. Additionally, using main effects plots and contour plots of energy consumption, they determined the optimal range of printing parameter combinations. Enemuoh et al. applied Taguchi orthogonal design, signal-to-noise ratio analysis, and variance analysis to study the effects of five printing parameters: infill rate, infill pattern, layer thickness, printing speed, and number of perimeters on energy consumption^[4]. Their findings showed that layer thickness exerted the greatest influence on energy consumption, followed by printing speed, infill rate, infill pattern, and number of perimeters. They identified optimized parameter levels based on the signal-to-noise ratio analysis. Galetto et al. used experimental design and variance analysis to examine the impact of layer height, infill rate, nozzle temperature, printing orientation, number of perimeters, printing speed, and retraction speed, as well as the interactions between these parameters, on FDM energy consumption^[5]. They found that low infill rate, large layer thickness, and fewer perimeters were conducive to reducing energy consumption. Furthermore, based on the experimental design analysis results, they fitted a polynomial to approximate the relationship between printing parameters and energy consumption, identifying the corresponding optimized parameter combinations for different optimization needs. Elkaseer et al. utilized Taguchi orthogonal design, variance analysis, and polynomial fitting methods to establish a quadratic regression model for the relationships between infill rate, layer height, printing speed, printing temperature, and printing orientation with energy consumption^[6]. They used this model to predict how printing parameters affect energy consumption, finding that printing speed and layer thickness exerted an important influence on energy consumption. Camposeco-Negrete et al. employed Taguchi design and variance analysis to study the effects of layer thickness, infill pattern, printing orientation, and printing position on energy consumption^[7]. They further determined the optimized combination of printing parameters through desirability analysis. Al-Ghamdi et al. used experimental design and variance analysis to investigate the impact of layer thickness, infill rate, printing speed, and wall thickness on FDM energy consumption^[8]. They found strong interactions between the printing parameters and developed a cubic nonlinear regression model. Hassan et al. used full factorial design and variance analysis to study how six different infill patterns and four infill rates influence FDM energy consumption^[9]. They established a polynomial regression model discovering that both of them had a notable impact.

In quantitative analysis, it is rare to find studies that utilize physics-based functions to clearly describe how printing parameters affect energy consumption. This is mainly due to the complexity of the relationship, making it challenging to establish an expression using physics-based mathematical models. Therefore, polynomial fitting methods based on quadratic regression are primarily used to establish the relation between printing parameters and energy consumption. Feng et al., based on experimental design results, employed the group method of data handling (GMDH) to establish second-order nonlinear relationship models between three printing parameters and FDM printing energy consumption^[10]. Tian et al. used a linear regression model to predict the influence of printing parameters on the FDM energy consumption, and proposed a nonlinear optimization approach to identify the optimal combination of printing parameters that minimizes energy consumption while meeting part quality requirements^[11]. Alizadeh et al. used polynomial fitting to establish a quadratic polynomial regression model for the relationship between layer thickness, printing speed, nozzle temperature, and energy consumption^[12]. They then used the nondominated sorting genetic algorithm II (NSGA-II) and technique for order preference by similarity to ideal solution optimization (TOPSIS) methods to solve a multi-objective optimization model for energy efficiency and geometric accuracy of the parts, obtaining optimized combinations of printing parameters based on the quadratic polynomial regression model.

In recent years, with the swift advancements in artificial intelligence, machine learning (ML) has found extensive applications in the mechanical manufacturing sector. Applications such as fault diagnosis^[13] and defect monitoring in part manufacturing^[14] have shown promising results. In terms of regression prediction, ML techniques are frequently employed to forecast the correlation between design parameters and performance metrics in aerospace optimization design^[15], establishing surrogate models for optimization, and achieving effective results. As a data-driven modeling and prediction method, ML has shown unique advantages in multi-input single-output regression prediction and has achieved broader adoption in the field of AM. Xia et al. employed three ML algorithms, adaptive neuro fuzzy inference system (ANFIS), extreme learning machine (ELM), and support vector regression (SVR), to establish predictive models for the surface roughness of parts formed by arc AM^[16]. The study showed that the genetic algorithm (GA)-ANFIS model, optimized by a GA, had relatively better predictive performance. Li et al., aiming to study the influence of process parameters on the performance of parts formed by selective laser melting (SLM), utilized Kriging, radial basis function (RBF), and SVR to build a composite surrogate model based on local error for predicting material utilization, energy consumption, and tensile strength of SLM parts^[17]. The predictive method’s effectiveness was verified through experiments. Baturynska et al., addressing the low prediction accuracy of linear regression methods in predicting geometric errors of AM parts, proposed applying ML algorithms such as SVR, decision tree (DT) regression, and multilayer perceptron to predict geometric errors^[18]. The comparison with linear regression models indicated that ML algorithms had better prediction performance for geometric errors in AM parts. Cai et al., studying the relationship between dynamic strength and printing parameters of 3D-printed polypropylene composites, used six ML algorithms to establish predictive models^[19]. The study found that artificial neural networks (ANNs) had the best predictive performance but required longer prediction times, while SVR performed well in both predictive performance and prediction time. Gor et al. used ML algorithms such as ANN, K-nearest neighbors (KNNs), and support vector machines (SVMs) to predict the density of parts produced by powder bed fusion AM^[20]. Their results indicated that among the ML algorithms used, SVM exhibited relatively the best predictive performance. Kumar et al. applied the KNN algorithm to predict the influence of process parameters on the surface roughness of parts formed by microplasma arc metal AM^[21]. Experimental validation showed that the prediction error of surface roughness confirmed the effectiveness of the KNN algorithm in predicting surface roughness for microplasma arc metal AM. Ranjan et al. used the random forest (RF) algorithm to predict how process parameters influence mechanical properties, such as bending performance and fracture strength of Al-reinforced ABS composite parts fabricated by fused filament fabrication (FFF) 3D printing^[22]. Wu et al. employed RF, SVR, ridge regression (RR), and least absolute shrinkage and selection operator (LASSO) regression algorithms to establish predictive models to link FDM printing parameters with the surface roughness of formed parts^[23]. The experimental validation results demonstrated the effectiveness of these methods in predicting the surface roughness of FDM parts. Furthermore, Li et al. utilized an ensemble learning method to create a ML model comprising six heterogeneous base learners to predict the influence of FFF printing parameters on the surface roughness of formed parts^[24]. Their study showed that this predictive method achieved high accuracy in forecasting the surface roughness of FFF parts. Feng et al. utilized the GMDH to establish an energy consumption predictive model^[10]. They further applied the differential evolution (DE) algorithm to optimize printing parameters for low energy consumption and conducted variance analysis to study the impact of three printing parameters on energy consumption. Zhang et al. used a back-propagation neural network (BPNN) to establish an energy efficiency predictive model^[25]. Based on the model, they employed an adaptive niche GA to solve an energy efficiency optimization model and obtain optimized printing parameter combinations.

Based on the above analysis, it is evident that ML algorithms have realized broader adoption in AM and predicted the correlation between printing parameters and the quality of formed parts, including surface roughness and mechanical properties, resulting in accurate predictive outcomes. However, for different research problems, such as the regression modeling prediction between FDM printing parameters and energy consumption, the accuracy of predictive results varies depending on the ML algorithm used. In the current research on the correlation between FDM printing parameters and energy consumption, whether using polynomial fitting methods or ML methods for regression prediction, there is a lack of in-depth analysis regarding the predictive accuracy of the methods employed. This leads to an insufficient understanding of the most effective ML methods. Furthermore, it is clear that research on energy efficiency optimization in FDM based on ML methods is limited. Consequently, exploring various ML predictive approaches for FDM energy consumption is essential.

METHODS

Establishment of RF model based on Bayesian optimization

DT

DT is a commonly used ML algorithm, and its model training process is based on a tree structure, conducting a series of attribute tests according to the sample datasets. Typically, a single DT model consists of a root node, several internal nodes, and leaf nodes, where the leaf nodes correspond to the results obtained by the model’s decision, and the root node and internal nodes correspond to specific attribute judgments. Based on the results of attribute judgments, the sample datasets contained in each node are divided into subsets and assigned to corresponding child nodes. The root node serves as the initial attribute judgment node and contains the complete sample datasets.

For the training sample dataset:

(1) $$ \mathrm{D=\left \{(x_{1},y_{1}),(x_{2},y_{2}),\dots ,(x_{m},y_{m}) \right \} \ y_{i}\in R} $$

Attribute set:

(2) $$ \mathrm{A=\left \{ a_{1},a_{2},\dots ,a_{d} \right \}} $$

In the case of a classification problem, the training process of a DT model is illustrated in Figure 1, showing that the training of the DT model is a recursive process.

Figure 1. DT model training process. DT: Decision tree.

The key issue in achieving effective modeling of DTs is how to divide the optimal attributes. In DT learning, the division of optimal attributes is mainly achieved through the following three criteria:

(1) Information Gain:

(3) $$ \mathrm{Gain(D,a)=Ent(D)-\sum_{v=1}^{V}\frac{|D^{v}|}{|D|}Ent(D^{v})} $$

(4) $$ \mathrm{Ent(D)=\sum_{k=1}^{|y|}p_{k}log_{2}p_{k}} $$

Where Ent (D) represents the information entropy, and P_k denotes the proportion of samples belonging to the k class in the training sample datasets D, k = 1, 2, …, |y|. D^v indicates the number of samples containing the value of the attribute corresponding to the internal node in the sample datasets D. A larger information gain for a particular attribute indicates that the attribute is more favorable for splitting.

(2) Gain Ratio:

(5) $$ \mathrm{Gain_{ratio(D,a)}=\frac{Gain(D,a)}{IV(a)}} $$

(6) $$ \mathrm{IV(a)=\sum_{v=1}^{V}\frac{|D^{v}|}{|D|}log_{2}\frac{|D^{v}|}{|D|}} $$

The gain ratio helps address the issue of attribute preference when using information gain as a criterion. Specifically, for attributes with a wider range of values, a higher information gain might occur, potentially leading to erroneous attribute partitioning, such as using sample ID as the optimal splitting attribute.

(3) Gini Index:

(7) $$ \mathrm{Gini(D)=\sum_{k=1}^{|y|}\sum_{k^{'}\neq k}p_{k}p_{k^{'}}=1-\sum_{k=1}^{|y|}p_{k}^{2}} $$

(8) $$ \mathrm{Gini_{index(D,a)}=\sum_{v=1}^{V}\frac{|D^{v}|}{|D|}Gini(D^{v})} $$

The aforementioned optimal attribute partitioning criteria correspond to three classic DT algorithms: the ID3 DT algorithm, the C4.5 DT algorithm, and the classification and regression tree (CART) DT algorithm. The CART algorithm is applicable to both classification and regression tasks. Therefore, in the predictive modeling of energy consumption in FDM 3D printing, a DT model based on the CART algorithm is used.

RF model

RF is an ensemble learning algorithm based on the Bagging algorithm, using DTs as base learners. Its primary modeling principle involves establishing several sample datasets through bootstrapping, constructing CART regression tree models for each dataset, and ultimately computing the arithmetic average of the predictive results from these regression tree models to yield the prediction of the RF model.

The key steps in RF modeling include:
(1) Utilizing bootstrapping to randomly sample the original training datasets, resulting in T training subsets.
(2) Based on the T training subsets obtained in step (1), further establishing T CART regression tree models through random selection of attributes.
(3) Employing each CART regression tree model to predict output results f_i (x).
(4) Calculating the prediction output results f (x) of the RF model through simple averaging, as given in

(9) $$ \mathrm{f(x)=\frac{1}{T}\sum_{i=1}^{T}f_{i}(x)} $$

According to the above analysis, the modeling process of RF is illustrated in Figure 2.

Figure 2. RF modeling process. RF: Random forest.

Bayesian optimization algorithm

The numbers of regression trees in a RF, leaf nodes, and attributes in the randomly selected subsets all significantly influence the model’s prediction performance. Therefore, parameter optimization is crucial for enhancing the performance of predictive models. Commonly used parameter optimization methods include grid search, GAs, particle swarm optimization (PSO), etc. In this paper, we adopt the Bayesian optimization algorithm for parameter optimization of the RF model. Compared to other optimization algorithms, Bayesian optimization algorithm leverages known sampled information to determine new sampling points, thus possessing the advantage of high optimization efficiency.

The problem of optimizing RF model parameters based on the Bayesian optimization algorithm can be formulated as:

(10) $$ \mathrm{x^{\ast}=argmin_{x\in X}f(x)} $$

Where x^* represents the set of model optimization parameters, x denotes the model parameters to be optimized, X stands for the parameter space for the optimization parameters, and f (x) signifies the objective function to be optimized, which is the prediction error of the RF model.

The solution steps of the Bayesian optimization algorithm mainly include:
(1) Estimating the posterior distribution of the objective function using the probability surrogate model of Gaussian process based on known sampled points and their function values.
(2) Determining the optimal sampling point for the next iteration according to the sampling rule of the acquisition function.

Based on the above analysis, the modeling process of RF based on Bayesian optimization is illustrated in Figure 3.

Figure 3. BO-RF modeling process. BO-RF: Bayesian-optimized random forest.

EXAMPLE VERIFICATION

Experimental materials and equipment

The printing material used in this study is PLA filament, and the selected FDM printing device is the UP PLUS 2. The part diagram of the printed specimen is shown in Figure 4.

Figure 4. Prototype part drawing.

After completing the 3D modeling of the part using SolidWorks based on the part drawing, the next step involves setting up the print parameters and performing the slicing operation using Cura software. First, the set values for various process parameters are obtained from the Test design results based on Box-Benhnken. Next, these parameters are configured in the Cura software. Then, the model is exported and sent to the printer for printing. During the printing process, the Aitek AWS2013 power meter is used to measure the energy consumption. Finally, the energy consumption data for FDM 3D printing is obtained. The primary function of the slicing software is to slice the model layer by layer and generate different paths based on the model’s shape and the set process parameters, ultimately producing the GCode file for the entire 3D model, which can be exported for offline printing.

RESULTS

In the BO-RF model, the Bayesian optimization algorithm is used to optimize the parameters of the RF model, including the numbers of DTs, leaf nodes, and attributes in the randomly selected subsets. The sample datasets consist of the 47 experimental data sets introduced in the data preprocessing section, with 35 sets used for the training set and the remaining 12 sets for the testing set. Figure 5 shows the energy efficiency prediction results based on the RF and the BO-RF model. Here, E_SEC is used to represent energy consumption, which is the energy consumption per unit volume of material processed.

Figure 5. Energy efficiency modeling prediction results of RF and BO-RF models. (A) Prediction results of the training set for the RF model; (B) Prediction results of the test set for the RF model; (C) Prediction results of the training set for the BO-RF model; (D) Prediction results of the test set for the BO-RF model. RF: Random forest; BO-RF: Bayesian-optimized random forest.

To validate the superiority of the BO-RF model, comparisons are made among the BO-RF energy efficiency prediction model, RF-based energy efficiency prediction model, SVR-based energy efficiency prediction model, and BPNN-based energy efficiency prediction model in terms of the four evaluation metrics: MSE, MPE, MAPE, R² [Table 4]. To visually represent the comparison results, a comparison histogram is plotted [Figure 6]. Among these models, GS-SVR stands for grid search-based SVR, GA-SVR for GA-based SVR, PSO-SVR for PSO-based SVR, BO-SVR for Bayesian optimization-based BVR, BO-MK-SVR for Bayesian optimization-based multi-kernel SVR, GA-BPNN for GA-based BPNN, PSO-BPNN for PSO-based BPNN, and BO-BPNN for Bayesian optimization-based BPNN.

Figure 6. The results of model evaluation metrics for ten models. (A) The comparison of MSE among the prediction results of various models; (B) The comparison of R² among the prediction results of various models; (C) The comparison of MAPE among the prediction results of various models; (D) The comparison of MPE among the prediction results of various models. MSE: Mean square error; R²: coefficient of determination; MAPE: mean absolute percentage error; MPE: maximum percentage error.

According to the comparison results in Table 4, it is evident that the energy efficiency prediction model based on BO-RF exhibits the best predictive performance. The SVR-based energy efficiency prediction model also demonstrates good predictive performance, albeit slightly inferior to the BO-RF model. However, the predictive performance of the BPNN-based energy efficiency prediction model is relatively poor. Analysis suggests that the BPNN model requires a large amount of sample data, and its predictive performance improves with a larger sample size. When predicting with small sample data, the RF and SVR models perform better.

CONCLUSION AND FUTURE WORK

In this study, we addressed the prediction problem of the relationship between processing parameters and energy consumption in FDM. We proposed a FDM energy consumption prediction model based on BO-RF. Through experimental validation from the perspectives of MSE, MPE, MAPE, and R², we compared the prediction results of the BO-RF energy consumption prediction model with those of nine other models. The results demonstrate the superiority of the BO-RF-based FDM energy consumption prediction model.

The proposed approach in this study is data-driven, which may have limitations in model interpretability. In future work, we will integrate data-driven and mechanistic knowledge-driven models to enhance model interpretability and improve predictive performance.