The efficacy of three-dimensional printing for plastic surgery education: a narrative review
0 
, ...
Abstract
Three-dimensional (3D) printed models offer potential advantages over traditional teaching methods by providing realistic, tactile learning aids. The overall efficacy of 3D printing in plastic surgery education has not been previously systematically analysed. A review of PubMed, Web of Science, and Embase databases up to October 2023 identified studies using 3D printed models in plastic surgery education. Inclusion criteria were set to select before-after studies or studies comparing 3D printed models to traditional teaching methods. Outcome measures included Likert scales, Multiple choice quest tests or other scoring systems. 37 studies met the inclusion criteria. Learners demonstrated enhanced anatomical understanding and procedural knowledge after engaging with 3D models. The comparative studies included in the review further highlight the superiority of 3D models over traditional learning tools, with average increases in test scores and procedural confidence, quantified through Likert scales and multiple-choice questionnaires. Ultimately, the findings of this review suggest that 3D printing enhances learning, making educational experiences more interactive and effective than traditional methods. While costs, accessibility, and a lack of technical expertise may pose challenges, integrating 3D models into training could enhance plastic surgical education. High-quality randomized controlled trials are necessary to confirm these findings and standardise outcomes for broader applications.
Keywords
INTRODUCTION
Initially intended for use in the aerospace industry, the introduction of additive manufacturing, also known as three-dimensional (3D) printing, has continued to transform and expand its applicability and utility into other fields, including medicine[1,2]. 3D printing has enhanced preoperative planning, enabling the precise fabrication of medical hardware like prostheses and implants, and fostering individualized patient care that leads to improved clinical outcomes[3-8]. Further, the integration of 3D models in healthcare has also improved the clinician’s capacity to predict complications, representing a significant advancement in patient care. Even still, given its capacity to produce high-fidelity anatomical models, 3D printing offers users tactile feedback and enhanced visuospatial comprehension, enabling it to be an instrumental tool for surgical and medical education[9,10].
Like other surgical fields, plastic surgery is a field characterized by a reliance on a comprehensive spatial understanding of human anatomy. Traditional educational tools, such as textbooks and cadaveric workshops, have limitations in terms of accessibility, ethical considerations, and the ability to represent complex 3D structures[10,11]. The introduction of 3D printed models offers a novel solution to these challenges, allowing for repeated practice, personalized learning experiences, and the bridging of gaps between theoretical knowledge and clinical application[11,12].
The literature indicates a growing interest in 3D printing for surgical education, with studies suggesting its benefits in enhancing comprehension, engagement, and practical skills among learners[7,13]. However, the body of evidence remains fragmented and heterogeneous in terms of methodologies, outcomes measured, and contexts applied[14-16]. To the best of our knowledge, there are no existing reviews that collate and synthesise this literature to provide a robust assessment of the educational value of modern 3D printing in plastic surgery. Hence, there is a rationale for the present review.
This study represents a narrative review evaluating the efficacy of using 3D printing as an educational tool in plastic surgery.
METHODS
Study identification
This study was performed as a narrative review with a systematic approach. The methodology largely adhered to the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) guidelines[17] [Figure 1] and was listed prospectively on the International Prospective Register of Systematic Reviews (PROSPERO). A comprehensive search was conducted to identify relevant studies on the use of 3D printing for plastic surgery education. The electronic databases of PubMed, Embase, and Web of Science were systematically searched from their inception until October 2023. The search strategy utilized a combination of relevant keywords and MeSH terms for a broad coverage of the literature. The following search terms were used: (“Three Dimension OR” “Three Dimensional” OR “Three-Dimension” OR “Three-Dimensional” OR “3D” OR “3-D” OR “Additive Manufacturing” OR “Rapid Prototyping” OR “Layered Manufacturing” OR “Stereolithography”) AND (“Print*” OR “Printing” OR “Printer” OR “Fabrication” OR “Building”) AND (“Plastic Surgery” OR “Cosmetic Surgery” OR “Reconstructive Surgery” OR “Aesthetic Surgery” OR “Craniofacial Surgery” OR “Hand Surgery” OR “Rhinoplasty” OR “Cleft” OR “Craniosynostosis” OR “Microsurgery” OR “Breast surgery” OR “Burns” OR “Peripheral nerve”) AND (“Education” OR “Training” OR “Teaching” OR “Learning” OR “Simulation”). In addition to the electronic database searches, the reference lists of all included studies and relevant reviews were thoroughly screened for any additional studies that may have been missed in the initial search.
Figure 1. PRISMA flowchart of included studies. PRISMA: Preferred Reporting Items for Systematic reviews and Meta-Analysis.
Study inclusion
Inclusion criteria included the following predefined criteria: Study design was limited to original randomised control, case-control, cohort studies, cross-sectional studies, and case series that evaluated the use of 3D printed anatomical models for use in plastic surgical education. Only studies that provided quantitative analysis of the efficacy of these models for teaching were included. These outcomes include the use of Likert scales prior and/or after the intervention, multiple choice quizzes, and other relevant scoring systems [e.g., Cleft Palate Objective Structured Assessment of Technical Skill (CLOSATS) or Global Rating Index for Technical Skills (GRITS) score]. Studies that compared the use of the intervention to controls were eligible for consideration. On the other hand, studies were excluded from the review if they were simple validation studies without quantitative assessment, case reports, reviews, conference presentations, editorials, letters to the editor, or if they did not report relevant outcomes. This review was limited to studies with human subjects.
Data extraction
Titles and abstracts were reviewed by two independent reviewers (JC/OSh) and discrepancies were resolved through discussion and consensus or the involvement of a third reviewer where necessary. Data were extracted into data extraction tables. Multiple data points were carefully extracted to provide a comprehensive overview of the included studies. These data points included the author of the study, year of publication, country of study origin, educational level of participants, details of the 3D model used as the intervention, any details of a relevant control group if included, the cost of models used, and lastly, a summary of the main findings/outcomes reported in the included literature.
Risk of bias/quality assessment
The risk of bias in included studies was assessed using the Cochrane Risk-of-Bias (RoB) tool for randomised control trials[18]. This tool consists of seven domains where bias could be introduced: random sequence generation, allocation concealment, blinding of participants/personnel, blinding of outcome assessors, incomplete outcome data, selective reporting, and other forms of bias. In each of these seven areas, the risk of bias is assessed as low, unclear, or high depending on criteria set by the Cochrane Collaboration[18]. Furthermore, the National Institute of Health quality assessment tool for Before-After studies with no control group was used to assess those without control groups[19]. This tool involves 12 questions as criteria that evaluate various aspects of the study. A single point was awarded for each criterion present in the assessed studies. A total of 1-4 points was deemed “poor”, 5-8 as “fair”, and 9-12 as “good” quality. Risk of bias and quality assessment were performed independently by two independent reviewers (JC/OSh), and any discrepancies were resolved through discussion and the involvement of a third reviewer if necessary.
RESULTS
Literature search
From the initial search, 2,135 studies were identified from the included databases. Following the removal of duplicates, 1,776 studies underwent an initial review of titles and abstracts. Of these, 137 met the criteria for full-text review. Following the full-text review, 29 publications were deemed eligible for inclusion. Furthermore, an additional 8 publications were found from citations of relevant literature and deemed eligible for inclusion. A total of 37 studies were included in the final analysis [Figure 1].
Study characteristics
Included studies were published between 2015-2023, with the majority from the United States (19 studies) and Canada (7 studies) [Tables 1 and 2]. Most studies focused on surgical residents as the learners (21 studies), while the remainder included medical students, fellows, and attending surgeons. The most common 3D printed models were used for education of surgical knowledge of cleft lip and palate (13 studies), followed by craniosynostosis (6 studies), rhinoplasty (5 studies), and otoplasty (4 studies) [Table 3]. Other models used for education included facial fractures, facial flaps, hand fractures, mandibular reconstruction, flexor tendon repair, and microsurgery training models. The cost of the utilised 3D models ranged widely from $0.55-$750 USD.
Characteristics of single-arm intervention studies
| Author | Year | Country | Educational Level | Details of 3D model | Cost (USD) | Main outcomes |
| D’Souza et al.[20] | 2015 | Canada | Plastic surgery residents | Facial fracture model for surgical education | NS | Use of the 3D model significantly improved diagnosis of fracture patterns (P < 0.001), choice of incisions for adequate exposure (P = 0.04) and the ability to identify the sequence of repair (P = 0.019) |
| Berens et al.[21] | 2016 | USA | Microtia surgeons | Costal cartilage model for ear reconstruction | $0.55-$0.60 | The starch/silicone model scored an average overall score of 4/5 for factors around operative simulation value and similarity to anatomical structures as based on a 5-point Likert scale |
| Alreefi et al.[15] | 2016 | Canada | Surgeons and surgical residents | Septoplasty model | $134 | Average scores of 4.05/5 for anatomic accuracy and 4.2/5 for operative realism as assessed on a 5-point Likert scale |
| Podolsky (A) et al.[22] | 2017 | Canada | Plastic surgery residents and fellows | Cleft palate model | NS | Significant improvement in knowledge as judged by improvement in multiple-choice test scores before and after intervention (P < 0.05) |
| Brichacek et al.[23] | 2018 | Canada | Plastic surgery residents | Hand model for Kirschner wire placement | $50 | Mean educational utility and overall usefulness of 5/5 as assessed by Likert scale by residents |
| Cheng (A) et al.[24] | 2018 | USA | Plastic surgery residents and fellow | Cleft palate model | NS | Significant improvements in procedural confidence (P < 0.05) and cleft palate knowledge (P = 0.05) following 3D model use |
| Cote et al.[25] | 2018 | USA | Trainees and surgeons | Cleft palate model | $7.31 | Mean educational utility of 4.8/5 as assessed by Likert scale by students |
| Eastwood et al.[26] | 2018 | Canada | Plastic surgery residents, fellows, and surgeons | Metopic and Sagittal craniosynostosis models | $195 | The model was deemed to provide a realistic representation of anatomy by greater than 80% of participants (mean Likert score of 4.21/5 for metopic model and 4.01/5 for sagittal synostosis model). Over 90% of participants felt the operative steps were realistic |
| Podolsky (B) et al.[16] | 2018 | Canada | Plastic surgery residents | Cleft palate model | NS | Increased scores on the CLOSATS by residents after each simulation session |
| Reighard (A) et al.[27] | 2018 | USA | Cleft surgeons | Cleft lip model | $11.43 | Global average rating of 2.4/5 and average scores of 3.8/5 for value as a training tool, and 3.2/5 for anatomical attributes as assessed on a 5-point Likert scale |
| Cheng (B) et al.[28] | 2019 | USA | Plastic surgery residents, fellows, and surgeons | Craniosynostosis models | NS | 100% of participants agreed that the model was a valuable training tool. 94% of residents reported that the model improved their understanding of the anatomy and operative steps of cranial vault reconstruction |
| Iida et al.[29] | 2019 | Japan | Plastic surgery residents | Cleft lip and alveolar cleft model | NS | Significant improvement in overall confidence in performing alveolar bone graft after using the model (P < 0.001). Significant improvements in anatomical knowledge (P < 0.05) and procedural confidence (P < 0.001) |
| Kantar et al.[30] | 2019 | USA | Plastic surgery residents | Cleft lip model | $1.84 | The model was deemed by the residents to offer significantly better learning of surgical markings (P < 0.001) than standard photographs. It was also significantly more interesting (P = 0.005) and more stimulating (P = 0.008) |
| Powell et al.[31] | 2019 | USA | Facial plastic surgeons | Facial flap model | $4.61-$8.14 | Average score of 3.86/4 on a 4-point Likert scale for value as a training/educational tool and 3.86/4 for anatomical realism |
| Riedle et al.[32] | 2019 | Germany | Plastic surgery residents | Cleft lip and palate model | NS | Average scores of 6.9/10 for anatomic accuracy and 6.1/10 for operative realism from participants after using the model on a 10-point scale |
| Oh et al.[33] | 2019 | USA | Surgical residents and surgeons | Rhinoplasty model | NS | Average scores of 3.7/5 for operative realism and 4.5/5 for value as an educational tool as assessed on a 5-point Likert scale* |
| Lobb et al.[34] | 2019 | USA | Plastic surgery residents | Craniosynostosis and mandibular distraction models | $281.61 | Number of inaccuracies in a written surgical plan decreased from 5 to 0 for sagittal synostosis and 4 to 0 for mandibular distraction after using the 3D models |
| Chang et al.[35] | 2020 | USA | Surgical residents and fellows | Microtia + cleft lip/palate models | NS | Use of the 3D printed model led to significant increases in self-confidence (P < 0.05) and self-reported expertise (P < 0.05) |
| Jovic et al.[36] | 2020 | UK | Plastic surgery trainees | Costal cartilage models for ear reconstruction | NS | Use of the model results in significant improvements in the understanding of the components of a “Firmin ear” (P < 0.0001). Significant improvements in procedural confidence (P < 0.0001) and understanding (P < 0.0001) |
| Reighard (B) et al.[37] | 2021 | USA | Surgeons | Mandibular distraction osteogenesis model | $19.56 | Average global rating of 2.67/4 and average scores of 4/4 for value as a training tool, and 3.38 for anatomical attributes as assessed on a 4-point Likert scale |
| Papavasiliou et al.[38] | 2021 | UK | Plastic surgery residents | Flexor tendon repair model | NS | Significant improvement in surgical knowledge and operative skills of all trainees after using the model (P < 0.01) |
| Gupta et al.[39] | 2021 | USA | Plastic surgery residents | Rhinoplasty model | NS | After using the model, residents reported improvements across multiple domains of the rhinoplasty procedure assessed based on a 5-point Likert scale |
| Nicholas et al.[40] | 2022 | UK | Plastic surgery trainees | Cleft palate model | $46 | Significant improvements in surgical knowledge and confidence after using the 3D model (P < 0.001) |
| Geoghegan et al.[41] | 2022 | UK | Plastic surgery residents | Microsurgery training model | NS | Significant improvements in overall technique (P = 0.0045) along with vessel preparation ability (P = 0.0035), and microsuturing skills (P = 0.0047), after use of the 3D model |
| Khoo et al.[42] | 2022 | Australia | Plastic surgery trainees | Cleft lip repair | $175 | Significant improvements in post-intervention operative confidence (P = 0.008) as assessed using a structured questionnaire and a 5-point Likert scale |
| Wright et al.[43] | 2023 | USA | Plastic surgery residents | Craniosynostosis model | $750 | Significant improvement in knowledge (P = 0.02) and technical skills (P < 0.001) among junior residents after training on the 3D model |
| Podolsky (C) et al.[44] | 2023 | Canada | Plastic surgeons | Rhinoplasty model | $275 | Overall, participants scored the model as 4.33/5 for anatomical realism and 4.85/5 for value as an educational tool on a 5-point Likert scale |
| Schlegel et al.[45] | 2023 | USA | Plastic surgeons | Rhinoplasty model | $4.54-$50.37 | Average scores ranged from 1.7-4.7/5 for overall educational value and 1.7-4.3/5 for overall anatomic realism as assessed on a 5-point Likert scale |
| Witsberger et al.[46] | 2023 | USA | Plastic surgeons | Otoplasty model | $3.99 | Average scores of 3.83/5 as a training tool, 3.83/5 as a competency evaluation tool and 4/5 as a rehearsal tool based on a 5-point Likert scale |
Characteristics of included studies with comparison groups
| Author | Year | Country | Educational Level | Intervention | Control | Cost (USD) | Main outcomes | Outcome measure |
| Alali et al.[14] | 2017 | UK | Medical students | Powerpoint presentation + cleft lip and palate model | Powerpoint presentation | $32 | Cohort of participants who used the 3D model demonstrated significantly improved knowledge gained (P = 0.038), visualisation of the anatomical defect (P = 0.001) and higher learning satisfaction (P = 0.005) compared to cohort relying on traditional teaching methods | MCQ |
| Lane and Black et al.[47] | 2020 | USA | Medical students | Powerpoint presentation + craniosynostosis model | Powerpoint presentation | $281.61 | Cohort of patients who were taught with the aid of the 3D printed model displayed better anatomical understanding (P = 0.0001) and visualisation (P = 0.0064). They also developed a significantly improved understanding of the surgical approach (P = 0.0266) | Likert MCQ |
| Hweidi et al.[48] | 2021 | Egypt | Plastic surgery trainees | Standard training* + cleft palate model | Standard training* | NS | Cohort of patients who practiced on the 3D model displayed significantly better operative technique (as assessed using the GRITS score) (P < 0.001) and reduced operative time (P < 0.001) | GRITS |
| Yang et al.[49] | 2021 | USA | Head and neck surgery trainees | Facial flap model | Paper-based illustration | NS | The group that utilised the 3D model displayed significant improvements in understanding and expertise of facial flaps (P < 0.05). No significant improvement was observed in the control group that utilised paper-based teaching methods | Likert |
| Al-Badri et al.[50] | 2022 | France | Medical students | Craniosynostosis 3D models | 2D images of skulls with craniosynostosis | NS | Significantly higher post-teaching test scores in the cohort of participants who utilised the 3D models compared to the cohort of participants who utilised the standard 2D teaching (P < 0.0001) | MCQ |
| Nicot et al.[51] | 2022 | France | Medical students | Facial fracture 3D models | 2D images of facial fractures | NS | Post-intervention test scores were significantly higher among the students who utilised the 3D model compared to the cohort who relied on standard 2D teaching (P = 0.008) | MCQ |
| Lerner et al.[52] | 2023 | USA | Medical students and Plastic surgery residents | Facial flaps model | Textbook chapter | $2.50 | Significant improvement in the performance of banner flap (P = 0.001) and bilobed flap (P = 0.001) by residents after using the model. Medical students who utilised the 3D printed model exhibited significantly improved knowledge and skills compared to those who did not (P < 0.05) | Likert |
| Rama et al.[53] | 2023 | USA | Surgical residents and fellows | Facial fracture 3D model + CT scan | CT Scan alone | NS | Significant improvement (P < 0.05) in participant confidence across three of the four tested 3D models in comparison to CT scans alone as assessed on a 5-point Likert scale |
Summary of educational areas utilising 3D printing among included studies
| Educational topic | Number of studies | Ref. |
| Cleft lip or palate | 13 | Podolsky et al. 2017[22] Alali et al. 2017[14] Cheng et al. 2018[24] Cote et al. 2018[25] Podolsky et al. 2018[16] Reighard et al. 2019[27] Iida et al. 2019[29] Kantar et al. 2019[30] Riedle et al. 2019[32] Chang et al. 2020[35] Nicholas et al. 2021[40] Khoo et al. 2022[42] Hweidi et al. 2021[48] |
| Craniosynostosis | 6 | Eastwood et al. 2018[26] Cheng et al. 2019[28] Lobb et al. 2019[34] Lane and Black, 2020[47] Wright et al. 2023[43] Al-badri et al. 2022[50] |
| Rhinoplasty | 5 | Alreefi et al. 2016[15] Oh et al. 2019[33] Gupta et al. 2021[39] Podolsky et al. 2023[44] Schlegel et al. 2023[45] |
| Ear reconstruction | 4 | Berens et al. 2016[21] Chang et al. 2020[35] Jovic et al. 2020[36] Witsberger et al. 2023[46] |
| Facial flaps | 3 | Powell et al. 2019[31] Yang et al. 2021[49] Lerner et al. 2023[52] |
| Facial fractures | 3 | D’Souza et al. 2015[20] Nicot et al. 2022[51] Rama et al. 2023[53] |
| Mandibular distraction | 2 | Reighard et al. 2021[37] Lobb et al. 2019[34] |
| Hand fractures | 1 | Brichacek et al. 2018[23] |
| Flexor tendon repair | 1 | Papavasiliou et al. 2021[38] |
| Microsurgery | 1 | Geoghegan et al. 2022[41] |
Single arm studies
The 28 single-arm pre-post studies evaluated a wide variety of educational outcomes before and after utilizing 3D-printed models. The most frequently assessed outcomes were procedural knowledge, anatomical understanding, technical skills, and trainee confidence [Table 1]. Outcome measures most often included Likert scale-based questionnaires of participants before and after using the 3D models. Furthermore, questionnaires or relevant scoring systems were also utilised in some studies to assess participants’ knowledge before and after the intervention.
Overall, all included studies reported positive outcomes from the use of the 3D printed models. Those that involved statistical analyses found that the use of 3D models led to significant improvements across at least one major educational domain. Trainees’ knowledge and understanding of anatomy, surgical steps, and/or techniques consistently showed significant gains after practicing surgical simulations on 3D printed models of various anatomical structures and defects. Additionally, studies that evaluated technical skills found significant improvements in operative abilities after hands-on simulation with 3D models.
Comparative studies
Seven of the included studies included the addition of a control group who utilised a form of traditional teaching and compared their outcomes to that of an intervention group utilising the 3D printed models [Table 2]. These comparative studies displayed similar consistency to the single-arm studies in demonstrating the superior educational value of 3D printed models over other traditional training modalities. Across different learners, 3D model training showed significantly greater improvements in knowledge, technical skills, and confidence compared to training with 2D illustrations, text/written materials, PowerPoint presentations, and other standard curricula.
Risk of bias
The risk of bias was most often judged as unclear or high across the included randomised control trials [Supplementary Figures 1 and 2]. Details of the random sequence generation or allocation concealment were often not reported in many studies, leaving room for bias. Blinding of participants was not possible given the nature of the interventions and was therefore deemed high-risk across all included studies. Blinding of outcome assessment was only definitively reported in one randomised control trial (RCT). Attrition and selective reporting bias were mostly deemed low-risk. The risk of bias in the Before-After studies lacking a comparison group as assessed using the NIH quality assessment tool is demonstrated in Table 4. Most studies were deemed to be of “fair” quality. Only two studies were of “good” quality. No studies justified their sample size or included multiple measurements before or after the intervention. Most studies were not blinded and did not clearly describe inclusion/exclusion criteria for participants that were applied a priori.
Quality assessment results using NIH Before-After Study quality assessment tool
| Author | Q1 | Q2 | Q3 | Q4 | Q5 | Q6 | Q7 | Q8 | Q9 | Q10 | Q11 | Q12 | Total | Grade |
| D’Souza et al.[20] | Y | Y | Y | N | N | Y | Y | N | Y | Y | N | N/A | 7 | Fair |
| Berens et al.[21] | Y | Y | Y | N | N | Y | N | Y | Y | N | N | N/A | 6 | Fair |
| Alreefi et al.[15] | Y | Y | Y | Y | N | Y | Y | Y | Y | Y | N | N/A | 9 | Good |
| Podolsky (A) et al.[22] | Y | Y | Y | N | N | Y | Y | N | Y | Y | N | N/A | 7 | Fair |
| Brichacek et al.[23] | Y | Y | Y | N | N | N | Y | N | Y | N | N | N/A | 5 | Fair |
| Cheng (A) et al.[24] | Y | Y | Y | N | N | N | Y | N | Y | Y | N | N/A | 6 | Fair |
| Cote et al.[25] | Y | N | Y | N | N | N | Y | N | Y | N | N | N/A | 4 | Poor |
| Eastwood et al.[26] | Y | Y | Y | Y | N | Y | Y | N | Y | Y | N | N/A | 8 | Fair |
| Podolsky (B) et al.[16] | Y | Y | Y | N | N | N | Y | Y | Y | Y | N | N/A | 7 | Fair |
| Reighard (A) et al.[27] | N | Y | Y | N | N | Y | N | N | Y | N | N | N/A | 4 | Poor |
| Cheng (B) et al.[28] | Y | Y | Y | N | N | Y | Y | N | Y | Y | N | N/A | 7 | Fair |
| Iida et al.[29] | Y | N | Y | N | N | N | N | N | Y | N | N | N/A | 3 | Poor |
| Kantar et al.[30] | Y | Y | Y | N | N | Y | Y | N | Y | Y | N | N/A | 7 | Fair |
| Powell et al.[31] | Y | N | Y | N | N | Y | N | N | Y | N | N | N/A | 4 | Poor |
| Riedle et al.[32] | Y | Y | Y | N | N | Y | Y | N | Y | N | N | N/A | 6 | Fair |
| Oh et al.[33] | Y | Y | Y | N | N | Y | Y | N | Y | Y | N | N/A | 7 | Fair |
| Lobb et al.[34] | Y | Y | Y | Y | N | Y | Y | Y | Y | N | N | N/A | 8 | Fair |
| Chang et al.[35] | Y | Y | Y | N | N | Y | Y | N | Y | Y | N | N/A | 7 | Fair |
| Jovic et al.[36] | Y | Y | Y | N | N | Y | N | N | Y | Y | N | N/A | 6 | Fair |
| Nicholas et al.[40] | Y | Y | Y | N | N | Y | Y | N | Y | Y | N | N/A | 7 | Fair |
| Reighard (B) et al.[37] | Y | N | Y | N | N | Y | N | N | Y | N | N | N/A | 4 | Poor |
| Papavasiliou et al.[38] | Y | Y | Y | Y | N | Y | Y | N | Y | Y | N | N/A | 8 | Fair |
| Gupta et al.[39] | Y | N | Y | N | N | Y | Y | N | Y | N | N | N/A | 5 | Fair |
| Geoghegan et al.[41] | Y | Y | Y | Y | N | Y | Y | Y | Y | Y | N | N/A | 9 | Good |
| Khoo et al.[42] | Y | Y | Y | N | N | Y | Y | N | Y | Y | N | N/A | 7 | Fair |
| Wright et al.[43] | Y | Y | Y | N | N | Y | Y | N | Y | Y | N | N/A | 7 | Fair |
| Podolsky (C) et al.[44] | Y | N | Y | N | N | Y | N | N | Y | N | N | N/A | 4 | Poor |
| Schlegel et al.[45] | Y | Y | Y | N | N | Y | N | Y | Y | N | N | N/A | 6 | Fair |
| Witsberger et al.[46] | Y | Y | Y | N | N | Y | Y | N | Y | N | N | N/A | 6 | Fair |
DISCUSSION
The findings of this review suggest that 3D printing may play a beneficial role in the education of plastic surgery trainees. Among the single-arm studies, training with 3D models led to statistically significant gains across knowledge, technical skills, confidence, and other educational domains pertinent to the specific procedures. These advantages were further corroborated by the comparative studies, where 3D printing showed superiority over standard teaching tools such as textbooks, 2D images, and presentations. These comparative studies highlighted that 3D models provide a more interactive and realistic experience, which can enhance understanding and retention of complex anatomical structures, ultimately leading to better preparedness in surgical settings.
The educational theory underpinning the value of 3D printing lies in its ability to promote multiple facets of learning. By offering a tangible representation of anatomical structures, 3D models promote an enriched learning experience. Unlike their two-dimensional counterparts, these models facilitate the exploration of complex spatial relationships more intuitively[54,55]. Engaging with these models allows trainees to interact with an anatomical replica that closely mirrors true anatomy, fostering a deeper understanding of structures[10,11,56]. Furthermore, the ability to practice procedures multiple times on accurately fabricated models caters to experiential learning, which is crucial for acquiring surgical competency. Through iterative practice, trainees can repeatedly perform surgical procedures, honing their skills in a risk-free setting[24,57]. This active learning fosters the development of surgical competencies prior to real-life application which is more readily retrievable during actual performance. As 3D printing technology advances and the materials more closely mimic human anatomy, or if bioprinting with real biological tissue emerges as a possible option, the educational utility of these models will continue to increase[58].
Our findings corroborate and build on previous reviews highlighting the promise of 3D printing for surgical education. Studies have found similar benefits across specialties like neurosurgery, otolaryngology, and colorectal surgery[59-61]. Leung et al. in 2022 reviewed the literature regarding the use of 3D printing in the field of otolaryngology for educational purposes[59]. They demonstrated from an extensive meta-analysis that 3D printing offered substantial surgical, anatomical, and educational value as a training tool[59]. Similarly, To et al. in 2023 demonstrated the emerging use of 3D printing in colorectal surgery[60]. They found that 3D printing offered objective benefits in anatomical education as assessed by academic outcomes at all academic levels[60]. Furthermore, they identified a growing role for surgical simulation using 3D printed models in the field of colorectal surgery[60]. The current study is the first review to provide a synthesis of the evidence for the efficacy of 3D printing specifically in the context of plastic surgery training. The overwhelmingly positive results are consistent with the findings outlined in previous literature documenting the benefits of this rapidly growing technology in other surgical fields.
Several implications can be derived from this review. 3D printing has substantial potential to supplement and enhance traditional curricula for plastic surgery education. The technology empowers programs to provide highly customisable and learner-focused experiences. Trainees can learn at their own pace, receiving iterative practice on specific case models tailored to their needs and learning gaps. Consequently, wider adoption of 3D printing can be recommended in plastic surgical training programs. However, certain barriers currently impede more ubiquitous integration. The main obstacles are costs, access, and lack of expertise. However, these limitations are being rapidly overcome as the costs continue to fall and access improves with the wider availability of printers[62]. In the current study, creating models was generally inexpensive, with only a minority of studies reporting costs over $100 USD per model. To accelerate adoption, training programs could consider investing in in-house desktop printers and recruiting personnel with skills to operate them. This may mitigate costs and reliance on external companies over the long term. There is also the challenge of requiring specialized technical knowledge to create and manipulate these models, which could limit their use to institutions with access to appropriate expertise and resources. Developing effective teaching methods that leverage the unique capabilities of 3D models also presents a substantial challenge. Instructors may need specific training to integrate these tools effectively into the curriculum, ensuring that the technology enhances rather than complicates the learning process. Moreover, it is often hypothesized that the more a model closely resembles its real-world counterpart, especially in terms of function and complexity, the more effective it is as a learning tool. This resemblance allows for a deeper understanding and retention of knowledge, as users can explore the intricacies of the model in a way that mirrors real-life scenarios. However, the more realistic a model, the more challenging and expensive it is to create; thus, models are often simple and rudimentary, limiting their similarity to real-world situations. Addressing these challenges is essential for maximizing the educational potential of 3D models and ensuring that their integration into medical education is both effective and sustainable.
While costs associated with 3D printing, particularly high-fidelity simulators, can often be prohibitive, alternative options include 3D computer images and augmented/virtual reality simulations[2,63,64]. 3D computer imaging provides a highly dynamic and financially accessible option. These virtual models can be manipulated in real time, allowing users to explore complex anatomical details from multiple angles. This capability not only facilitates a deeper understanding of spatial relationships critical for surgical training but also eliminates the material costs associated with physical models[2,63]. Augmented reality (AR) and virtual reality (VR) technologies further extend these benefits by creating immersive learning environments that simulate surgical procedures and scenarios[56,65-67]. AR overlays digital information onto the real world, enhancing the learning experience by integrating virtual components with physical tools, which is particularly useful in surgical training[67]. VR, on the other hand, immerses the user in a completely virtual environment, offering an interactive experience that can replicate various surgical settings and procedures without the need for physical resources[67]. Both technologies can reduce costs by circumventing the need for repeated model fabrications and updates. The cost-effectiveness of AR and VR is significant, especially when considering their scalability and the ability to update and distribute new content without additional material costs. Moreover, these technologies can be implemented on a broader scale across different institutions without the logistical challenges associated with physical models.
Further, the continued advancements in artificial intelligence (AI) can enhance the capacity of AR and VR technology to deliver advanced surgical education. AI has the potential to provide real-time feedback, personalized training programs, and adaptive learning experiences[68]. Machine learning algorithms can evaluate a user’s performance, identify areas for improvement, and adjust the difficulty of simulations to match the skill level of the trainee[69,70]. In addition, AI-powered virtual assistants can guide users through complex procedures, providing[71] detailed instructions and ensuring compliance with best practices[2]. Therefore, educational programs employing these advanced digital simulations can provide consistent training experiences, standardize educational content, and facilitate remote learning opportunities.
This narrative review encountered several significant limitations that must be acknowledged. Chief among these was the predominance of pre-post single-arm and non-randomised studies, which inherently carry a high risk of selection and confirmation bias. Furthermore, the inability to blind participants in these studies further exacerbates this issue, introducing performance bias, where the knowledge of intervention may influence outcomes. Moreover, the assessment of outcomes was compromised by suboptimal blinding of outcome assessors in many studies, leading to the potential for detection bias. The heterogeneity of the study designs, populations, interventions, and reported outcomes significantly hampered the ability to perform a comprehensive quantitative synthesis of the data, such as a meta-analysis, which limits the comparability of studies and the pooling of data for a more powerful aggregate effect estimate. Additionally, there is a concern regarding publication bias, as studies with positive results are more likely to be published, and this review found a preponderance of unequivocally positive outcomes, which may not reflect the true balance of evidence. Despite these limitations, the consistency of positive findings across various models, learners, and outcomes provides some reassurance. However, future research must include higher quality evidence from randomised, blinded studies with rigorous methodological designs to overcome these limitations and provide a more reliable evidence base to inform practice.
CONCLUSION
3D printing appears to offer significant advantages for plastic surgery education. It enhances the learning experience by providing tactile feedback and a three-dimensional perspective that conventional teaching methods cannot offer. However, the integration of 3D printing into medical curricula requires careful planning, resources, and training of faculty to maximize its potential. Future research should aim to conduct high-quality studies with a low risk of bias and standardised outcomes to better understand the impact of this technology. As demonstrated in this review, the potential of 3D printing in the realm of plastic surgery education is vast and could offer significant benefits to the training of the next generation of surgeons.
DECLARATIONS
Authors’ contributions
Made substantial contributions to the conception and design of the study: Cevik J, Shadid O, Hornby A, Pang S, Salehi O, Seth I, Rozen WM
Performed data acquisition or corroboration: Cevik J, Shadid O, Hornby A, Pang S, Salehi O
Performed data analysis and interpretation: Cevik J, Shadid O, Hornby A, Seth I
Writing initial draft: Cevik J, Shadid O, Hornby A
Critical manuscript review and writing editing: Cevik J, Shadid O, Pang S, Salehi O, Seth I, Rozen WM
Provided leadership, administrative, technical, and material support: Rozen WM
Availability of data and materials
Not applicable.
Financial support and sponsorship
None.
Conflicts of interest
Rozen WM is the Guest Editor of the Special Issue “The Application of 3D Printing Technology in Plastic Surgery”, while the other authors have declared that there are no conflicts of interest.
Ethical approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Copyright
© The Author(s) 2024.
Supplementary Materials
REFERENCES
1. Aimar A, Palermo A, Innocenti B. The role of 3D printing in medical applications: a state of the art. J Healthc Eng 2019;2019:5340616.
2. Cevik J, Seth I, Hunter-Smith DJ, Rozen WM. A history of innovation: tracing the evolution of imaging modalities for the preoperative planning of microsurgical breast reconstruction. J Clin Med 2023;12:5246.
3. Chae MP, Chung RD, Smith JA, Hunter-Smith DJ, Rozen WM. The accuracy of clinical 3D printing in reconstructive surgery: literature review and in vivo validation study. Gland Surg 2021;10:2293-303.
4. Chae MP, Hunter-Smith DJ, Chung RD, Smith JA, Rozen WM. 3D-printed, patient-specific DIEP flap templates for preoperative planning in breast reconstruction: a prospective case series. Gland Surg 2021;10:2192-9.
5. Cevik J, Hunter-Smith DJ, Rozen WM. Current advances in breast reconstruction. J Clin Med 2022;11:3328.
6. Choi JW, Kim N. Clinical application of three-dimensional printing technology in craniofacial plastic surgery. Arch Plast Surg 2015;42:267-77.
7. Lynn AQ, Pflibsen LR, Smith AA, Rebecca AM, Teven CM. Three-dimensional printing in plastic surgery: current applications, future directions, and ethical implications. Plast Reconstr Surg Glob Open 2021;9:e3465.
8. Han HH, Shim JH, Lee H, et al. Reconstruction of complex maxillary defects using patient-specific 3D-printed biodegradable scaffolds. Plast Reconstr Surg Glob Open 2018;6:e1975.
9. Bauermeister AJ, Zuriarrain A, Newman MI. Three-dimensional printing in plastic and reconstructive surgery: a systematic review. Ann Plast Surg 2016;77:569-76.
10. Langridge B, Momin S, Coumbe B, Woin E, Griffin M, Butler P. Systematic review of the use of 3-dimensional printing in surgical teaching and assessment. J Surg Educ 2018;75:209-21.
11. Salazar D, Thompson M, Rosen A, Zuniga J. Using 3D printing to improve student education of complex anatomy: a systematic review and meta-analysis. Med Sci Educ 2022;32:1209-18.
12. Chae MP, Hunter-Smith DJ, Rostek M, Smith JA, Rozen WM. Enhanced preoperative deep inferior epigastric artery perforator flap planning with a 3D-printed perforasome template: technique and case report. Plast Reconstr Surg Glob Open 2018;6:e1644.
13. Claudic Y, Zopf DA, Ozkan M, Francia RD, Hu W. Current use of 3D printing in plastic surgery. Ann 3D Print Med 2023;11:100119.
14. AlAli AB, Griffin MF, Calonge WM, Butler PE. Evaluating the use of cleft lip and palate 3D-printed models as a teaching aid. J Surg Educ 2018;75:200-8.
15. AlReefi MA, Nguyen LH, Mongeau LG, et al. Development and validation of a septoplasty training model using 3-dimensional printing technology. Int Forum Allergy Rhinol 2017;7:399-404.
16. Podolsky DJ, Fisher DM, Wong Riff KW, et al. Assessing technical performance and determining the learning curve in cleft palate surgery using a high-fidelity cleft palate simulator. Plast Reconstr Surg 2018;141:1485-500.
17. Page MJ, McKenzie JE, Bossuyt PM, et al. The prisma 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71.
18. Higgins JPT, Altman DG, Gøtzsche PC, et al; Cochrane Bias Methods Group, Cochrane Statistical Methods Group. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928.
19. National Institute of Health. Study quality assessment tools. Available from: https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools. [Last accessed on 14 Aug 2024].
20. D’Souza N, Mainprize J, Edwards G, Binhammer P, Antonyshyn O. Teaching facial fracture repair: a novel method of surgical skills training using three-dimensiona biomodels. Plast Surg 2015;23:81-6.
21. Berens AM, Newman S, Bhrany AD, Murakami C, Sie KC, Zopf DA. Computer-aided design and 3D printing to produce a costal cartilage model for simulation of auricular reconstruction. Otolaryngol Head Neck Surg 2016;155:356-9.
22. Podolsky DJ, Fisher DM, Wong KW, Looi T, Drake JM, Forrest CR. Evaluation and implementation of a high-fidelity cleft palate simulator. Plast Reconstr Surg 2017;139:85e-96e.
23. Brichacek M, Diaz-Abele J, Shiga S, Petropolis C. Three-dimensional printed surgical simulator for kirschner wire placement in hand fractures. Plast Reconstr Surg Glob Open 2018;6:e1706.
24. Cheng H, Podolsky DJ, Fisher DM, et al. Teaching palatoplasty using a high-fidelity cleft palate simulator. Plast Reconstr Surg 2018;141:91e-8e.
25. Cote V, Schwartz M, Arbouin Vargas JF, et al. 3-Dimensional printed haptic simulation model to teach incomplete cleft palate surgery in an international setting. Int J Pediatr Otorhinolaryngol 2018;113:292-7.
26. Eastwood KW, Bodani VP, Haji FA, Looi T, Naguib HE, Drake JM. Development of synthetic simulators for endoscope-assisted repair of metopic and sagittal craniosynostosis. J Neurosurg Pediatr 2018;22:128-36.
27. Reighard CL, Green K, Rooney DM, Zopf DA. Development of a novel, low-cost, high-fidelity cleft lip repair surgical simulator using computer-aided design and 3-dimensional printing. JAMA Facial Plast Surg 2019;21:77-9.
28. Cheng D, Yuan M, Perera I, et al. Developing a 3D composite training model for cranial remodeling. J Neurosurg Pediatr 2019;24:632-41.
29. Iida C, Sakamoto Y, Ohara H, Ogata H, Kishi K. Simulation training for alveolar cleft closure using a double-size solid model. J Plast Reconstr Aesthet Surg 2020;73:783-808.
30. Kantar RS, Gonchar M, Maliha SG, et al. Three-dimensional affordable stone models for cleft lip markings: a prospective study of learner satisfaction. Ann Plast Surg 2019;83:340-3.
31. Powell AR, Srinivasan S, Green G, Kim J, Zopf DA. Computer-aided design, 3-D-printed manufacturing, and expert validation of a high-fidelity facial flap surgical simulator. JAMA Facial Plast Surg 2019;21:327-31.
32. Riedle H, Burkhardt AE, Seitz V, et al. Design and fabrication of a generic 3D-printed silicone unilateral cleft lip and palate model. J Plast Reconstr Aesthet Surg 2019;72:1669-74.
33. Oh CJ, Tripathi PB, Gu JT, Borden P, Wong BJ. Development and evaluation of rhinoplasty spreader graft suture simulator for novice surgeons. Laryngoscope 2019;129:344-50.
34. Lobb DC, Cottler P, Dart D, Black JS. The use of patient-specific three-dimensional printed surgical models enhances plastic surgery resident education in craniofacial surgery. J Craniofac Surg 2019;30:339-41.
35. Chang B, Powell A, Ellsperman S, et al. Multicenter advanced pediatric otolaryngology fellowship prep surgical simulation course with 3D printed high-fidelity models. Otolaryngol Head Neck Surg 2020;162:658-665.
36. Jovic TH, Combellack EJ, Jessop ZM, Whitaker IS. Using 3D Printing technology to teach cartilage framework carving for ear reconstruction. Front Surg 2020;7:44.
37. Reighard CL, Powell AR, Zurawski TY, Rooney DM, Keilin CA, Zopf DA. Development of a novel mandibular distraction osteogenesis simulator using Computer Aided Design and 3D printing. Int J Pediatr Otorhinolaryngol 2021;142:110616.
38. Papavasiliou T, Nicholas R, Cooper L, et al. Utilisation of a 3D printed ex vivo flexor tendon model to improve surgical training. J Plast Reconstr Aesthet Surg 2022;75:1255-60.
39. Gupta N, Fitzgerald CM, Ahmed MT, Tohidi S, Winkler M. Feasibility of a 3D printed nasal model for resident teaching in rhinoplasty. J Plast Reconstr Aesthet Surg 2021;74:2776-820.
40. Nicholas R, Heinze Z, Papavasiliou T, et al. Educational impact of a novel cleft palate surgical simulator: improvement in surgical trainees’ knowledge and confidence. J Plast Reconstr Aesthet Surg 2022;75:3817-25.
41. Geoghegan L, Papadopoulos D, Petrie N, Teo I, Papavasiliou T. Utilization of a 3D printed simulation training model to improve microsurgical training. Plast Reconstr Surg Glob Open 2023;11:e4898.
42. Khoo R, Ryan E, Vijayasekaran V, Gillett DA. Cleft lip repair simulation: an Australian trainee experience. AJOPS 2023;6:1-8.
43. Wright JM, Ford JM, Qamar F, et al. Design and validation of a 3D printed cranio-facial simulator: a novel tool for surgical education. Cleft Palate Craniofac J 2024;61:997-1006.
44. Podolsky DJ, Murphy BD, Ahmad J, et al. Development and evaluation of a high-fidelity rhinoplasty simulator. Plast Reconstr Surg 2024;153:619-22.
45. Schlegel L, Malani E, Belko S, et al. Design, printing optimization, and material testing of a 3D-printed nasal osteotomy task trainer. 3D Print Med 2023;9:20.
46. Witsberger CA, Michaels R, Monovoukas D, et al. Development of a high-fidelity, 3D printed otoplasty surgical simulator. Ann Otol Rhinol Laryngol 2023;132:607-13.
47. Lane JC, Black JS. Modeling medical education: the impact of three-dimensional printed models on medical student education in plastic surgery. J Craniofac Surg 2020;31:1018-21.
48. Hweidi AS, Massoud K, Zidane A, Ghanem M. A novel 3D model for training on basics of cleft palate surgery: development and validation study. Egypt J Plast Reconstr Surg 2021;45:7-14.
49. Yang SF, Powell A, Srinivasan S, et al. Addressing the pandemic training deficiency: filling the void with simulation in facial reconstruction. Laryngoscope 2021;131:E2444-8.
50. Al-Badri N, Touzet-Roumazeille S, Nuytten A, Ferri J, Charkaluk ML, Nicot R. Three-dimensional printing models improve long-term retention in medical education of pathoanatomy: a randomized controlled study. Clin Anat 2022;35:609-15.
51. Nicot R, Druelle C, Chazard E, et al. Three-dimensional printing model enhances craniofacial trauma teaching by improving morphologic and biomechanical understanding: a randomized controlled study. Plast Reconstr Surg 2022;149:475e-84e.
52. Lerner JL, Vishwanath N, Borrelli MR, Rao V, Crozier J, Woo AS. A cost-effective, 3D printed simulation model facilitates learning of bilobed and banner flaps for mohs nasal reconstruction: a pilot study. Plast Reconstr Surg 2024;154:358e-61e.
53. Rama M, Schlegel L, Wisner D, et al. Using three-dimensional printed models for trainee orbital fracture education. BMC Med Educ 2023;23:467.
54. Ford S, Minshall T. Invited review article: where and how 3D printing is used in teaching and education. Addit Manuf 2019;25:131-50.
55. Ford S, Minshall T. 3D printing in education: a literature review. 2016. Available from: https://www.researchgate.net/profile/Simon-Ford-3/publication/308204531_3D_printing_in_education_a_literature_review/links/5c196d35a6fdccfc70586e3a/3D-printing-in-education-a-literature-review.pdf. [Last accessed on 14 Aug 2024].
56. Chae MP, Hunter-smith DJ, Rozen WM. Imaging and printing in plastic and reconstructive surgery part 2: emerging techniques. Australas J Plast Surg 2019;2:69-77.
57. Li K, Kui C, Lee E, et al. The role of 3D printing in anatomy education and surgical training: a narrative review. MedEdPublish 2017;6:92.
58. Garcia J, Yang Z, Mongrain R, Leask RL, Lachapelle K. 3D printing materials and their use in medical education: a review of current technology and trends for the future. BMJ Simul Technol Enhanc Learn 2018;4:27-40.
59. Leung G, Pickett AT, Bartellas M, et al. Systematic review and meta-analysis of 3D-printing in otolaryngology education. Int J Pediatr Otorhinolaryngol 2022;155:111083.
60. To G, Hawke JA, Larkins K, et al. A systematic review of the application of 3D-printed models to colorectal surgical training. Tech Coloproctol 2023;27:257-70.
61. Blohm JE, Salinas PA, Avila MJ, Barber SR, Weinand ME, Dumont TM. Three-dimensional printing in neurosurgery residency training: a systematic review of the literature. World Neurosurg 2022;161:111-22.
62. Tack P, Victor J, Gemmel P, Annemans L. 3D-printing techniques in a medical setting: a systematic literature review. Biomed Eng Online 2016;15:115.
63. Diaz-Siso JR, Plana NM, Stranix JT, Cutting CB, McCarthy JG, Flores RL. Computer simulation and digital resources for plastic surgery psychomotor education. Plast Reconstr Surg 2016;138:730e-8e.
64. Cevik J, Rozen W. A novel optimization technique of computed tomography angiographic 3D-reconstructions for pre-operative planning of DIEP flaps. JPRAS Open 2023;35:38-41.
65. Chae MP, Hunter-smith DJ, Rozen WM. Imaging and printing in plastic and reconstructive surgery part 1: established techniques. Australas J Plast Surg 2019;2:15-28.
66. Kaplan N, Marques M, Scharf I, et al. Virtual reality and augmented reality in plastic and craniomaxillofacial surgery: a scoping review. Bioengineering 2023;10:480.
67. Cheng L, Matinrazm S, Kass N, et al. Augmented reality in plastic and reconstructive surgery: what it is, how far it’s come, and the limitations impacting further adoption. Plast Reconstr Surg Glob Open 2023;11:112-3.
68. Cevik J, Seth I, Rozen WM. Transforming breast reconstruction: the pioneering role of artificial intelligence in preoperative planning. Gland Surg 2023;12:1271-5.
69. Guerrero DT, Asaad M, Rajesh A, Hassan A, Butler CE. Advancing surgical education: the use of artificial intelligence in surgical training. Am Surg 2023;89:49-54.
70. Winkler-Schwartz A, Bissonnette V, Mirchi N, et al. Artificial intelligence in medical education: best practices using machine learning to assess surgical expertise in virtual reality simulation. J Surg Educ 2019;76:1681-90.
Cite This Article
Open AccessHow to Cite
Cevik, J.; Shadid O.; Hornby A.; Pang S.; Salehi O.; Seth I.; Rozen W. M. The efficacy of three-dimensional printing for plastic surgery education: a narrative review. Plast. Aesthet. Res. 2024, 11, 39. http://dx.doi.org/10.20517/2347-9264.2024.65
Download Citation
Export Citation File:
Type of Import
Tips on Downloading Citation
Citation Manager File Format
Type of Import
Direct Import: When the Direct Import option is selected (the default state), a dialogue box will give you the option to Save or Open the downloaded citation data. Choosing Open will either launch your citation manager or give you a choice of applications with which to use the metadata. The Save option saves the file locally for later use.
Indirect Import: When the Indirect Import option is selected, the metadata is displayed and may be copied and pasted as needed.
About This Article
Special Issue
Copyright
Data & Comments
Data
0
Comments
Comments must be written in English. Spam, offensive content, impersonation, and private information will not be permitted. If any comment is reported and identified as inappropriate content by OAE staff, the comment will be removed without notice. If you have any queries or need any help, please contact us at support@oaepublish.com.