Laparoscopic skills training of surgical residents: a comparison of two proficiency-based independent approaches
Abstract
Aim: Current financial and work hour constraints make proctored on-site laparoscopic simulation training challenging. An independent learning approach utilizing proficiency-based training is a potential solution. The purpose of this study was to determine if an independent approach using a portable, laparoscopic training device within one’s home environment could effectively train novices in laparoscopic procedural skills.
Methods: After baseline testing, laparoscopic novices (n = 16) were randomized to one of two study groups. The on-site group (n = 7) received unlimited access to the workplace laparoscopic trainers and the home group (n = 9) received portable laparoscopic trainers for home. Both groups underwent self-directed, proficiency-based training for three months then were retested. Results were compared with parametric and non-parametric statistical tests.
Results: Baseline characteristics were similar between groups. The practice rate (56%) and practice time (range, 0.18 to 2.6 h) were poor in both groups during the training period. At post-test, the number of participants who demonstrated an improvement (86% on-site, 78% home) on the peg task was not different between groups. The successful completion of the suturing task post-test had significantly improved compared with pre-test in both groups (71% vs. 29% on-site; 44% vs. 22% home, P < 0.001). Although the majority of participants reported it was difficult to practice on a regular basis (86% on-site, 89% home), 56% of the home group participants agreed that the at-home trainer was a helpful teaching modality.
Conclusion: Learning of laparoscopic skills by novice trainees can be augmented by an independent learning approach using either home or on-site laparoscopic trainers. Although over half the candidates found it was useful to have the training device at home, none of the participants practiced more than an hour or two in the three month training period. Thus, the solution to conducting training does not lie in merely providing home training, but rather to understand the work-related stressors and reconfigure jobs.
Keywords
Introduction
To maximize the benefit of time in the operating room, laparoscopic skills training outside the operating room has become the gold standard for educating surgical residents. It is well documented that residents benefit from this form of training and can lead to improved operative performance. In fact, training is an integral and essential component of a trainee’s job. However, there is no consensus regarding the optimal method or learning environment for teaching laparoscopic skills in order to maximize trainees’ education while maintaining an equitable and sustainable work-life balance. Recently, several studies have suggested that mandatory, proctored, proficiency-based goal-directed training is the best method for training.[1-3] However, this approach is not without disadvantages. For example, this type of training requires a significant amount of practice time per trainee in a workplace environment where shortened work hours are being mandated.[4-6] This laborious process of proctored, on-site training programs is time consuming and significantly impinges on the residents’ pre-existing didactic requirements, patient care responsibilities, and overall operative experience.[7-9] In fact, duty-hour limitations have led to a significant re-evaluation of the traditional surgical education paradigm, with emphasis on increased efficiency of educational efforts. Accordingly, a disadvantage of these on-site training facilities is that they are often not readily accessible to the resident. Unless dedicated training time is made mandatory for residents, few residents independently take advantage of these costly facilities during their already busy week.[10] If educators seek to facilitate laparoscopic learning, they should provide the trainees with the opportunities for extracurricular practice. Finally, there is also an economical aspect to consider as the use of surgical educators can be costly, time consuming, and difficult to achieve for smaller institutions.
An independent learning approach where trainees are given self-study resources (i.e. video/didactic tutorials) as well as unlimited access to the on-site training facility and encouraged to practice at their own pace is one alternative strategy.[6] The advantages of such self-directed training method are reduced instructor time and simplified scheduling. Moreover, several studies have shown that learning can be facilitated if learners are able to self-direct their own training experience.[11-13] However, in the United States, the training time on-site within the workplace environment is counted toward the already restricted resident duty hours. A hybrid approach that employs an independent learning approach using at-home, portable laparoscopic trainer outside the time constraints of the hospital environment, has not been fully investigated. As medical educators and mentors, the best methods of teaching laparoscopic skills and the most efficacious learning environment where trainees feel motivated and allocate a high priority to practice needs to be the standard for resident education. Clearly, laparoscopic training should not be considered after exhaustion from work-related activities. In fact, there is evidence to suggest that motor training during periods of exhaustion can deteriorate and may create an environment that promotes poor technique.[14] This is precisely why home training may be beneficial. It would be portable, accessible, convenient, flexible and inexpensive once initiated. Indeed, trainees could practice the laparoscopic skills at their preferred time, for example when are well-rested and away from work stressors, and not at a time that fits others’ schedules. Therefore, we hypothesized that an at-home independent, proficiency-based laparoscopy training method using a low-cost device will improve the laparoscopic skills of novice trainees and that this training method-environment combination would be equivalent to traditional, independent hospital-based, laparoscopic training. The primary endpoint of the study was to compare trainee performance before and after at home practice training.
Methods
Participants
Novice laparoscopists [First and Second Post Graduate Year (PGY-1 and 2) general surgery residents, (n = 8) and medical students (n = 9), at any year of training] were recruited into the study through word of mouth at Tulane University Medical Center. Prior to beginning training, all participants completed a questionnaire assessing demographics and previous exposure to laparoscopy. The Tulane University Institutional Review Board approved this study, and all subjects gave informed consent prior to participation.
Apparatus
The Minimally Invasive Surgical Trainer (Joystick SimScope™; 3D Med, Franklin, Ohio) was used in this study for the home training device. This training device is a self-contained, lightweight (15 lbs), and portable box equipped with a camera and 10 inch LCD color monitor, which offered good visual resolution, and optics with all the close-up and rotation options. The trainer is also equipped with seven ports fitted with grommets designed to hold an instrument or a trocar. Tasks were carried out with the monitor at eye level and the laparoscopic instruments at a standard surgical height between the monitor and the participant.
The on-site “standard” conventional trainer (Karl Storz Endoscopy, Culver City, CA) used in this study included a 15 inch video monitor (Sony Corporation, New York, NY), Xenon-nova light source, Telecam SL camera system, Hopkins II laparoscope and a Plexiglas box trainer. Tasks were carried out using a 0° Strorz 10-mm laparoscope connected to a light source and with the images directed to the Sony television monitor.
Training and testing protocol
After an orientation and viewing a introductory video on the “Fundamentals of Laparoscopic Surgery” (FLS) peg board transfer and intra-corporal suturing and knot tying tasks,[15] all trainees completed a baseline assessment (pre-test) on these tasks using the on-site “standard” video-trainer. The peg transfer and an FLS-type video-trainer laparoscopic suturing model were used to assess their baseline skills. Performance scores were calculated and recorded for each peg transfer attempt using time (s) and for each laparoscopic suturing and knot tying attempt using the previously published formula: 600 – [time (s) + 10* accuracy error + 10* security error].[16] If there was failure to complete the later task (i.e. tie a functional knot) within the time limit (10 min), the task was terminated and the participant was given a score of 0. Baseline performance was defined as the mean score of the first three repetitions at the beginning of training (in the absence live, proctored instruction). Subjects were then ranked according to the sum of the overall scores for the three attempts, stratified into blocks of two and randomized into two groups. On-site group received unlimited 24-h access to the on-site skills laboratory for independent practice. Home group received a self-contained, portable laparoscopic - minimally invasive training system box (Joystick SimScope™; 3D Med, Franklin, Ohio).
Both groups were then allowed to self-direct their training for a three months period of time. During this independent training period, both groups were given access and allowed unlimited viewing of the didactic tutorials on the two tasks. The peg transfer model, an FLS-type video-trainer laparoscopic suturing model, and 6-inch pre-cut 3-0 silk sutures were provided to each participant. All participants were given previously established task specific proficiency levels for the peg transfer (48 s)[17] and the laparoscopic suturing model (score 512)[16] at the start of the training period to guide practice. To further foster goal-directed learning, all participants were encouraged to train as long as they needed in their spare time until they reached the pre-defined proficiency criterion. Both groups were given a journal to record number of practice days, and time spent practicing on each task over the 3-month independent training period. After training completion, all participants underwent repeated evaluation (post-test) on the same initial two laparoscopic tasks using the on-site “standard” video-trainer.
Questionnaire
Each participant completed a questionnaire on the educational experience, at the completion of the study investigating the perceived benefit of the training method. One set of issues concerned the opportunity for practice during the study period while another concerned the usefulness of the home training device in term of the learning of surgical skills. Participants in the home group were asked to evaluate using 10-point Likert scale (1-10) their satisfaction regarding the home training with higher numbers being more positive responses.
Statistics
Data are expressed as means ± standard error of the mean. Comparisons of the pre-training and the post-training continuous variables for each domain within groups were performed using a Wilcoxon matched pairs test. Comparisons of continuous variables between groups were conducted by using an un-paired two-tailed t-test. Categorical variables were compared using Fisher’s exact or chi square test. Computer software (GraphPad Instat software, San Diego CA) was used for all statistical analyses. A P value of less than 0.05 was considered significant.
Results
Study population
Seventeen subjects were enrolled, but one subject (medical student) dropped out secondary medical reasons during the training period, and this individual was not included in the final study analysis. Therefore, the subsequent analysis was per protocol. The mean age of the study population was 31.0 ± 1.5 years (range 24-47 years). Nine subjects were female (52.9%) and 15 were right-hand dominant (88.2%). There were no significant differences in age, gender, or self-reported laparoscopic experience, between groups. Moreover, trainee baseline simulator performances for the two groups were equal (P > 0.05).
Training period
As a group, only 56 % (n = 9/16) of the participants actually practiced the laparoscopic tasks during the training period (n = 4 in on-site group, n = 5 in home group). Of the participants that practiced (50% of medical students, 40% of PGY-1, and 33% of PGY-2 residents) only one subject (on-site group; medical student) practiced on a regular basis with a total training time of over 29 h. For the other participants who practiced the average total recorded training time was just 1.0 h (range 0.18-2.6 h), over 5 different days (range 1-15 days) during the entire 3-month training period.
The task-specific practice was as follows: for the pegboard task, only one subject (a medical student) in the on-site training group actually trained on this task during the training period. By comparison, 4 out of 9 (44.4%) subject in the home group practice this task. For the suturing task, nine people (56%) practiced during the 3-month training period (4 in the on-site group with total training time 5.0 ± 4.3 h; 5 in the home group with total training time 1.0 ± 0.8 h).
Post-test skills assessment performance
Since the number of participants in each group that actually practiced during the study period was small, this situation precluded any meaningful statistical breakdown for this subgroup. Therefore, for the post-test analysis all participants (i.e. those who did and did not train at all) were included. Interestingly, most participants achieved improvements regardless of which group they were initially assigned for training or the amount of practice they recorded. For the pegboard task, post-training times for the participants in the on-site group improved on average 65 s (37%) compared to pretest scores (177.7 ± 23.8 s pre vs. 112.2 ± 9.3 s post, P = 0.047). By comparison, the home group improved on average 41 s (22%), compared to pre-test scores (183 ± 21.5 vs. 142.9 ± 16.6 s, P = 0.039). The most marked improvement was noted with one participant improving the pegboard time by 196 s (on-site group). Interestingly, the number of participants who improved on the pegboard task (86% vs. 78%) and the average time to task completion after training was not statistically significantly between the home and on-site groups, respectively (P = 0.47). Importantly, despite these improvements no participants, in either group, achieved proficiency at re-testing for the pegboard task.
As expected, the more complex task (suturing) provided the greater training challenge, however an improvement was noted in both groups. The suturing task completion rate for both groups had significantly improved after the training period (71% vs. 29% on-site group; P < 0.001 and 44% vs. 22% home group; P < 0.001). Finally, at re-test, the on-site group score improved by an average of 113 points (114.9 ± 74.6 pre vs. 228.4 ± 83.6 post, P > 0.05) compared to 39 points for the group trained at home on the portable device (80 ± 53.5 pre vs. 118.7 ± 60.2 post, P > 0.05 Wilcoxon matched pairs test). Although the magnitude of improvement on the suturing task was significantly greater for the on-site trained group on both forms of assessment, no statistically significant difference could be demonstrated (P = 0.54). Only one participant from the on-site group achieved proficiency on the suturing task. However, this individual practiced significantly more than all other participants (18 h) during the independent training period.
End-of-study questionnaire
In total, 93% reported no or minimal exposure to laparoscopic surgery during the study period. The ease of use of the home training device was evaluated using a 10-point Likert scale with anchored end points; 1 being easy and 10 being difficult. Many respondents felt that the home box trainer was easy to use with a median score of 2. According to half of participants, the study experience was beneficial to their laparoscopic skills education. Importantly, regardless of the training location, a majority of each group stated they had difficulty practicing regularly (86% in on-site group, 89% in home group). The 44% who do not practice cited the following reasons: lack of time (57%), away rotations (29%), and the remaining 14% cited various other reasons. Of the 56% who did practice still cited a lack of time (56%) as the major reason they did not practice more often.
Discussion
Laparoscopic training is an integral and essential component of a surgical trainee’s job. However, within a changing surgical environment alternative methods for laparoscopic training must be sought for training which incorporate opportunities to practice. In this un-blinded, randomized study, two proficiency-based independent approaches were employed to teach laparoscopic skills to beginners, with one tactic employing standard on-site physical box trainers at the workplace and the other relying on a similar device used in the trainee’s home, outside the stress of the work environment. Both methods allowed trainees to practice their laparoscopic skills at their own pace. In the beginning, pre-training skills were homogenous with minimal baseline experience in the two groups. In the end, novice trainees showed improvement in their laparoscopic skills using our self-directed, proficiency-based home training program. However, at most, it was comparable to our on-site program in terms of feasibility and rates of participation. Several studies have shown that learning can be facilitated if learners are able to self-direct their own training experience[11-13] while other studies have shown that low cost, portable training device can improve laparoscopic skills.[18,19] However, the ability of a self-directed laparoscopic training experience using low-cost portable training devices to improve laparoscopic procedural skills with the home environment has not been investigated. We used a proficiency-based, self-directed training approach both within the home and workplace environment. Both groups were instructed to practice at their discretion to achieve expert proficiency on each task with no ramifications if they did not succeed. We believed this would be successful based on the observation that surgical trainees and those interested in surgery are highly motivated to learn the required skills and invest the necessary time. In addition, there seems to be natural selection within the surgical population itself. Trainee motivation is essential for learning because it promotes deliberate practice and persistent efforts to improve objective performance.[20] Without motivated learners, any educational efforts will have limited achievements and the skills laboratories will be attended infrequently. At the end of the day, our method was met with mixed results. First of all, the self-directed training approach did enhance the novices’ laparoscopic skills in both groups. But, we also observed that few subjects attended the on-site skill laboratory during their independent training period. More surprising, was the observation that when subjects were given the training device to use at home, they still did not find the time to frequently train independently. Interestingly, on average, the home trainees found only about 1-2 h to practice within a three-month period of time and 44% never practiced once during this period. These results demonstrated that one of the most important factors in getting trainees to practice and hone their laparoscopic skills was providing them with enough free time during their work week not changing their learning environment. This ultimately disproved our original hypothesis that moving practice opportunities out of the busy and stressful clinical environment to “free time” would be beneficial. Thus the hard, but unavoidable truth appears to be that there are barriers to practice in both settings that need to be better understood.
The fact that the groups demonstrated improvement despite low recorded practice rates during the training period may indicate that this is due to random effects, rather than due to the amount of deliberate practice. However, there are many different components for procedural skills learning besides the time spent practicing on physical simulator with specific tasks such as utilizing didactic, and video-based instruction, reflection and supervised practice with feedback and formative assessment.[21-25] In our study, all subjects most likely received learning from the video-based instruction and during the practice for the period of the pre-test scoring. Indeed, video-based instruction has been shown to be efficacious in the development of laparoscopic skills.[26,27] It has been recently demonstrated that under prescribed practice conditions, video based instruction is equally effective as faculty instruction in teaching basic surgical skills to novice trainees.[23,24,28]
Supervised practice-feedback is believed to be another foundation of effective learning. Feedback and formative assessment refer to information about performance that is intended to guide learning. The purpose of giving feedback is to encourage learners to think about their performance and how they might improve. However, feedback in practice is often vague and evaluative (e.g. “good suturing”).[29] While direct intensive practice feedback was not given in our study, the use of pre-set task-specific proficiency criteria to guide practice was utilized. This has been shown to give the informative feedback and opportunity for error correction vital for deliberative practice and can improve laparoscopic training.[6] This may be another reason for the observed skill improvements in both groups.
Our study has several limitations and must be viewed cautiously as they may not apply for other simulators or other subjects with different motivation, interests, and backgrounds. If all of our subjects were surgical residents that were required to achieve proficiency prior to being allowed into the operating room, we believe that nearly 100% would attain the pre-defined proficiency levels as opposed to the 6% noted in the current study.[30] Another limitation of our study was that we assumed that the participants knew how to self-direct their learning. Perhaps with more oversight, including feedback and good practice reinforcement, the path to greater task improvements would have been identified. Another limitation was that the number of participants in our study was small. Interestingly, for both tasks the greater improvements were with the on-site training - although statistical significance between the two groups could not be demonstrated this may be a function of small numbers. A larger cohort might result in statistically significant differences between the two groups. A further limitation of this study is it did not attempt to correlate improvement in task performance with improved performance in live human operations. Despite these limitations, this study has provided the foundation for additional assessment of the home trainer as a means of improving operative performance.
Essentially, our investigation is a feasibility study evaluating if home training is viable, however it was not performed to determine if programs can “avoid obligations” and “negate” work hours regulations. Our study found that novice individuals trained on either the home or the on-site training device were able to improve their laparoscopic skills objectively and subjectively using an independent proficiency-based training method. However, none of the participants practiced for more than 1 or 2 h in the 3-month training period. Therefore, we feel that the only valid conclusion that can be drawn is that when overworked and exhausted trainees who are expected to train on their own time, do not regularly engage in training exercises, regardless of whether these are provided at home or in the workplace. The solution to conducting training in a stressful work environment may not be to simply “suggest” that trainees practice at home, but rather to understand the work-related stressors and reconfigure jobs, and perhaps even to increase staffing, to minimize stress exposure.
Declarations
Authors’ contributionsStudy design and manuscript preparation: C.F. Bellows
Manuscript preparation and data analysis: A.A. Smith
AcknowledgmentsThe authors thank Cheri Touchard and Steven Au for their help with the conduct of this study.
Financial support and sponsorshipNone.
Conflicts of interestThere are no conflicts of interest.
Patient consentAll study subjects provided appropriate consent as per the Institutional review board protocol prior to participating in the study.
Ethics approvalInstitutional review board approval was obtained prior to initiation of the study.
REFERENCES
1. Madan AK, Harper JL, Taddeucci RJ, Tichansky DS. Goal-directed laparoscopic training leads to better laparoscopic skill acquisition. Surgery 2008;144:345-50.
2. Gauger PG, Hauge LS, Andreatta PB, Hamstra SJ, Hillard ML, Arble EP, Kasten SJ, Mullan PB, Cederna PS, Minter RM. Laparoscopic simulation training with proficiency targets improves practice and performance of novice surgeons. Am J Surg 2010;199:72-80.
3. Rosenthal ME, Ritter EM, Goova MT, Castellvi AO, Tesfay ST, Pimentel EA, Hartzler R, Scott DJ. Proficiency-based fundamentals of laparoscopic surgery skills training results in durable performance improvement and a uniform certification pass rate. Surg Endosc 2010;24:2453-7.
4. Scott DJ, Bergen PC, Rege RV, Laycock R, Tesfay ST, Valentine RJ, Euhus DM, Jeyarajah DR, Thompson WM, Jones DB. Laparoscopic training on bench models: better and more cost effective than operating room experience? J Am Coll Surg 2000;191:272-83.
5. Gonzalez R, Bowers SP, Smith CD, Ramshaw BJ. Does setting specific goals and providing feedback during training result in better acquisition of laparoscopic skills? Am Surg 2004;70:35-9.
6. Snyder CW, Vandromme MJ, Tyra SL, Hawn MT. Proficiency-based laparoscopic and endoscopic training with virtual reality simulators: a comparison of proctored and independent approaches. J Surg Educ 2009;66:201-7.
8. Damadi A, Davis AT, Saxe A, Apelgren K. ACGME duty-hour restrictions decrease resident operative volume: a 5-year comparison at an ACGME-accredited university general surgery residency. J Surg Educ 2007;64:256-9.
9. Pape HC, Pfeifer R. Restricted duty hours for surgeons and impact on residents quality of life, education, and patient care: a literature review. Patient Saf Surg 2009;3:3.
10. Chang L, Petros J, Hess DT, Rotondi C, Babineau TJ. Integrating simulation into a surgical residency program: is voluntary participation effective? Surg Endosc 2007;21:418-21.
11. Brydges R, Carnahan H, Safir O, Dubrowski A. How effective is self-guided learning of clinical technical skills? It's all about process. Med Educ 2009;43:507-15.
12. Wulf G, Raupach M, Pfeiffer F. Self-controlled observational practice enhances learning. Res Q Exerc Sport 2005;76:107-11.
13. Safir O, Dubrowski A, Hui Y, Backstein D, Carnahan H. Self-directed practice scheduling is equivalent to instructor guided practice when learning a complex surgical skill. Procedia Social Behavioral Sci 2010;2:792-6.
14. Leff DR, Aggarwal R, Rana M, Nakhjavani B, Purkayastha S, Khullar V, Darzi AW. Laparoscopic skills suffer on the first shift of sequential night shifts: program directors beware and residents prepare. Ann Surg 2008;247:530-9.
15. Fundamentals of Laparoscopic Surgery. Available from: https://www.flsprogram.org/. [Last accessed on August 16, 2017].
16. Korndorffer JR Jr, Dunne JB, Sierra R, Stefanidis D, Touchard CL, Scott DJ. Simulator training for laparoscopic suturing using performance goals translates to the operating room. J Am Coll Surg 2005;201:23-9.
17. Ritter EM, Scott DJ. Design of a proficiency-based skills training curriculum for the fundamentals of laparoscopic surgery. Surg Innov 2007;14:107-12.
18. Chung SY, Landsittel D, Chon CH, Ng CS, Fuchs GJ. Laparoscopic skills training using a webcam trainer. J Urol 2005;173:180-3.
19. Sharpe BA, MacHaidze Z, Ogan K. Randomized comparison of standard laparoscopic trainer to novel, at-home, low-cost, camera-less laparoscopic trainer. Urology 2005;66:50-4.
20. Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 2004;79:S70-81.
21. Jowett N, LeBlanc V, Xeroulis G, MacRae H, Dubrowski A. Surgical skill acquisition with self-directed practice using computer-based video training. Am J Surg 2007;193:237-42.
22. Mann K, Gordon J, MacLeod A. Reflection and reflective practice in health professions education: a systematic review. Adv Health Sci Educ Theory Pract 2009;14:595-621.
23. Summers AN, Rinehart GC, Simpson D, Redlich PN. Acquisition of surgical skills: a randomized trial of didactic, videotape, and computer-based training. Surgery 1999;126:330-6.
24. Xeroulis GJ, Park J, Moulton CA, Reznick RK, Leblanc V, Dubrowski A. Teaching suturing and knot-tying skills to medical students: a randomized controlled study comparing computer-based video instruction and (concurrent and summary) expert feedback. Surgery 2007;141:442-9.
25. Ford JK, Kraiger K. The application of cognitive constructs and principles to the instructional systems of model training: implications for needs assessment, design and transfer. In: Cooper CL, Robertson IT, eds. International Review of Industrial and Organizational Psychology. Chichester, England: Wiley; 1995. pp. 1-48.
26. Rosser JC, Herman B, Risucci DA, Murayama M, Rosser LE, Merrell RC. Effectiveness of a CD-ROM multimedia tutorial in transferring cognitive knowledge essential for laparoscopic skill training. Am J Surg 2000;179:320-4.
27. Takiguchi S, Sekimoto M, Yasui M, Miyata H, Fujiwara Y, Yasuda T, Yano M, Monden M. Cyber visual training as a new method for the mastery of endoscopic surgery. Surg Endosc 2005;19:1204-10.
28. Stefanidis D, Korndorffer JR Jr, Heniford BT, Scott DJ. Limited feedback and video tutorials optimize learning and resource utilization during laparoscopic simulator training. Surgery 2007;142:202-6.
Cite This Article
How to Cite
Bellows, C. F.; Smith, A. A. Laparoscopic skills training of surgical residents: a comparison of two proficiency-based independent approaches. Mini-invasive. Surg. 2017, 1, 126-32. http://dx.doi.org/10.20517/2574-1225.2017.12
Download Citation
Export Citation File:
Type of Import
Tips on Downloading Citation
Citation Manager File Format
Type of Import
Direct Import: When the Direct Import option is selected (the default state), a dialogue box will give you the option to Save or Open the downloaded citation data. Choosing Open will either launch your citation manager or give you a choice of applications with which to use the metadata. The Save option saves the file locally for later use.
Indirect Import: When the Indirect Import option is selected, the metadata is displayed and may be copied and pasted as needed.
Comments
Comments must be written in English. Spam, offensive content, impersonation, and private information will not be permitted. If any comment is reported and identified as inappropriate content by OAE staff, the comment will be removed without notice. If you have any queries or need any help, please contact us at support@oaepublish.com.