The current state of artificial intelligence in robotic esophageal surgery

INTRODUCTION

The treatment of esophageal malignancy is complex and requires coordination among a multidisciplinary team including gastroenterologists, medical and radiation oncologists, and thoracic surgeons. Similar to its applications in lung cancer, artificial intelligence (AI) and artificial neural networks (ANNs) are increasingly used to guide clinicians in all stages of esophageal cancer management, from diagnosis to non-operative management and surgical intervention^[1-4]. The algorithms are trained on large amounts of clinical data and customized to solve unique clinical problems in all stages of cancer care. From a screening perspective, AI has been employed to supplement thorough endoscopic esophageal exams in order to identify premalignant and malignant conditions^[4]. The utilization of machine learning (ML) with Barrett’s esophagus is in its infancy, but the growth of this application is imperative to avoid missed high-risk lesions and interval cancers. As more surgeons adopt the robotic-assisted minimally invasive esophagectomy (RAMIE) technique, AI is increasingly integrated into the operating room to reduce the surgical learning curve, enhance safety and efficiency, and improve postoperative outcomes. In this review, we will explore the current applications for AI in the realm of robotic esophageal surgery, which, while still in its infancy, has shown great potential as a tool for esophageal surgeons.

SCREENING AND PREOPERATIVE DECISION MAKING

Screening and early detection of esophageal premalignant lesions is imperative for early endoscopic and surgical treatments. AI has been applied to diagnostic upper endoscopies in aiding endoscopists in early detection and screening for premalignant and malignant esophageal diseases. Applications include computer-aided detection based on endoscopic images, deep learning algorithms of histologic specimens, and real-time video analysis. These AI algorithms demonstrate high sensitivity in identifying high-risk esophageal lesions and may enhance the traditional esophageal exam using high-definition white light and narrow band imaging^[5,6]. Miss rates for esophageal adenocarcinoma and Barrett’s esophagus with existing biopsy strategies are estimated to be as high as > 20% and 50%, respectively. Narrow band imaging and more advanced imaging techniques such as chromoendoscopy have improved diagnostic accuracy for endoscopists. However, they require greater expertise^[6]. Deep learning algorithms have shown promise as a novel adjunct to endoscopists in identifying high-risk esophageal lesions.

Computer-aided detection systems trained on large subsets of white light endoscopy images were able to identify Barrett’s esophagus with exceptionally high sensitivity and specificity with near-perfect localization of the disease^[7-9]. A similar approach has been applied using ANN analysis of endoscopic videos and yielded similar detection rates of esophageal dysplasia^[10]. ML algorithms applied to histologic specimen slides can identify and differentiate non-dysplastic Barrett’s esophagus, low-grade dysplasia, and high-grade dysplasia with > 90% sensitivity and specificity^[11,12]. Similar approaches have been applied to the detection of malignancy with some neural network systems shown to detect gastroesophageal junction cancer on traditional white light endoscopy images with 66% accuracy, compared to an accuracy of 63% when analyzed by board-certified expert endoscopists^[13]. AI-based tools have also been employed to ensure quality and consistency during screening endoscopy by providing automatic image capture and blind-spot recognition during screening routine endoscopy. In a randomized controlled trial, 153 patients were randomized to include real-time quality improvement system during routine screening endoscopy compared to 150 routine controls. In the experimental group, the quality of screening endoscopy was significantly improved including an approximate 15% reduction in blind spot rate [95% confidence interval (CI) -19.23 to -11.54]^[14]. This quality improvement system, called WISENSE, employed deep convolutional neural networks (DCNN) and deep reinforcement learning (DRL) trained on over 34,000 EGD images to classify gastric images into specific sites. Training this model involved first testing DCNN on still images and eventually integrating DCNN and DRL for testing on real EGD videos. After this initial training process, the system was formally tested in a randomized control trial.

As medical care has become increasingly individualized, AI has been integrated into the medical decision-making process for staging, prognosticating and treating premalignant and malignant esophageal lesions. Current estimations for progression of Barrett’s esophagus and surveillance intervals have been based on previously published large studies. In part due to interobserver variability in endoscopic and histologic evaluation, interval cancers have been seen with what would typically be considered “low risk” lesions. As a result of this alarming issue, a tissue systems pathology test (TissueCypher Barrett’s Esophagus Test, TSP-9) was developed and incorporates AI with biomarkers in an effort to predict the individual’s true risk of Barrett’s progression^[15]. As patient response to therapy is variable, AI tools can guide multidisciplinary treatment plans and optimize patient outcomes. Rice et al., in their study using ML to analyze optimal treatment for esophageal cancer, found that 61% of patients who underwent esophagectomy alone received optimal treatment. In contrast, only 36% of patients who received neoadjuvant therapy were deemed to have received optimal therapy. These estimates were based on random forest technology in a sequential analysis. Optimal therapy was classified as the treatment modality that, after survival analysis using patient and cancer characteristics, maximized lifetime as measured by restricted mean survival time^[16]. Another large study on 418 patients with esophageal adenocarcinoma undergoing surgery with curative intent utilized 65 variables from individualized patient data and used ANNs to predict one- and five-year survival. These variables spanned the full spectrum of care from symptoms at presentation to postoperative data such as tumor gene expression. Two ANNs were developed and compared against a linear discriminant analysis (LDA) to assess accuracy. Their results showed that the ANNs were more accurate than the LDA models and were superior to a model based solely on the tumor-node-metastasis (TNM) staging criteria when predicting survival^[17].

Other groups have used AI to predict a patient’s responsiveness to chemotherapy. Using ANNs, real time polymerase chain reaction assays of pre- and post-treatment esophageal cancer specimens were analyzed for 17 genes in an attempt to predict histopathologic tumor response to chemoradiation. Not only did the analysis identify specific independent risk factors in the study population, it also outperformed univariate and multivariate analysis in terms of predicting response to treatment. The analysis of these gene expression arrays could predict tumor response to traditional neoadjuvant therapy with 85% accuracy^[18]. Alternatively, ANNs trained on 18-fluorodeoxyglucose positron emission tomography (PET) scans from 107 patients with esophageal cancer were able to predict chemotherapy non-responders with > 80% sensitivity/specificity. Ypsilantis et al. used an “radiomics” approach where large amounts of quantitative data were extracted from pretreatment PET images to compile a tumor phenotype and employed an ANN to learn from intra-tumor slices seen on PET scans. Their model was able to extract PET image representations that could predict non-responders to treatment with 80.7% sensitivity and 81.6% specificity^[19]. Oftentimes, non-responders to neoadjuvant chemotherapy have a worse prognosis compared with those individuals treated with upfront surgery^[19]. This study highlights the potential use of AI in non-invasively predicting cancer treatment response which would allow for a more personalized approach to esophageal cancer patients.

INTRAOPERATIVE AND POSTOPERATIVE SURGICAL CARE

From a surgeon’s perspective, there are a number of hypothesized uses for AI applications to current surgical techniques including intraoperative support, surgical training, and postoperative care^[20]. AI systems have potential for intraoperative utility by way of anatomic structure identification. As outlined in a meta-analysis by Anteby et al., ANNs have shown the ability to harness unlabeled laparoscopic footage to achieve precise tasks such as anatomy detection, instrument identification, action recognition, and surgical phase categorization^[21]. Analysis of ANN performance in quantifying data from laparoscopic videos showed sensitivity as high as 95%, with a limitation being the heterogeneous nature of their pooled data from multiple different procedure types^[22].

Similar techniques have been applied to RAMIEs where ANNs were trained to identify key anatomic structures including the azygos vein, superior vena cava, aorta, lung parenchyma, or the recurrent laryngeal nerve^[22,23]. Sato et al. created an AI model to identify the recurrent laryngeal nerve after training on 2,000 static images extracted from 20 operative recordings. In each image, the recurrent laryngeal nerve was labeled by expert thoracic surgeons. Using the Dice coefficient to assess performance, the AI model outperformed general surgeons in identifying the recurrent laryngeal nerve and was only slightly less accurate than expert thoracic surgeons (Dice coefficient 0.58 vs. 0.62)^[23]. The previous studies highlight real-time, autonomous intraoperative anatomical segmentation as a promising application for AI systems. While further research is undoubtedly necessary, existing literature suggests a strong potential for the expansion of AI and deep learning applications in the realm of robotic thoracic surgery.

AI has been utilized to minimize the steep learning curve of robotic surgery and bridge the gap between inexperienced and experienced surgeons. One study, which used Temporal Convolutional Networks for the Operating room (TeCNO), sought to develop an AI-based phase recognition system for RAMIE^[24]. Video was incorporated from 31 RAMIE procedures and κ-fold cross-validation to train their model to recognize nine pre-determined surgical phases. By analyzing intraoperative recordings, their model was able to identify RAMIE phases with 84% accuracy. For trainees, automatic phase recognition provides objective data about surgical timing and efficiency for streamlined review and teaching. Intraoperatively, this information can be used to alert support staff to an operation’s current goals or needs. Automatic phase recognition is likely the first step in making truly autonomous platforms and sets the foundation for innovation and future robotic applications.

Postoperatively, there are multiple ML models that have been developed to predict patients at risk for complications following esophagectomy. ML algorithms can predict anastomotic leak rates with high sensitivity based on various patient characteristics with an area-under-the-receiver-operator curve (AUC) of 0.72-0.87^[25,26]. Another model based on over 2,000 esophagectomy patients could predict early readmission with AUC of 0.72-0.74^[27]. In a study of 864 patients with distal esophageal adenocarcinoma undergoing Ivor-Lewis Esophagectomy, an ANN was developed to predict clinically significant complications based on Clavien-Dindo classification. Based on 96 variables encompassing all phases of care, this model could predict Clavien-Dindo IIIa and above complications with an AUC of 0.67. It was also capable of discriminating between medical and surgical complications with AUCs of 0.70 and 0.66, respectively^[28].

CONCLUSION

AI has emerged as a compelling tool for endoscopists, medical oncologists, and thoracic surgeons in the management of premalignant esophageal conditions and esophageal malignancy. While still in its infancy, it is clear that it will play an important role in assisting surgeons to complete RAMIEs safely and efficiently. Additionally, the multidisciplinary nature of esophageal malignancy offers multiple avenues for AI and ANN implementation including computer-aided detection on screening/surveillance endoscopy, predicting rates of progression of premalignant esophageal lesions or forecasting responses to medical-oncologic interventions. AI will be an important adjunct in optimizing patient outcomes by implementing predictive algorithms regarding preoperative and postoperative care. In concert with ongoing developments in robotic surgical platforms, current ML systems have set the foundation for future surgical innovation, which will continue to shape the field of thoracic surgery.