1
|
Ryu S, Imaizumi Y, Goto K, Iwauchi S, Kobayashi T, Ito R, Nakabayashi Y. Artificial intelligence-enhanced navigation for nerve recognition and surgical education in laparoscopic colorectal surgery. Surg Endosc 2025; 39:1388-1396. [PMID: 39762611 PMCID: PMC11794642 DOI: 10.1007/s00464-024-11489-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2024] [Accepted: 12/14/2024] [Indexed: 02/06/2025]
Abstract
BACKGROUND Devices that help educate young doctors and enable safe, minimally invasive surgery are needed. Eureka is a surgical artificial intelligence (AI) system that can intraoperatively highlight loose connective tissues (LCTs) in the dissected layers and nerves in the surgical field displayed on a monitor. In this study, we examined whether AI navigation (AIN) with Eureka can assist trainees in recognizing nerves during colorectal surgery. METHODS In left-sided colorectal surgery (n = 51, between July 2023 and February 2024), Eureka was connected to the laparoscopic system side by side, and the nerve was highlighted on the monitor during the surgery. We examined the rate of failure to recognize nerves by trainee surgeons over a total of 101 scenarios after it was recognized intraoperatively by the supervising surgeon (certified by the Japanese Society of Endoscopic Surgery). We also examined the frequency of nerve recognition by the trainee physicians viewing the Eureka monitor when recognition was not possible (recognition assistance rate). RESULTS The nerve recognition failure rate and recognition assistance rate with AIN were as follows: right hypogastric nerve during sigmoid colon mobilization, 44/101 (43.6%) and 19/44 (43.2%); left hypogastric nerves during dissection of the dorsal rectum, 27/101 (26.7%) and 13/27 (48.1%); right lumbar splanchnic nerves, 32/101 (31.7%) and 29/32 (90.6%); left lumbar splanchnic nerves, 44/101 (43.6%) and 39/44 (88.6%); and pelvic visceral nerves during dissection of the dorsal rectum, 29/45 (64.4%) and 6/29 (20.7%), respectively. CONCLUSION Although the rate of recognition with assistance from AIN differed for the different nerves, this system can potentially assist in anatomic recognition, enhance surgical education, and contribute to nerve preservation. TRIAL REGISTRATION Improvement of AI navigation in minimally invasive surgery and examination of its intraoperative support and educational effectiveness. Research Ethics Committee of the Kawaguchi Municipal Medical Center (Saitama, Japan) approval number: 2022-27.
Collapse
Affiliation(s)
- Shunjin Ryu
- Department of Digestive Surgery, Kawaguchi Municipal Medical Center, Kawaguchi City, Saitama, 180, Nishiaraijuku333-0833, Japan.
| | - Yuta Imaizumi
- Department of Digestive Surgery, Kawaguchi Municipal Medical Center, Kawaguchi City, Saitama, 180, Nishiaraijuku333-0833, Japan
| | - Keisuke Goto
- Department of Digestive Surgery, Kawaguchi Municipal Medical Center, Kawaguchi City, Saitama, 180, Nishiaraijuku333-0833, Japan
| | - Sotaro Iwauchi
- Department of Digestive Surgery, Kawaguchi Municipal Medical Center, Kawaguchi City, Saitama, 180, Nishiaraijuku333-0833, Japan
| | - Takehiro Kobayashi
- Department of Digestive Surgery, Kawaguchi Municipal Medical Center, Kawaguchi City, Saitama, 180, Nishiaraijuku333-0833, Japan
| | - Ryusuke Ito
- Department of Digestive Surgery, Kawaguchi Municipal Medical Center, Kawaguchi City, Saitama, 180, Nishiaraijuku333-0833, Japan
| | - Yukio Nakabayashi
- Department of Digestive Surgery, Kawaguchi Municipal Medical Center, Kawaguchi City, Saitama, 180, Nishiaraijuku333-0833, Japan
| |
Collapse
|
2
|
Nakajima Y, Nemoto D, Guo Z, Boyuan P, Ruiyao Z, Katsuki S, Takezawa T, Maemoto R, Kawasaki K, Inoue K, Akutagawa T, Tanaka H, Sato K, Omori T, Hayashi Y, Miyakura Y, Matsumoto T, Yoshida N, Esaki M, Uraoka T, Kato H, Inoue Y, Yamamoto H, Zhu X, Togashi K. Differences in regions of interest to identify deeply invasive colorectal cancers: Computer-aided diagnosis vs expert endoscopists. Endosc Int Open 2024; 12:E1260-E1266. [PMID: 39524197 PMCID: PMC11543284 DOI: 10.1055/a-2401-6611] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Accepted: 08/23/2024] [Indexed: 11/16/2024] Open
Abstract
Background and study aims Diagnostic performance of a computer-aided diagnosis (CAD) system for deep submucosally invasive (T1b) colorectal cancer was excellent, but the "regions of interest" (ROI) within images are not obvious. Class activation mapping (CAM) enables identification of the ROI that CAD utilizes for diagnosis. The purpose of this study was a quantitative investigation of the difference between CAD and endoscopists. Patients and methods Endoscopic images collected for validation of a previous study were used, including histologically proven T1b colorectal cancers (n = 82; morphology: flat 36, polypoid 46; median maximum diameter 20 mm, interquartile range 15-25 mm; histological subtype: papillary 5, well 51, moderate 24, poor 2; location: proximal colon 26, distal colon 27, rectum 29). Application of CAM was limited to one white light endoscopic image (per lesion) to demonstrate findings of T1b cancers. The CAM images were generated from the weights of the previously fine-tuned ResNet50. Two expert endoscopists depicted the ROI in identical images. Concordance of the ROI was rated by intersection over union (IoU) analysis. Results Pixel counts of ROIs were significantly lower using 165K[x103] [108K-227K] than by endoscopists (300K [208K-440K]; P < 0.0001) and median [interquartile] of the IoU was 0.198 [0.024-0.349]. IoU was significantly higher in correctly identified lesions (n = 54, 0.213 [0.116-0.364]) than incorrect ones (n=28, 0.070 [0.000-0.2750, P = 0.033). Concusions IoU was larger in correctly diagnosed T1b colorectal cancers. Optimal annotation of the ROI may be the key to improving diagnostic sensitivity of CAD for T1b colorectal cancers.
Collapse
Affiliation(s)
- Yuki Nakajima
- Department of Gastroenterology, Aizu Medical Center, Fukushima Medical University, Aizuwakamatsu, Japan
| | - Daiki Nemoto
- Department of Coloproctology, Aizu Medical Center, Fukushima Medical University, Aizuwakamatsu, Japan
| | - Zhe Guo
- Biomedical Information Engineering Lab, The University of Aizu, Aizuwakamatsu, Japan
| | - Peng Boyuan
- Biomedical Information Engineering Lab, The University of Aizu, Aizuwakamatsu, Japan
| | - Zhang Ruiyao
- Biomedical Information Engineering Lab, The University of Aizu, Aizuwakamatsu, Japan
| | - Shinichi Katsuki
- Department of Gastroenterology, Otaru Ekisaikai Hospital, Otaru, Japan
| | - Takahito Takezawa
- Department of Medicine, Division of Gastroenterology, Jichi Medical University, Shimotsuke, Japan
| | - Ryo Maemoto
- Department of Surgery, Saitama Medical Center, Jichi Medical University, Saitama, Japan
| | - Keisuke Kawasaki
- Department of Gastroenterology, Iwate Medical University, Morioka, Japan
| | - Ken Inoue
- Department of Molecular Gastroenterology and Hepatology, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Takashi Akutagawa
- Division of Gastroenterology, Department of Internal Medicine, Faculty of Medicine, Saga University, Saga, Japan
| | - Hirohito Tanaka
- Department of Gastroenterology and Hepatology, Gunma University Graduate School of Medicine, Maebashi, Japan
| | - Koichiro Sato
- Department of Clinical Laboratory and Endoscopy, Tokyo Women's Medical University Medical Center East, Tokyo, Japan
| | - Teppei Omori
- Institute of Gastroenterology, Tokyo Women's Medical University, Tokyo, Japan
| | - Yoshikazu Hayashi
- Department of Medicine, Division of Gastroenterology, Jichi Medical University, Shimotsuke, Japan
| | - Yasuyuki Miyakura
- Department of Surgery, Saitama Medical Center, Jichi Medical University, Saitama, Japan
| | - Takayuki Matsumoto
- Department of Gastroenterology, Iwate Medical University, Morioka, Japan
| | - Naohisa Yoshida
- Department of Molecular Gastroenterology and Hepatology, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Motohiro Esaki
- Division of Gastroenterology, Department of Internal Medicine, Faculty of Medicine, Saga University, Saga, Japan
| | - Toshio Uraoka
- Department of Gastroenterology and Hepatology, Gunma University Graduate School of Medicine, Maebashi, Japan
| | - Hiroyuki Kato
- Department of Clinical Laboratory and Endoscopy, Tokyo Women's Medical University Medical Center East, Tokyo, Japan
| | - Yuji Inoue
- Institute of Gastroenterology, Tokyo Women's Medical University, Tokyo, Japan
| | - Hironori Yamamoto
- Department of Medicine, Division of Gastroenterology, Jichi Medical University, Shimotsuke, Japan
| | - Xin Zhu
- Biomedical Information Engineering Lab, The University of Aizu, Aizuwakamatsu, Japan
| | - Kazutomo Togashi
- Department of Coloproctology, Aizu Medical Center, Fukushima Medical University, Aizuwakamatsu, Japan
| |
Collapse
|
3
|
Khorsand B, Rajabnia M, Jahanian A, Fathy M, Taghvaei S, Houri H. Enhancing the accuracy and effectiveness of diagnosis of spontaneous bacterial peritonitis in cirrhotic patients: A machine learning approach utilizing clinical and laboratory data. Adv Med Sci 2024; 70:1-7. [PMID: 39419440 DOI: 10.1016/j.advms.2024.10.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2024] [Revised: 08/07/2024] [Accepted: 10/14/2024] [Indexed: 10/19/2024]
Abstract
PURPOSE Spontaneous bacterial peritonitis (SBP) is a bacterial infection of ascitic fluid that develops naturally, without being triggered by any surgical conditions or procedures, and is a common complication of cirrhosis. With a potential mortality rate of 40 %, accurate diagnosis and prompt initiation of appropriate antibiotic therapy are crucial for optimizing patient outcomes and preventing life-threatening complications. This study aimed to expand the use of computational models to improve the diagnostic accuracy of SBP in cirrhotic patients by incorporating a broader range of data, including clinical variables and laboratory values. PATIENTS AND METHODS We employed 5 machine learning classification methods - Decision Tree, Support Vector Machine, Naive Bayes, K-Nearest Neighbor, and Random Forest, utilizing a variety of demographic, clinical, and laboratory features and biomarkers. RESULTS Ascitic fluid markers, including white blood cell (WBC) count, lactate dehydrogenase (LDH), total protein, and polymorphonuclear cells (PMN), significantly differentiated between SBP and non-SBP patients. The Random Forest model demonstrated the highest overall accuracy at 86 %, while the Naive Bayes model achieved the highest sensitivity at 72 %. Utilizing 10 key features instead of the full feature set improved model performance, notably enhancing specificity and accuracy. CONCLUSION Our analysis highlights the potential of machine learning to enhance the accuracy of SBP diagnosis in cirrhotic patients. Integrating these models into clinical workflows could substantially improve patient outcomes. To achieve this, ongoing multidisciplinary research is crucial. Ensuring model interpretability, continuous monitoring, and rigorous validation will be essential for the successful implementation of real-time clinical decision support systems.
Collapse
Affiliation(s)
- Babak Khorsand
- Department of Neurology, University of California, Irvine, CA, USA
| | - Mohsen Rajabnia
- Non-Communicable Diseases Research Center, Alborz University of Medical Sciences, Karaj, Iran.
| | - Ali Jahanian
- Gastroenterology and Liver Diseases Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Mobin Fathy
- Gastroenterology and Liver Diseases Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Somayye Taghvaei
- Department of Medical Biotechnology, National Institute of Genetic Engineering and Biotechnology, Tehran, Iran
| | - Hamidreza Houri
- Foodborne and Waterborne Diseases Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
| |
Collapse
|
4
|
Thijssen A, Schreuder RM, Dehghani N, Schor M, de With PH, van der Sommen F, Boonstra JJ, Moons LM, Schoon EJ. Improving the endoscopic recognition of early colorectal carcinoma using artificial intelligence: current evidence and future directions. Endosc Int Open 2024; 12:E1102-E1117. [PMID: 39398448 PMCID: PMC11466514 DOI: 10.1055/a-2403-3103] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/22/2024] [Accepted: 08/21/2024] [Indexed: 10/15/2024] Open
Abstract
Background and study aims Artificial intelligence (AI) has great potential to improve endoscopic recognition of early stage colorectal carcinoma (CRC). This scoping review aimed to summarize current evidence on this topic, provide an overview of the methodologies currently used, and guide future research. Methods A systematic search was performed following the PRISMA-Scr guideline. PubMed (including Medline), Scopus, Embase, IEEE Xplore, and ACM Digital Library were searched up to January 2024. Studies were eligible for inclusion when using AI for distinguishing CRC from colorectal polyps on endoscopic imaging, using histopathology as gold standard, reporting sensitivity, specificity, or accuracy as outcomes. Results Of 5024 screened articles, 26 were included. Computer-aided diagnosis (CADx) system classification categories ranged from two categories, such as lesions suitable or unsuitable for endoscopic resection, to five categories, such as hyperplastic polyp, sessile serrated lesion, adenoma, cancer, and other. The number of images used in testing databases varied from 69 to 84,585. Diagnostic performances were divergent, with sensitivities varying from 55.0% to 99.2%, specificities from 67.5% to 100% and accuracies from 74.4% to 94.4%. Conclusions This review highlights that using AI to improve endoscopic recognition of early stage CRC is an upcoming research field. We introduced a suggestions list of essential subjects to report in research regarding the development of endoscopy CADx systems, aiming to facilitate more complete reporting and better comparability between studies. There is a knowledge gap regarding real-time CADx system performance during multicenter external validation. Future research should focus on development of CADx systems that can differentiate CRC from premalignant lesions, while providing an indication of invasion depth.
Collapse
Affiliation(s)
- Ayla Thijssen
- GROW Research Institute for Oncology and Reproduction, Maastricht University, Maastricht, Netherlands
- Department of Gastroenterology and Hepatology, Maastricht Universitair Medisch Centrum+, Maastricht, Netherlands
| | - Ramon-Michel Schreuder
- GROW Research Institute for Oncology and Reproduction, Maastricht University, Maastricht, Netherlands
- Department of Gastroenterology and Hepatology, Catharina Hospital, Eindhoven, Netherlands
| | - Nikoo Dehghani
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
| | - Marieke Schor
- University Library, Department of Education and Support, Maastricht University, Maastricht, Netherlands
| | - Peter H.N. de With
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
| | - Fons van der Sommen
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
| | - Jurjen J. Boonstra
- Department of Gastroenterology and Hepatology, Leids Universitair Medisch Centrum, Leiden, Netherlands
| | - Leon M.G. Moons
- Department of Gastroenterology and Hepatology, University Medical Center Utrecht, Utrecht, Netherlands
| | - Erik J. Schoon
- GROW Research Institute for Oncology and Reproduction, Maastricht University, Maastricht, Netherlands
- Department of Gastroenterology and Hepatology, Catharina Hospital, Eindhoven, Netherlands
| |
Collapse
|
5
|
Bou Jaoude J, Al Bacha R, Abboud B. Will artificial intelligence reach any limit in gastroenterology? Artif Intell Gastroenterol 2024; 5:91336. [DOI: 10.35712/aig.v5.i2.91336] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/27/2023] [Revised: 04/25/2024] [Accepted: 06/07/2024] [Indexed: 08/08/2024] Open
Abstract
Endoscopy is the cornerstone in the management of digestive diseases. Over the last few decades, technology has played an important role in the development of this field, helping endoscopists in better detecting and characterizing luminal lesions. However, despite ongoing advancements in endoscopic technology, the incidence of missed pre-neoplastic and neoplastic lesions remains high due to the operator-dependent nature of endoscopy and the challenging learning curve associated with new technologies. Artificial intelligence (AI), an operator-independent field, could be an invaluable solution. AI can serve as a “second observer”, enhancing the performance of endoscopists in detecting and characterizing luminal lesions. By utilizing deep learning (DL), an innovation within machine learning, AI automatically extracts input features from targeted endoscopic images. DL encompasses both computer-aided detection and computer-aided diagnosis, assisting endoscopists in reducing missed detection rates and predicting the histology of luminal digestive lesions. AI applications in clinical gastrointestinal diseases are continuously expanding and evolving the entire digestive tract. In all published studies, real-time AI assists endoscopists in improving the performance of non-expert gastroenterologists, bringing it to a level comparable to that of experts. The development of DL may be affected by selection biases. Studies have utilized different AI-assisted models, which are heterogeneous. In the future, algorithms need validation through large, randomized trials. Theoretically, AI has no limit to assist endoscopists in increasing the accuracy and the quality of endoscopic exams. However, practically, we still have a long way to go before standardizing our AI models to be accepted and applied by all gastroenterologists.
Collapse
Affiliation(s)
- Joseph Bou Jaoude
- Department of Gastroenterology, Levant Hospital, Beirut 166830, Lebanon
| | - Rose Al Bacha
- Department of Gastroenterology, Levant Hospital, Beirut 166830, Lebanon
| | - Bassam Abboud
- Department of General Surgery, Geitaoui Hospital, Faculty of Medicine, Lebanese University, Lebanon, Beirut 166830, Lebanon
| |
Collapse
|
6
|
Ichimasa K, Kudo SE, Misawa M, Takashina Y, Yeoh KG, Miyachi H. Role of the artificial intelligence in the management of T1 colorectal cancer. Dig Liver Dis 2024; 56:1144-1147. [PMID: 38311532 DOI: 10.1016/j.dld.2024.01.202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/29/2023] [Accepted: 01/24/2024] [Indexed: 02/06/2024]
Abstract
Approximately 10% of submucosal invasive (T1) colorectal cancers demonstrate extraintestinal lymph node metastasis, necessitating surgical intervention with lymph node dissection. The ability to identify T1b (submucosal invasion depth ≥ 1000 µm) as a risk factor for lymph node metastasis via pre-treatment endoscopy is crucial in guiding treatment strategies. Accurately distinguishing T1b from T1a (submucosal invasion depth < 1000 µm) or dysplasia remains a significant challenge for artificial intelligence (AI) systems, which require high and consistent diagnostic capabilities. Moreover, as endoscopic therapies like endoscopic full-thickness resection and endoscopic intermuscular dissection evolve, and the focus on reducing unnecessary surgeries intensifies, the initial management of T1 colorectal cancers via endoscopic treatment is anticipated to increase. Consequently, the development of highly accurate and reliable AI systems is essential, not only for pre-treatment depth assessment but also for post-treatment risk stratification of lymph node metastasis. While such AI diagnostic systems are still under development, significant advancements are expected in the near future to improve decision-making in T1 colorectal cancer management.
Collapse
Affiliation(s)
- Katsuro Ichimasa
- Digestive Disease Center, Showa University Northern Yokohama Hospital, 35-1 Chigasaki Chuo, Tsuzuki-ku, Yokohama, Kanagawa 224-8503, Japan; Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore.
| | - Shin-Ei Kudo
- Digestive Disease Center, Showa University Northern Yokohama Hospital, 35-1 Chigasaki Chuo, Tsuzuki-ku, Yokohama, Kanagawa 224-8503, Japan
| | - Masashi Misawa
- Digestive Disease Center, Showa University Northern Yokohama Hospital, 35-1 Chigasaki Chuo, Tsuzuki-ku, Yokohama, Kanagawa 224-8503, Japan
| | - Yuki Takashina
- Digestive Disease Center, Showa University Northern Yokohama Hospital, 35-1 Chigasaki Chuo, Tsuzuki-ku, Yokohama, Kanagawa 224-8503, Japan
| | - Khay Guan Yeoh
- Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Hideyuki Miyachi
- Digestive Disease Center, Showa University Northern Yokohama Hospital, 35-1 Chigasaki Chuo, Tsuzuki-ku, Yokohama, Kanagawa 224-8503, Japan
| |
Collapse
|
7
|
Mandarino FV, Danese S, Uraoka T, Parra-Blanco A, Maeda Y, Saito Y, Kudo SE, Bourke MJ, Iacucci M. Precision endoscopy in colorectal polyps' characterization and planning of endoscopic therapy. Dig Endosc 2024; 36:761-777. [PMID: 37988279 DOI: 10.1111/den.14727] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Accepted: 11/19/2023] [Indexed: 11/23/2023]
Abstract
Precision endoscopy in the management of colorectal polyps and early colorectal cancer has emerged as the standard of care. It includes optical characterization of polyps and estimation of submucosal invasion depth of large nonpedunculated colorectal polyps to select the appropriate endoscopic resection modality. Over time, several imaging modalities have been implemented in endoscopic practice to improve optical performance. Among these, image-enhanced endoscopy systems and magnification endoscopy represent now well-established tools. New advanced technologies, such as endocytoscopy and confocal laser endomicroscopy, have recently shown promising results in predicting the histology of colorectal polyps. In recent years, artificial intelligence has continued to enhance endoscopic performance in the characterization of colorectal polyps, overcoming the limitations of other imaging modes. In this review we retrace the path of precision endoscopy, analyzing the yield of various endoscopic imaging techniques in personalizing management of colorectal polyps and early colorectal cancer.
Collapse
Affiliation(s)
- Francesco Vito Mandarino
- Department of Gastroenterology and Gastrointestinal Endoscopy, San Raffaele Hospital IRCSS, Milan, Italy
- Department of Gastrointestinal Endoscopy, Westmead Hospital, Sydney, NSW, Australia
| | - Silvio Danese
- Department of Gastroenterology and Gastrointestinal Endoscopy, San Raffaele Hospital IRCSS, Milan, Italy
| | - Toshio Uraoka
- Department of Gastroenterology and Hepatology, Gunma University Graduate School of Medicine, Gumma, Japan
| | - Adolfo Parra-Blanco
- NIHR Nottingham Biomedical Research Centre, Nottingham University Hospitals NHS Trust and the University of Nottingham, Nottingham, UK
| | - Yasuharu Maeda
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Kanagawa, Japan
| | - Yutaka Saito
- Endoscopy Division, National Cancer Center Hospital, Tokyo, Japan
| | - Shin-Ei Kudo
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Kanagawa, Japan
| | - Michael J Bourke
- Department of Gastrointestinal Endoscopy, Westmead Hospital, Sydney, NSW, Australia
| | - Marietta Iacucci
- Department of Gastroenterology, University College Cork, Cork, Ireland
| |
Collapse
|
8
|
Bangolo A, Wadhwani N, Nagesh VK, Dey S, Tran HHV, Aguilar IK, Auda A, Sidiqui A, Menon A, Daoud D, Liu J, Pulipaka SP, George B, Furman F, Khan N, Plumptre A, Sekhon I, Lo A, Weissman S. Impact of artificial intelligence in the management of esophageal, gastric and colorectal malignancies. Artif Intell Gastrointest Endosc 2024; 5:90704. [DOI: 10.37126/aige.v5.i2.90704] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/12/2023] [Revised: 01/28/2024] [Accepted: 03/04/2024] [Indexed: 05/11/2024] Open
Abstract
The incidence of gastrointestinal malignancies has increased over the past decade at an alarming rate. Colorectal and gastric cancers are the third and fifth most commonly diagnosed cancers worldwide but are cited as the second and third leading causes of mortality. Early institution of appropriate therapy from timely diagnosis can optimize patient outcomes. Artificial intelligence (AI)-assisted diagnostic, prognostic, and therapeutic tools can assist in expeditious diagnosis, treatment planning/response prediction, and post-surgical prognostication. AI can intercept neoplastic lesions in their primordial stages, accurately flag suspicious and/or inconspicuous lesions with greater accuracy on radiologic, histopathological, and/or endoscopic analyses, and eliminate over-dependence on clinicians. AI-based models have shown to be on par, and sometimes even outperformed experienced gastroenterologists and radiologists. Convolutional neural networks (state-of-the-art deep learning models) are powerful computational models, invaluable to the field of precision oncology. These models not only reliably classify images, but also accurately predict response to chemotherapy, tumor recurrence, metastasis, and survival rates post-treatment. In this systematic review, we analyze the available evidence about the diagnostic, prognostic, and therapeutic utility of artificial intelligence in gastrointestinal oncology.
Collapse
Affiliation(s)
- Ayrton Bangolo
- Department of Internal Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Nikita Wadhwani
- Department of Internal Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Vignesh K Nagesh
- Department of Internal Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Shraboni Dey
- Department of Internal Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Hadrian Hoang-Vu Tran
- Department of Internal Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Izage Kianifar Aguilar
- Department of Internal Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Auda Auda
- Department of Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Aman Sidiqui
- Department of Internal Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Aiswarya Menon
- Department of Internal Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Deborah Daoud
- Department of Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - James Liu
- Department of Internal Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Sai Priyanka Pulipaka
- Department of Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Blessy George
- Department of Internal Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Flor Furman
- Department of Internal Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Nareeman Khan
- Department of Internal Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Adewale Plumptre
- Department of Internal Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Imranjot Sekhon
- Department of Internal Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Abraham Lo
- Department of Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| | - Simcha Weissman
- Department of Internal Medicine, Palisades Medical Center, North Bergen, NJ 07047, United States
| |
Collapse
|
9
|
Li JW, Wang LM, Ichimasa K, Lin KW, Ngu JCY, Ang TL. Use of artificial intelligence in the management of T1 colorectal cancer: a new tool in the arsenal or is deep learning out of its depth? Clin Endosc 2024; 57:24-35. [PMID: 37743068 PMCID: PMC10834280 DOI: 10.5946/ce.2023.036] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Accepted: 05/11/2023] [Indexed: 09/26/2023] Open
Abstract
The field of artificial intelligence is rapidly evolving, and there has been an interest in its use to predict the risk of lymph node metastasis in T1 colorectal cancer. Accurately predicting lymph node invasion may result in fewer patients undergoing unnecessary surgeries; conversely, inadequate assessments will result in suboptimal oncological outcomes. This narrative review aims to summarize the current literature on deep learning for predicting the probability of lymph node metastasis in T1 colorectal cancer, highlighting areas of potential application and barriers that may limit its generalizability and clinical utility.
Collapse
Affiliation(s)
- James Weiquan Li
- Department of Gastroenterology and Hepatology, Changi General Hospital, Singapore Health Services, Singapore
- Academic Medicine Center, Duke-NUS Medical School, Singapore
| | - Lai Mun Wang
- Department of Laboratory Medicine, Changi General Hospital, Singapore Health Services, Singapore
| | - Katsuro Ichimasa
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan
- Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Kenneth Weicong Lin
- Department of Gastroenterology and Hepatology, Changi General Hospital, Singapore Health Services, Singapore
- Academic Medicine Center, Duke-NUS Medical School, Singapore
| | - James Chi-Yong Ngu
- Department of General Surgery, Changi General Hospital, Singapore Health Services, Singapore
| | - Tiing Leong Ang
- Department of Gastroenterology and Hepatology, Changi General Hospital, Singapore Health Services, Singapore
- Academic Medicine Center, Duke-NUS Medical School, Singapore
| |
Collapse
|
10
|
Bai J, Liu K, Gao L, Zhao X, Zhu S, Han Y, Liu Z. Computer-aided diagnosis in predicting the invasion depth of early colorectal cancer: a systematic review and meta-analysis of diagnostic test accuracy. Surg Endosc 2023; 37:6627-6639. [PMID: 37430125 DOI: 10.1007/s00464-023-10223-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2023] [Accepted: 06/16/2023] [Indexed: 07/12/2023]
Abstract
BACKGROUND Endoscopic resection (ER) is widely applied to treat early colorectal cancer (CRC). Predicting the invasion depth of early CRC is critical in determining treatment strategies. The use of computer-aided diagnosis (CAD) algorithms could theoretically make accurate and objective predictions regarding the suitability of lesions for ER indication based on invasion depth. This study aimed to assess diagnostic test accuracy of CAD algorithms in predicting the invasion depth of early CRC and to compare the performance between the CAD algorithms and endoscopists. METHODS Multiple databases were searched until June 30, 2022 for studies that evaluated the diagnostic performance of CAD algorithms for invasion depth of CRC. Meta-analysis of diagnostic test accuracy using a bivariate mixed-effects model was performed. RESULTS Ten studies consisting of 13 arms (13,918 images from 1472 lesions) were included. Due to significant heterogeneity, studies were stratified into Japan/Korea-based or China-based studies. For the former, the area under the curve (AUC), sensitivity, and specificity of the CAD algorithms were 0.89 (95% CI 0.86-0.91), 62% (95% CI 50-72%), and 96% (95% CI 93-98%), respectively. For the latter, AUC, sensitivity, and specificity were 0.94 (95% CI 0.92-0.96), 88% (95% CI 78-94%), and 88% (95% CI 80-93%), respectively. The performance of the CAD algorithms in Japan/Korea-based studies was not significantly different from that of all endoscopists (0.88 vs. 0.91, P = 0.10) but was inferior to that of expert endoscopists (0.88 vs. 0.92, P = 0.03). The performance of the CAD algorithms in China-based studies was better than that of all endoscopists (0.94 vs. 0.90, P = 0.01). CONCLUSION The CAD algorithms showed comparable accuracy for prediction of invasion depth of early CRC compared to all endoscopists, which was still lower than expert endoscopists in diagnostic accuracy; more improvements should be achieved before it can be extensively applied to clinical practice.
Collapse
Affiliation(s)
- Jiawei Bai
- Xijing Hospital of Digestive Diseases, Air Force Medical University (Fourth Military Medical University), 127 Changle West Road, Xi'an, 710032, Shaanxi, China
- School of Medicine, Yan'an University, Yan'an, China
| | - Kai Liu
- Xijing Hospital of Digestive Diseases, Air Force Medical University (Fourth Military Medical University), 127 Changle West Road, Xi'an, 710032, Shaanxi, China
| | - Li Gao
- Xijing Hospital of Digestive Diseases, Air Force Medical University (Fourth Military Medical University), 127 Changle West Road, Xi'an, 710032, Shaanxi, China
| | - Xin Zhao
- Xijing Hospital of Digestive Diseases, Air Force Medical University (Fourth Military Medical University), 127 Changle West Road, Xi'an, 710032, Shaanxi, China
| | - Shaohua Zhu
- Xijing Hospital of Digestive Diseases, Air Force Medical University (Fourth Military Medical University), 127 Changle West Road, Xi'an, 710032, Shaanxi, China
| | - Ying Han
- Xijing Hospital of Digestive Diseases, Air Force Medical University (Fourth Military Medical University), 127 Changle West Road, Xi'an, 710032, Shaanxi, China.
| | - Zhiguo Liu
- Xijing Hospital of Digestive Diseases, Air Force Medical University (Fourth Military Medical University), 127 Changle West Road, Xi'an, 710032, Shaanxi, China.
| |
Collapse
|
11
|
van Bokhorst QNE, Houwen BBSL, Hazewinkel Y, Fockens P, Dekker E. Advances in artificial intelligence and computer science for computer-aided diagnosis of colorectal polyps: current status. Endosc Int Open 2023; 11:E752-E767. [PMID: 37593158 PMCID: PMC10431975 DOI: 10.1055/a-2098-1999] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/04/2023] [Accepted: 05/08/2023] [Indexed: 08/19/2023] Open
Affiliation(s)
- Querijn N E van Bokhorst
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, location Academic Medical Center, Amsterdam, the Netherlands
- Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam, the Netherlands
| | - Britt B S L Houwen
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, location Academic Medical Center, Amsterdam, the Netherlands
- Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam, the Netherlands
| | - Yark Hazewinkel
- Department of Gastroenterology and Hepatology, Tergooi Medical Center, Hilversum, the Netherlands
| | - Paul Fockens
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, location Academic Medical Center, Amsterdam, the Netherlands
- Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam, the Netherlands
| | - Evelien Dekker
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, location Academic Medical Center, Amsterdam, the Netherlands
- Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam, the Netherlands
| |
Collapse
|
12
|
Nemoto D, Guo Z, Katsuki S, Takezawa T, Maemoto R, Kawasaki K, Inoue K, Akutagawa T, Tanaka H, Sato K, Omori T, Takanashi K, Hayashi Y, Nakajima Y, Miyakura Y, Matsumoto T, Yoshida N, Esaki M, Uraoka T, Kato H, Inoue Y, Peng B, Zhang R, Hisabe T, Matsuda T, Yamamoto H, Tanaka N, Lefor AK, Zhu X, Togashi K. Computer-aided diagnosis of early-stage colorectal cancer using nonmagnified endoscopic white-light images (with videos). Gastrointest Endosc 2023; 98:90-99.e4. [PMID: 36738793 DOI: 10.1016/j.gie.2023.01.050] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Revised: 01/05/2023] [Accepted: 01/25/2023] [Indexed: 02/06/2023]
Abstract
BACKGROUND AND AIMS Differentiation of colorectal cancers (CRCs) with deep submucosal invasion (T1b) from CRCs with superficial invasion (T1a) or no invasion (Tis) is not straightforward. This study aimed to develop a computer-aided diagnosis (CADx) system to establish the diagnosis of early-stage cancers using nonmagnified endoscopic white-light images alone. METHODS From 5108 images, 1513 lesions (Tis, 1074; T1a, 145; T1b, 294) were collected from 1470 patients at 10 academic hospitals and assigned to training and testing datasets (3:1). The ResNet-50 network was used as the backbone to extract features from images. Oversampling and focal loss were used to compensate class imbalance of the invasive stage. Diagnostic performance was assessed using the testing dataset including 403 CRCs with 1392 images. Two experts and 2 trainees read the identical testing dataset. RESULTS At a 90% cutoff for the per-lesion score, CADx showed the highest specificity of 94.4% (95% confidence interval [CI], 91.3-96.6), with 59.8% (95% CI, 48.3-70.4) sensitivity and 87.3% (95% CI, 83.7-90.4) accuracy. The area under the characteristic curve was 85.1% (95% CI, 79.9-90.4) for CADx, 88.2% (95% CI, 83.7-92.8) for expert 1, 85.9% (95% CI, 80.9-90.9) for expert 2, 77.0% (95% CI, 71.5-82.4) for trainee 1 (vs CADx; P = .0076), and 66.2% (95% CI, 60.6-71.9) for trainee 2 (P < .0001). The function was also confirmed on 9 short videos. CONCLUSIONS A CADx system developed with endoscopic white-light images showed excellent per-lesion specificity and accuracy for T1b lesion diagnosis, equivalent to experts and superior to trainees. (Clinical trial registration number: UMIN000037053.).
Collapse
Affiliation(s)
- Daiki Nemoto
- Department of Coloproctology, Aizu Medical Center Fukushima Medical University, Aizuwakamatsu, Japan
| | - Zhe Guo
- Biomedical Information Engineering Lab, The University of Aizu, Aizuwakamatsu, Japan; Department of Laboratory Medicine, The Second Xiangya Hospital, Central South University, Changsha, China
| | - Shinichi Katsuki
- Department of Gastroenterology, Otaru Ekisaikai Hospital, Otaru, Japan
| | - Takahito Takezawa
- Department of Medicine, Division of Gastroenterology, Jichi Medical University, Shimotsuke, Japan
| | - Ryo Maemoto
- Department of Surgery, Saitama Medical Center, Jichi Medical University, Saitama, Japan
| | - Keisuke Kawasaki
- Department of Gastroenterology, Iwate Medical University, Morioka, Japan
| | - Ken Inoue
- Department of Molecular Gastroenterology and Hepatology, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Takashi Akutagawa
- Division of Gastroenterology, Department of Internal Medicine, Faculty of Medicine, Saga University, Saga, Japan
| | - Hirohito Tanaka
- Department of Gastroenterology and Hepatology, Gunma University Graduate School of Medicine, Maebashi, Japan
| | - Koichiro Sato
- Department of Clinical Laboratory and Endoscopy, Tokyo Women's Medical University Medical Center East, Tokyo, Japan
| | - Teppei Omori
- Institute of Gastroenterology, Tokyo Women's Medical University, Tokyo, Japan
| | | | - Yoshikazu Hayashi
- Department of Medicine, Division of Gastroenterology, Jichi Medical University, Shimotsuke, Japan
| | - Yuki Nakajima
- Department of Coloproctology, Aizu Medical Center Fukushima Medical University, Aizuwakamatsu, Japan
| | - Yasuyuki Miyakura
- Department of Surgery, Saitama Medical Center, Jichi Medical University, Saitama, Japan
| | - Takayuki Matsumoto
- Department of Gastroenterology, Iwate Medical University, Morioka, Japan
| | - Naohisa Yoshida
- Department of Molecular Gastroenterology and Hepatology, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Motohiro Esaki
- Division of Gastroenterology, Department of Internal Medicine, Faculty of Medicine, Saga University, Saga, Japan
| | - Toshio Uraoka
- Department of Gastroenterology and Hepatology, Gunma University Graduate School of Medicine, Maebashi, Japan
| | - Hiroyuki Kato
- Department of Clinical Laboratory and Endoscopy, Tokyo Women's Medical University Medical Center East, Tokyo, Japan
| | - Yuji Inoue
- Institute of Gastroenterology, Tokyo Women's Medical University, Tokyo, Japan
| | - Boyuan Peng
- Biomedical Information Engineering Lab, The University of Aizu, Aizuwakamatsu, Japan
| | - Ruiyao Zhang
- Biomedical Information Engineering Lab, The University of Aizu, Aizuwakamatsu, Japan
| | - Takashi Hisabe
- Department of Gastroenterology, Fukuoka University Chikushi Hospital, Fukuoka, Japan
| | - Tomoki Matsuda
- Department of Gastroenterology, Sendai Kosei Hospital, Sendai, Japan
| | - Hironori Yamamoto
- Department of Medicine, Division of Gastroenterology, Jichi Medical University, Shimotsuke, Japan
| | - Noriko Tanaka
- Health Data Science Research Section, Tokyo Metropolitan Institute of Gerontology, Tokyo, Japan
| | | | - Xin Zhu
- Biomedical Information Engineering Lab, The University of Aizu, Aizuwakamatsu, Japan
| | - Kazutomo Togashi
- Department of Coloproctology, Aizu Medical Center Fukushima Medical University, Aizuwakamatsu, Japan
| |
Collapse
|
13
|
Sharma A, Kumar R, Yadav G, Garg P. Artificial intelligence in intestinal polyp and colorectal cancer prediction. Cancer Lett 2023; 565:216238. [PMID: 37211068 DOI: 10.1016/j.canlet.2023.216238] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Revised: 05/17/2023] [Accepted: 05/17/2023] [Indexed: 05/23/2023]
Abstract
Artificial intelligence (AI) algorithms and their application to disease detection and decision support for healthcare professions have greatly evolved in the recent decade. AI has been widely applied and explored in gastroenterology for endoscopic analysis to diagnose intestinal cancers, premalignant polyps, gastrointestinal inflammatory lesions, and bleeding. Patients' responses to treatments and prognoses have both been predicted using AI by combining multiple algorithms. In this review, we explored the recent applications of AI algorithms in the identification and characterization of intestinal polyps and colorectal cancer predictions. AI-based prediction models have the potential to help medical practitioners diagnose, establish prognoses, and find accurate conclusions for the treatment of patients. With the understanding that rigorous validation of AI approaches using randomized controlled studies is solicited before widespread clinical use by health authorities, the article also discusses the limitations and challenges associated with deploying AI systems to diagnose intestinal malignancies and premalignant lesions.
Collapse
Affiliation(s)
- Anju Sharma
- Department of Pharmacoinformatics, National Institute of Pharmaceutical Education and Research, S.A.S Nagar, 160062, Punjab, India
| | - Rajnish Kumar
- Amity Institute of Biotechnology, Amity University Uttar Pradesh, Lucknow Campus, Uttar Pradesh, 226010, India; Department of Veterinary Medicine and Surgery, College of Veterinary Medicine, University of Missouri, Columbia, MO, USA
| | - Garima Yadav
- Amity Institute of Biotechnology, Amity University Uttar Pradesh, Lucknow Campus, Uttar Pradesh, 226010, India
| | - Prabha Garg
- Department of Pharmacoinformatics, National Institute of Pharmaceutical Education and Research, S.A.S Nagar, 160062, Punjab, India.
| |
Collapse
|
14
|
Wang KN, Zhuang S, Ran QY, Zhou P, Hua J, Zhou GQ, He X. DLGNet: A dual-branch lesion-aware network with the supervised Gaussian Mixture model for colon lesions classification in colonoscopy images. Med Image Anal 2023; 87:102832. [PMID: 37148864 DOI: 10.1016/j.media.2023.102832] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2022] [Revised: 01/20/2023] [Accepted: 04/20/2023] [Indexed: 05/08/2023]
Abstract
Colorectal cancer is one of the malignant tumors with the highest mortality due to the lack of obvious early symptoms. It is usually in the advanced stage when it is discovered. Thus the automatic and accurate classification of early colon lesions is of great significance for clinically estimating the status of colon lesions and formulating appropriate diagnostic programs. However, it is challenging to classify full-stage colon lesions due to the large inter-class similarities and intra-class differences of the images. In this work, we propose a novel dual-branch lesion-aware neural network (DLGNet) to classify intestinal lesions by exploring the intrinsic relationship between diseases, composed of four modules: lesion location module, dual-branch classification module, attention guidance module, and inter-class Gaussian loss function. Specifically, the elaborate dual-branch module integrates the original image and the lesion patch obtained by the lesion localization module to explore and interact with lesion-specific features from a global and local perspective. Also, the feature-guided module guides the model to pay attention to the disease-specific features by learning remote dependencies through spatial and channel attention after network feature learning. Finally, the inter-class Gaussian loss function is proposed, which assumes that each feature extracted by the network is an independent Gaussian distribution, and the inter-class clustering is more compact, thereby improving the discriminative ability of the network. The extensive experiments on the collected 2568 colonoscopy images have an average accuracy of 91.50%, and the proposed method surpasses the state-of-the-art methods. This study is the first time that colon lesions are classified at each stage and achieves promising colon disease classification performance. To motivate the community, we have made our code publicly available via https://github.com/soleilssss/DLGNet.
Collapse
Affiliation(s)
- Kai-Ni Wang
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, China; State Key Laboratory of Digital Medical Engineering, Southeast University, Nanjing, China; Jiangsu Key Laboratory of Biomaterials and Devices, Southeast University, Nanjing, China
| | - Shuaishuai Zhuang
- The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Qi-Yong Ran
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, China; State Key Laboratory of Digital Medical Engineering, Southeast University, Nanjing, China; Jiangsu Key Laboratory of Biomaterials and Devices, Southeast University, Nanjing, China
| | - Ping Zhou
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, China; State Key Laboratory of Digital Medical Engineering, Southeast University, Nanjing, China; Jiangsu Key Laboratory of Biomaterials and Devices, Southeast University, Nanjing, China
| | - Jie Hua
- The First Affiliated Hospital of Nanjing Medical University, Nanjing, China; Liyang People's Hospital, Liyang Branch Hospital of Jiangsu Province Hospital, Liyang, China
| | - Guang-Quan Zhou
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, China; State Key Laboratory of Digital Medical Engineering, Southeast University, Nanjing, China; Jiangsu Key Laboratory of Biomaterials and Devices, Southeast University, Nanjing, China.
| | - Xiaopu He
- The First Affiliated Hospital of Nanjing Medical University, Nanjing, China.
| |
Collapse
|
15
|
Shahid B, Abbas M, Ur Rehman A, Ul Abideen Z. IAPC2: Improved and Automatic Classification of Polyp for Colorectal Cancer. 2023 INTERNATIONAL CONFERENCE ON BUSINESS ANALYTICS FOR TECHNOLOGY AND SECURITY (ICBATS) 2023. [DOI: 10.1109/icbats57792.2023.10111431] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/01/2023]
Affiliation(s)
- Bisma Shahid
- Riphah International University,Department of Computer Science,Lahore,Pakistan
| | - Maria Abbas
- Riphah International University,Department of Computer Science,Lahore,Pakistan
| | - Abd Ur Rehman
- Riphah International University,Department of Computer Science,Lahore,Pakistan
| | | |
Collapse
|
16
|
Chadebecq F, Lovat LB, Stoyanov D. Artificial intelligence and automation in endoscopy and surgery. Nat Rev Gastroenterol Hepatol 2023; 20:171-182. [PMID: 36352158 DOI: 10.1038/s41575-022-00701-y] [Citation(s) in RCA: 29] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 10/03/2022] [Indexed: 11/10/2022]
Abstract
Modern endoscopy relies on digital technology, from high-resolution imaging sensors and displays to electronics connecting configurable illumination and actuation systems for robotic articulation. In addition to enabling more effective diagnostic and therapeutic interventions, the digitization of the procedural toolset enables video data capture of the internal human anatomy at unprecedented levels. Interventional video data encapsulate functional and structural information about a patient's anatomy as well as events, activity and action logs about the surgical process. This detailed but difficult-to-interpret record from endoscopic procedures can be linked to preoperative and postoperative records or patient imaging information. Rapid advances in artificial intelligence, especially in supervised deep learning, can utilize data from endoscopic procedures to develop systems for assisting procedures leading to computer-assisted interventions that can enable better navigation during procedures, automation of image interpretation and robotically assisted tool manipulation. In this Perspective, we summarize state-of-the-art artificial intelligence for computer-assisted interventions in gastroenterology and surgery.
Collapse
Affiliation(s)
- François Chadebecq
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
| | - Laurence B Lovat
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
| | - Danail Stoyanov
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK.
| |
Collapse
|
17
|
Cai H, Feng X, Yin R, Zhao Y, Guo L, Fan X, Liao J. MIST: multiple instance learning network based on Swin Transformer for whole slide image classification of colorectal adenomas. J Pathol 2023; 259:125-135. [PMID: 36318158 DOI: 10.1002/path.6027] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2022] [Revised: 09/30/2022] [Accepted: 10/28/2022] [Indexed: 12/12/2022]
Abstract
Colorectal adenoma is a recognized precancerous lesion of colorectal cancer (CRC), and at least 80% of colorectal cancers are malignantly transformed from it. Therefore, it is essential to distinguish benign from malignant adenomas in the early screening of colorectal cancer. Many deep learning computational pathology studies based on whole slide images (WSIs) have been proposed. Most approaches require manual annotation of lesion regions on WSIs, which is time-consuming and labor-intensive. This study proposes a new approach, MIST - Multiple Instance learning network based on the Swin Transformer, which can accurately classify colorectal adenoma WSIs only with slide-level labels. MIST uses the Swin Transformer as the backbone to extract features of images through self-supervised contrastive learning and uses a dual-stream multiple instance learning network to predict the class of slides. We trained and validated MIST on 666 WSIs collected from 480 colorectal adenoma patients in the Department of Pathology, The Affiliated Drum Tower Hospital of Nanjing University Medical School. These slides contained six common types of colorectal adenomas. The accuracy of external validation on 273 newly collected WSIs from Nanjing First Hospital was 0.784, which was superior to the existing methods and reached a level comparable to that of the local pathologist's accuracy of 0.806. Finally, we analyzed the interpretability of MIST and observed that the lesion areas of interest in MIST were generally consistent with those of interest to local pathologists. In conclusion, MIST is a low-burden, interpretable, and effective approach that can be used in colorectal cancer screening and may lead to a potential reduction in the mortality of CRC patients by assisting clinicians in the decision-making process. © 2022 The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- Hongbin Cai
- School of Science, China Pharmaceutical University, Nanjing, PR China
| | - Xiaobing Feng
- College of Electrical and Information Engineering, Hunan University, Changsha, PR China
| | - Ruomeng Yin
- School of Science, China Pharmaceutical University, Nanjing, PR China
| | - Youcai Zhao
- Department of Pathology, Nanjing First Hospital, Nanjing, PR China
| | - Lingchuan Guo
- Department of Pathology, The First Affiliated Hospital of Soochow University, Soochow, PR China
| | - Xiangshan Fan
- Department of Pathology, The Affiliated Drum Tower Hospital of Nanjing University Medical School, Nanjing, PR China
| | - Jun Liao
- School of Science, China Pharmaceutical University, Nanjing, PR China
| |
Collapse
|
18
|
Dilmaghani S, Coelho-Prabhu N. Role of Artificial Intelligence in Colonoscopy: A Literature Review of the Past, Present, and Future Directions. TECHNIQUES AND INNOVATIONS IN GASTROINTESTINAL ENDOSCOPY 2023; 25:399-412. [DOI: 10.1016/j.tige.2023.03.002] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/04/2025]
|
19
|
Ali S. Where do we stand in AI for endoscopic image analysis? Deciphering gaps and future directions. NPJ Digit Med 2022; 5:184. [PMID: 36539473 PMCID: PMC9767933 DOI: 10.1038/s41746-022-00733-3] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Accepted: 11/29/2022] [Indexed: 12/24/2022] Open
Abstract
Recent developments in deep learning have enabled data-driven algorithms that can reach human-level performance and beyond. The development and deployment of medical image analysis methods have several challenges, including data heterogeneity due to population diversity and different device manufacturers. In addition, more input from experts is required for a reliable method development process. While the exponential growth in clinical imaging data has enabled deep learning to flourish, data heterogeneity, multi-modality, and rare or inconspicuous disease cases still need to be explored. Endoscopy being highly operator-dependent with grim clinical outcomes in some disease cases, reliable and accurate automated system guidance can improve patient care. Most designed methods must be more generalisable to the unseen target data, patient population variability, and variable disease appearances. The paper reviews recent works on endoscopic image analysis with artificial intelligence (AI) and emphasises the current unmatched needs in this field. Finally, it outlines the future directions for clinically relevant complex AI solutions to improve patient outcomes.
Collapse
Affiliation(s)
- Sharib Ali
- School of Computing, University of Leeds, LS2 9JT, Leeds, UK.
| |
Collapse
|
20
|
Yao L, Lu Z, Yang G, Zhou W, Xu Y, Guo M, Huang X, He C, Zhou R, Deng Y, Wu H, Chen B, Gong R, Zhang L, Zhang M, Gong W, Yu H. Development and validation of an artificial intelligence-based system for predicting colorectal cancer invasion depth using multi-modal data. Dig Endosc 2022. [PMID: 36478234 DOI: 10.1111/den.14493] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Accepted: 12/05/2022] [Indexed: 01/20/2023]
Abstract
OBJECTIVES Accurate endoscopic optical prediction of the depth of cancer invasion is critical for guiding an optimal treatment approach of large sessile colorectal polyps but was hindered by insufficient endoscopists expertise and inter-observer variability. We aimed to construct a clinically applicable artificial intelligence (AI) system for the identification of presence of cancer invasion in large sessile colorectal polyps. METHODS A deep learning-based colorectal cancer invasion calculation (CCIC) system was constructed. Multi-modal data including clinical information, white light (WL) and image-enhanced endoscopy (IEE) were included for training. The system was trained using 339 lesions and tested on 198 lesions across three hospitals. Man-machine contest, reader study and video validation were further conducted to evaluate the performance of CCIC. RESULTS The overall accuracy of CCIC system using image and video validation was 90.4% and 89.7%, respectively. In comparison with 14 endoscopists, the accuracy of CCIC was comparable with expert endoscopists but superior to all the participating senior and junior endoscopists in both image and video validation set. With CCIC augmentation, the average accuracy of junior endoscopists improved significantly from 75.4% to 85.3% (P = 0.002). CONCLUSIONS This deep learning-based CCIC system may play an important role in predicting the depth of cancer invasion in colorectal polyps, thus determining treatment strategies for these large sessile colorectal polyps.
Collapse
Affiliation(s)
- Liwen Yao
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Zihua Lu
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Genhua Yang
- Department of Gastroenterology, Shenzhen Hospital of Southern Medical University, Shenzhen, China
| | - Wei Zhou
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Youming Xu
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Mingwen Guo
- Department of Gastroenterology, The First Hospital of Yichang, Yichang, China
| | - Xu Huang
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Chunping He
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Rui Zhou
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Yunchao Deng
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Huiling Wu
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Boru Chen
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Rongrong Gong
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Lihui Zhang
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Mengjiao Zhang
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Wei Gong
- Department of Gastroenterology, Shenzhen Hospital of Southern Medical University, Shenzhen, China
| | - Honggang Yu
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| |
Collapse
|
21
|
Howlader K, Liu L. Transfer Learning Pre-training Dataset and Fine-tuning Effect Analysis on Cancer Histopathology Images. 2022 IEEE INTERNATIONAL CONFERENCE ON BIOINFORMATICS AND BIOMEDICINE (BIBM) 2022:3015-3022. [DOI: 10.1109/bibm55620.2022.9995076] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/07/2025]
Affiliation(s)
| | - Lu Liu
- North Dakota State University,ND,USA
| |
Collapse
|
22
|
Komanduri S, Dominitz JA, Rabeneck L, Kahi C, Ladabaum U, Imperiale TF, Byrne MF, Lee JK, Lieberman D, Wang AY, Sultan S, Shaukat A, Pohl H, Muthusamy VR. AGA White Paper: Challenges and Gaps in Innovation for the Performance of Colonoscopy for Screening and Surveillance of Colorectal Cancer. Clin Gastroenterol Hepatol 2022; 20:2198-2209.e3. [PMID: 35688352 DOI: 10.1016/j.cgh.2022.03.051] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/04/2021] [Revised: 02/23/2022] [Accepted: 03/17/2022] [Indexed: 02/07/2023]
Abstract
In 2018, the American Gastroenterological Association's Center for GI Innovation and Technology convened a consensus conference, entitled "Colorectal Cancer Screening and Surveillance: Role of Emerging Technology and Innovation to Improve Outcomes." The conference participants, which included more than 60 experts in colorectal cancer, considered recent improvements in colorectal cancer screening rates and polyp detection, persistent barriers to colonoscopy uptake, and opportunities for performance improvement and innovation. This white paper originates from that conference. It aims to summarize current patient- and physician-centered gaps and challenges in colonoscopy, diagnostic and therapeutic challenges affecting colonoscopy uptake, and the potential use of emerging technologies and quality metrics to improve patient outcomes.
Collapse
Affiliation(s)
- Srinadh Komanduri
- Department of Department of Gastroenterology and Hepatology, Northwestern University, Chicago, Illinois
| | - Jason A Dominitz
- Veterans Affairs Puget Sound Health Care System and the Division of Gastroenterology, Department of Medicine, University of Washington School of Medicine, Seattle, Washington
| | - Linda Rabeneck
- Department of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Charles Kahi
- Indiana University School of Medicine, Richard L. Roudebush Veterans Affairs Medical Center, Indianapolis, Indiana
| | - Uri Ladabaum
- Division of Gastroenterology and Hepatology, Stanford University School of Medicine, Stanford, California
| | - Thomas F Imperiale
- Department of Medicine, Indiana University School of Medicine, the Regenstrief Institute, the Simon Cancer Center, and the Center for Innovation at Roudebush Veterans Affairs Medical Center, Indianapolis, Indiana
| | - Michael F Byrne
- Division of Gastroenterology, Vancouver General Hospital/University of British Columbia, Vancouver, British Columbia, Canada
| | - Jeffrey K Lee
- Collaborative Health Outcomes Research in Digestive Diseases (CHORD) Group, Kaiser Permanente Division of Research, Kaiser Permanente San Francisco, San Francisco, California
| | - David Lieberman
- Division of Gastroenterology and Hepatology, Oregon Health and Science University, Portland, Oregon
| | - Andrew Y Wang
- Division of Gastroenterology and Hepatology, University of Virginia, Charlottesville, Virginia
| | - Shahnaz Sultan
- Division of Gastroenterology, Hepatology and Nutrition, School of Medicine, University of Minnesota, Minneapolis, Minnesota
| | - Aasma Shaukat
- Division of Gastroenterology, Minneapolis Veterans Affairs Health Care System and Department of Medicine, School of Medicine, University of Minnesota, Minneapolis, Minnesota
| | - Heiko Pohl
- Veterans Affairs Medical Center White River Junction, Vermont; Dartmouth Geisel School of Medicine, Hanover, New Hampshire
| | - V Raman Muthusamy
- Vatche and Tamar Manoukian Division of Digestive Diseases, Department of Medicine, University of California Los Angeles, Los Angeles, California.
| |
Collapse
|
23
|
Deep Neural Network Models for Colon Cancer Screening. Cancers (Basel) 2022; 14:cancers14153707. [PMID: 35954370 PMCID: PMC9367621 DOI: 10.3390/cancers14153707] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Revised: 07/26/2022] [Accepted: 07/27/2022] [Indexed: 12/24/2022] Open
Abstract
Simple Summary Deep learning models have been shown to achieve high performance in diagnosing colon cancer compared to conventional image processing and hand-crafted machine learning methods. Hence, several studies have focused on developing hybrid learning, end-to-end, and transfer learning techniques to reduce manual interaction and for labelling the regions of interest. However, these weak learning techniques do not always provide a clear diagnosis. Therefore, it is necessary to develop a clear explainable learning method that can highlight factors and form the basis of clinical decisions. However, there has been little research carried out employing such transparent approaches. This study discussed the aforementioned models for colon cancer diagnosis. Abstract Early detection of colorectal cancer can significantly facilitate clinicians’ decision-making and reduce their workload. This can be achieved using automatic systems with endoscopic and histological images. Recently, the success of deep learning has motivated the development of image- and video-based polyp identification and segmentation. Currently, most diagnostic colonoscopy rooms utilize artificial intelligence methods that are considered to perform well in predicting invasive cancer. Convolutional neural network-based architectures, together with image patches and preprocesses are often widely used. Furthermore, learning transfer and end-to-end learning techniques have been adopted for detection and localization tasks, which improve accuracy and reduce user dependence with limited datasets. However, explainable deep networks that provide transparency, interpretability, reliability, and fairness in clinical diagnostics are preferred. In this review, we summarize the latest advances in such models, with or without transparency, for the prediction of colorectal cancer and also address the knowledge gap in the upcoming technology.
Collapse
|
24
|
Abstract
Artificial intelligence (AI) is rapidly developing in various medical fields, and there is an increase in research performed in the field of gastrointestinal (GI) endoscopy. In particular, the advent of convolutional neural network, which is a class of deep learning method, has the potential to revolutionize the field of GI endoscopy, including esophagogastroduodenoscopy (EGD), capsule endoscopy (CE), and colonoscopy. A total of 149 original articles pertaining to AI (27 articles in esophagus, 30 articles in stomach, 29 articles in CE, and 63 articles in colon) were identified in this review. The main focuses of AI in EGD are cancer detection, identifying the depth of cancer invasion, prediction of pathological diagnosis, and prediction of Helicobacter pylori infection. In the field of CE, automated detection of bleeding sites, ulcers, tumors, and various small bowel diseases is being investigated. AI in colonoscopy has advanced with several patient-based prospective studies being conducted on the automated detection and classification of colon polyps. Furthermore, research on inflammatory bowel disease has also been recently reported. Most studies of AI in the field of GI endoscopy are still in the preclinical stages because of the retrospective design using still images. Video-based prospective studies are needed to advance the field. However, AI will continue to develop and be used in daily clinical practice in the near future. In this review, we have highlighted the published literature along with providing current status and insights into the future of AI in GI endoscopy.
Collapse
Affiliation(s)
- Yutaka Okagawa
- Endoscopy Division, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan.,Department of Gastroenterology, Tonan Hospital, Sapporo, Japan
| | - Seiichiro Abe
- Endoscopy Division, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan.
| | - Masayoshi Yamada
- Endoscopy Division, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
| | - Ichiro Oda
- Endoscopy Division, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
| | - Yutaka Saito
- Endoscopy Division, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
| |
Collapse
|
25
|
Mizuno S, Okabayashi K, Ikebata A, Matsui S, Seishima R, Shigeta K, Kitagawa Y. Prediction of pouchitis after ileal pouch-anal anastomosis in patients with ulcerative colitis using artificial intelligence and deep learning. Tech Coloproctol 2022; 26:471-478. [PMID: 35233723 DOI: 10.1007/s10151-022-02602-3] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/22/2021] [Accepted: 02/16/2022] [Indexed: 02/07/2023]
Abstract
BACKGROUND Pouchitis is one of the major postoperative complications of ulcerative colitis (UC), and it is still difficult to predict the development of pouchitis after ileal pouch-anal anastomosis (IPAA) in UC patients. In this study, we examined whether a deep learning (DL) model could predict the development of pouchitis. METHODS UC patients who underwent two-stage restorative proctocolectomy with IPAA at Keio University Hospital were included in this retrospective analysis. The modified pouchitis disease activity index (mPDAI) was evaluated by the clinical and endoscopic findings. Pouchitis was defined as an mPDAI ≥ 5.860; endoscopic pouch images before ileostomy closure were collected. A convolutional neural network was used as the DL model, and the prediction rates of pouchitis after ileostomy closure were evaluated by fivefold cross-validation. RESULTS A total of 43 patients were included (24 males and 19 females, mean age 39.2 ± 13.2 years). Pouchitis occurred in 14 (33%) patients after ileostomy closure. In less than half of the patients, mPDAI scores matched before and after ileostomy closure. Most of patients whose mPDAI scores did not match before and after ileostomy closure had worse mPDAI scores after than before. The prediction rate of pouchitis calculated by the area under the curve using the DL model was 84%. Conversely, the prediction rate of pouchitis using mPDAI before ileostomy closure was 62%. CONCLUSION The prediction rate of pouchitis using the DL model was more than 20% higher than that using mPDAI, suggesting the utility of the DL model as a prediction model for the development of pouchitis. It could also be used to determine early interventions for pouchitis.
Collapse
Affiliation(s)
- S Mizuno
- Department of Surgery, Keio University School of Medicine, 35 Shinano-machi, Shinjuku-ku, Tokyo, 160-8582, Japan
| | - K Okabayashi
- Department of Surgery, Keio University School of Medicine, 35 Shinano-machi, Shinjuku-ku, Tokyo, 160-8582, Japan.
| | - A Ikebata
- Department of Surgery, Saitama Medical Center, Saitama, Japan
| | - S Matsui
- Department of Surgery, Keio University School of Medicine, 35 Shinano-machi, Shinjuku-ku, Tokyo, 160-8582, Japan
| | - R Seishima
- Department of Surgery, Keio University School of Medicine, 35 Shinano-machi, Shinjuku-ku, Tokyo, 160-8582, Japan
| | - K Shigeta
- Department of Surgery, Keio University School of Medicine, 35 Shinano-machi, Shinjuku-ku, Tokyo, 160-8582, Japan
| | - Y Kitagawa
- Department of Surgery, Keio University School of Medicine, 35 Shinano-machi, Shinjuku-ku, Tokyo, 160-8582, Japan
| |
Collapse
|
26
|
Classification of the Confocal Microscopy Images of Colorectal Tumor and Inflammatory Colitis Mucosa Tissue Using Deep Learning. Diagnostics (Basel) 2022; 12:diagnostics12020288. [PMID: 35204379 PMCID: PMC8870781 DOI: 10.3390/diagnostics12020288] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2021] [Revised: 01/21/2022] [Accepted: 01/21/2022] [Indexed: 12/09/2022] Open
Abstract
Confocal microscopy image analysis is a useful method for neoplasm diagnosis. Many ambiguous cases are difficult to distinguish with the naked eye, thus leading to high inter-observer variability and significant time investments for learning this method. We aimed to develop a deep learning-based neoplasm classification model that classifies confocal microscopy images of 10× magnified colon tissues into three classes: neoplasm, inflammation, and normal tissue. ResNet50 with data augmentation and transfer learning approaches was used to efficiently train the model with limited training data. A class activation map was generated by using global average pooling to confirm which areas had a major effect on the classification. The proposed method achieved an accuracy of 81%, which was 14.05% more accurate than three machine learning-based methods and 22.6% better than the predictions made by four endoscopists. ResNet50 with data augmentation and transfer learning can be utilized to effectively identify neoplasm, inflammation, and normal tissue in confocal microscopy images. The proposed method outperformed three machine learning-based methods and identified the area that had a major influence on the results. Inter-observer variability and the time required for learning can be reduced if the proposed model is used with confocal microscopy image analysis for diagnosis.
Collapse
|
27
|
Ragab M, Albukhari A. Automated Artificial Intelligence Empowered Colorectal Cancer Detection and Classification Model. COMPUTERS, MATERIALS & CONTINUA 2022; 72:5577-5591. [DOI: 10.32604/cmc.2022.026715] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/03/2022] [Accepted: 03/18/2022] [Indexed: 10/28/2024]
|
28
|
Ragab M, Eljaaly K, Farouk S. Sabir M, Bahaudien Ashary E, M. Abo-Dahab S, M. Khalil E. Optimized Deep Learning Model for Colorectal Cancer Detection and Classification Model. COMPUTERS, MATERIALS & CONTINUA 2022; 71:5751-5764. [DOI: 10.32604/cmc.2022.024658] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/26/2021] [Accepted: 12/08/2021] [Indexed: 10/28/2024]
|
29
|
Kandel P, Wallace MB. Advanced Imaging Techniques and In vivo Histology: Current Status and Future Perspectives (Lower G.I.). GASTROINTESTINAL AND PANCREATICO-BILIARY DISEASES: ADVANCED DIAGNOSTIC AND THERAPEUTIC ENDOSCOPY 2022:291-310. [DOI: 10.1007/978-3-030-56993-8_110] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/03/2025]
|
30
|
Okamoto Y, Yoshida S, Izakura S, Katayama D, Michida R, Koide T, Tamaki T, Kamigaichi Y, Tamari H, Shimohara Y, Nishimura T, Inagaki K, Tanaka H, Yamashita K, Sumimoto K, Oka S, Tanaka S. Development of multi-class computer-aided diagnostic systems using the NICE/JNET classifications for colorectal lesions. J Gastroenterol Hepatol 2022; 37:104-110. [PMID: 34478167 DOI: 10.1111/jgh.15682] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/22/2021] [Revised: 07/22/2021] [Accepted: 08/30/2021] [Indexed: 12/15/2022]
Abstract
BACKGROUND AND AIM Diagnostic support using artificial intelligence may contribute to the equalization of endoscopic diagnosis of colorectal lesions. We developed computer-aided diagnosis (CADx) support system for diagnosing colorectal lesions using the NBI International Colorectal Endoscopic (NICE) classification and the Japan NBI Expert Team (JNET) classification. METHODS Using Residual Network as the classifier and NBI images as training images, we developed a CADx based on the NICE classification (CADx-N) and a CADx based on the JNET classification (CADx-J). For validation, 480 non-magnifying and magnifying NBI images were used for the CADx-N and 320 magnifying NBI images were used for the CADx-J. The diagnostic performance of the CADx-N was evaluated using the magnification rate. RESULTS The accuracy of the CADx-N for Types 1, 2, and 3 was 97.5%, 91.2%, and 93.8%, respectively. The diagnostic performance for each magnification level was good (no statistically significant difference). The sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of the CADx-J were 100%, 96.3%, 82.8%, 100%, and 96.9% for Type 1; 80.3%, 93.7%, 94.1%, 79.2%, and 86.3% for Type 2A; 80.4%, 84.7%, 46.8%, 96.3%, and 84.1% for Type 2B; and 62.5%, 99.6%, 96.8%, 93.8%, and 94.1% for Type 3, respectively. CONCLUSIONS The multi-class CADx systems had good diagnostic performance with both the NICE and JNET classifications and may aid in educating non-expert endoscopists and assist in diagnosing colorectal lesions.
Collapse
Affiliation(s)
- Yuki Okamoto
- Department of Gastroenterology and Metabolism, Hiroshima University Hospital, Hiroshima, Japan
| | - Shigeto Yoshida
- Department of Gastroenterology, JR Hiroshima Hospital, Hiroshima, Japan
| | - Seiji Izakura
- Research Institute for Nanodevice and Bio Systems, Hiroshima University, Hiroshima, Japan
| | - Daisuke Katayama
- Research Institute for Nanodevice and Bio Systems, Hiroshima University, Hiroshima, Japan
| | - Ryuichi Michida
- Research Institute for Nanodevice and Bio Systems, Hiroshima University, Hiroshima, Japan
| | - Tetsushi Koide
- Research Institute for Nanodevice and Bio Systems, Hiroshima University, Hiroshima, Japan
| | - Toru Tamaki
- Department of Computer Science, Nagoya Institute of Technology, Nagoya, Japan
| | - Yuki Kamigaichi
- Department of Gastroenterology and Metabolism, Hiroshima University Hospital, Hiroshima, Japan
| | - Hirosato Tamari
- Department of Gastroenterology and Metabolism, Hiroshima University Hospital, Hiroshima, Japan
| | - Yasutsugu Shimohara
- Department of Gastroenterology and Metabolism, Hiroshima University Hospital, Hiroshima, Japan
| | - Tomoyuki Nishimura
- Department of Gastroenterology and Metabolism, Hiroshima University Hospital, Hiroshima, Japan
| | - Katsuaki Inagaki
- Department of Gastroenterology and Metabolism, Hiroshima University Hospital, Hiroshima, Japan
| | - Hidenori Tanaka
- Department of Endoscopy, Hiroshima University Hospital, Hiroshima, Japan
| | - Ken Yamashita
- Department of Endoscopy, Hiroshima University Hospital, Hiroshima, Japan
| | - Kyoku Sumimoto
- Department of Endoscopy, Hiroshima University Hospital, Hiroshima, Japan
| | - Shiro Oka
- Department of Gastroenterology and Metabolism, Hiroshima University Hospital, Hiroshima, Japan
| | - Shinji Tanaka
- Department of Endoscopy, Hiroshima University Hospital, Hiroshima, Japan
| |
Collapse
|
31
|
Cianci P, Restini E. Artificial intelligence in colorectal cancer management. Artif Intell Cancer 2021; 2:79-89. [DOI: 10.35713/aic.v2.i6.79] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/09/2021] [Revised: 12/22/2021] [Accepted: 12/29/2021] [Indexed: 02/06/2023] Open
Abstract
Artificial intelligence (AI) is a new branch of computer science involving many disciplines and technologies. Since its application in the medical field, it has been constantly studied and developed. AI includes machine learning and neural networks to create new technologies or to improve existing ones. Various AI supporting systems are available for a personalized and novel strategy for the management of colorectal cancer (CRC). This mini-review aims to summarize the progress of research and possible clinical applications of AI in the investigation, early diagnosis, treatment, and management of CRC, to offer elements of knowledge as a starting point for new studies and future applications.
Collapse
Affiliation(s)
- Pasquale Cianci
- Department of Surgery and Traumatology, ASL BAT, Lorenzo Bonomo Hospital, Andria 76123, Puglia, Italy
| | - Enrico Restini
- Department of Surgery and Traumatology, ASL BAT, Lorenzo Bonomo Hospital, Andria 76123, Puglia, Italy
| |
Collapse
|
32
|
Kudo SE, Mori Y, Abdel-Aal UM, Misawa M, Itoh H, Oda M, Mori K. Artificial intelligence and computer-aided diagnosis for colonoscopy: where do we stand now? Transl Gastroenterol Hepatol 2021; 6:64. [PMID: 34805586 DOI: 10.21037/tgh.2019.12.14] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/10/2019] [Accepted: 11/12/2019] [Indexed: 12/22/2022] Open
Abstract
Computer-aided diagnosis (CAD) for colonoscopy with use of artificial intelligence (AI) is catching increased attention of endoscopists. CAD allows automated detection and pathological prediction, namely optical biopsy, of colorectal polyps during real-time endoscopy, which help endoscopists avoid missing and/or misdiagnosing colorectal lesions. With the increased number of publications in this field and emergence of the AI medical device that have already secured regulatory approval, CAD in colonoscopy is now being implemented into clinical practice. On the other side, drawbacks and weak points of CAD in colonoscopy have not been thoroughly discussed. In this review, we provide an overview of CAD for optical biopsy of colorectal lesions with a particular focus on its clinical applications and limitations.
Collapse
Affiliation(s)
- Shin-Ei Kudo
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan
| | - Yuichi Mori
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan
| | - Usama M Abdel-Aal
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan.,Internal Medicine, Faculty of Medicine, Sohag University, Sohag, Egypt
| | - Masashi Misawa
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan
| | - Hayato Itoh
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
| | - Masahiro Oda
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
| | - Kensaku Mori
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
| |
Collapse
|
33
|
Cai YW, Dong FF, Shi YH, Lu LY, Chen C, Lin P, Xue YS, Chen JH, Chen SY, Luo XB. Deep learning driven colorectal lesion detection in gastrointestinal endoscopic and pathological imaging. World J Clin Cases 2021; 9:9376-9385. [PMID: 34877273 PMCID: PMC8610875 DOI: 10.12998/wjcc.v9.i31.9376] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/15/2021] [Revised: 07/26/2021] [Accepted: 08/13/2021] [Indexed: 02/06/2023] Open
Abstract
Colorectal cancer has the second highest incidence of malignant tumors and is the fourth leading cause of cancer deaths in China. Early diagnosis and treatment of colorectal cancer will lead to an improvement in the 5-year survival rate, which will reduce medical costs. The current diagnostic methods for early colorectal cancer include excreta, blood, endoscopy, and computer-aided endoscopy. In this paper, research on image analysis and prediction of colorectal cancer lesions based on deep learning is reviewed with the goal of providing a reference for the early diagnosis of colorectal cancer lesions by combining computer technology, 3D modeling, 5G remote technology, endoscopic robot technology, and surgical navigation technology. The findings will supplement the research and provide insights to improve the cure rate and reduce the mortality of colorectal cancer.
Collapse
Affiliation(s)
- Yu-Wen Cai
- Department of Clinical Medicine, Fujian Medical University, Fuzhou 350004, Fujian Province, China
| | - Fang-Fen Dong
- Department of Medical Technology and Engineering, Fujian Medical University, Fuzhou 350004, Fujian Province, China
| | - Yu-Heng Shi
- Computer Science and Engineering College, University of Alberta, Edmonton T6G 2R3, Canada
| | - Li-Yuan Lu
- Department of Clinical Medicine, Fujian Medical University, Fuzhou 350004, Fujian Province, China
| | - Chen Chen
- Department of Clinical Medicine, Fujian Medical University, Fuzhou 350004, Fujian Province, China
| | - Ping Lin
- Department of Clinical Medicine, Fujian Medical University, Fuzhou 350004, Fujian Province, China
| | - Yu-Shan Xue
- Department of Clinical Medicine, Fujian Medical University, Fuzhou 350004, Fujian Province, China
| | - Jian-Hua Chen
- Endoscopy Center, Fujian Cancer Hospital, Fujian Medical University Cancer Hospital, Fuzhou 350014, Fujian Province, China
| | - Su-Yu Chen
- Endoscopy Center, Fujian Cancer Hospital, Fujian Medical University Cancer Hospital, Fuzhou 350014, Fujian Province, China
| | - Xiong-Biao Luo
- Department of Computer Science, Xiamen University, Xiamen 361005, Fujian, China
| |
Collapse
|
34
|
Goyal H, Sherazi SAA, Mann R, Gandhi Z, Perisetti A, Aziz M, Chandan S, Kopel J, Tharian B, Sharma N, Thosani N. Scope of Artificial Intelligence in Gastrointestinal Oncology. Cancers (Basel) 2021; 13:5494. [PMID: 34771658 PMCID: PMC8582733 DOI: 10.3390/cancers13215494] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2021] [Accepted: 10/27/2021] [Indexed: 12/12/2022] Open
Abstract
Gastrointestinal cancers are among the leading causes of death worldwide, with over 2.8 million deaths annually. Over the last few decades, advancements in artificial intelligence technologies have led to their application in medicine. The use of artificial intelligence in endoscopic procedures is a significant breakthrough in modern medicine. Currently, the diagnosis of various gastrointestinal cancer relies on the manual interpretation of radiographic images by radiologists and various endoscopic images by endoscopists. This can lead to diagnostic variabilities as it requires concentration and clinical experience in the field. Artificial intelligence using machine or deep learning algorithms can provide automatic and accurate image analysis and thus assist in diagnosis. In the field of gastroenterology, the application of artificial intelligence can be vast from diagnosis, predicting tumor histology, polyp characterization, metastatic potential, prognosis, and treatment response. It can also provide accurate prediction models to determine the need for intervention with computer-aided diagnosis. The number of research studies on artificial intelligence in gastrointestinal cancer has been increasing rapidly over the last decade due to immense interest in the field. This review aims to review the impact, limitations, and future potentials of artificial intelligence in screening, diagnosis, tumor staging, treatment modalities, and prediction models for the prognosis of various gastrointestinal cancers.
Collapse
Affiliation(s)
- Hemant Goyal
- Department of Internal Medicine, The Wright Center for Graduate Medical Education, 501 S. Washington Avenue, Scranton, PA 18505, USA
| | - Syed A. A. Sherazi
- Department of Medicine, John H Stroger Jr Hospital of Cook County, 1950 W Polk St, Chicago, IL 60612, USA;
| | - Rupinder Mann
- Department of Medicine, Saint Agnes Medical Center, 1303 E. Herndon Ave, Fresno, CA 93720, USA;
| | - Zainab Gandhi
- Department of Medicine, Geisinger Wyoming Valley Medical Center, 1000 E Mountain Dr, Wilkes-Barre, PA 18711, USA;
| | - Abhilash Perisetti
- Division of Interventional Oncology & Surgical Endoscopy (IOSE), Parkview Cancer Institute, 11050 Parkview Circle, Fort Wayne, IN 46845, USA; (A.P.); (N.S.)
| | - Muhammad Aziz
- Department of Gastroenterology and Hepatology, University of Toledo Medical Center, 3000 Arlington Avenue, Toledo, OH 43614, USA;
| | - Saurabh Chandan
- Division of Gastroenterology and Hepatology, CHI Health Creighton University Medical Center, 7500 Mercy Rd, Omaha, NE 68124, USA;
| | - Jonathan Kopel
- Department of Medicine, Texas Tech University Health Sciences Center, 3601 4th St, Lubbock, TX 79430, USA;
| | - Benjamin Tharian
- Department of Gastroenterology and Hepatology, The University of Arkansas for Medical Sciences, 4301 W Markham St, Little Rock, AR 72205, USA;
| | - Neil Sharma
- Division of Interventional Oncology & Surgical Endoscopy (IOSE), Parkview Cancer Institute, 11050 Parkview Circle, Fort Wayne, IN 46845, USA; (A.P.); (N.S.)
| | - Nirav Thosani
- Division of Gastroenterology, Hepatology & Nutrition, McGovern Medical School, UTHealth, 6410 Fannin, St #1014, Houston, TX 77030, USA;
| |
Collapse
|
35
|
Chen Z, Lin L, Wu C, Li C, Xu R, Sun Y. Artificial intelligence for assisting cancer diagnosis and treatment in the era of precision medicine. Cancer Commun (Lond) 2021; 41:1100-1115. [PMID: 34613667 PMCID: PMC8626610 DOI: 10.1002/cac2.12215] [Citation(s) in RCA: 65] [Impact Index Per Article: 16.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2021] [Revised: 07/10/2021] [Accepted: 09/01/2021] [Indexed: 12/12/2022] Open
Abstract
Over the past decade, artificial intelligence (AI) has contributed substantially to the resolution of various medical problems, including cancer. Deep learning (DL), a subfield of AI, is characterized by its ability to perform automated feature extraction and has great power in the assimilation and evaluation of large amounts of complicated data. On the basis of a large quantity of medical data and novel computational technologies, AI, especially DL, has been applied in various aspects of oncology research and has the potential to enhance cancer diagnosis and treatment. These applications range from early cancer detection, diagnosis, classification and grading, molecular characterization of tumors, prediction of patient outcomes and treatment responses, personalized treatment, automatic radiotherapy workflows, novel anti-cancer drug discovery, and clinical trials. In this review, we introduced the general principle of AI, summarized major areas of its application for cancer diagnosis and treatment, and discussed its future directions and remaining challenges. As the adoption of AI in clinical use is increasing, we anticipate the arrival of AI-powered cancer care.
Collapse
Affiliation(s)
- Zi‐Hang Chen
- Department of Radiation OncologyState Key Laboratory of Oncology in South ChinaCollaborative Innovation Center for Cancer MedicineGuangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and TherapySun Yat‐sen University Cancer CenterGuangzhouGuangdong510060P. R. China
- Zhongshan School of MedicineSun Yat‐sen UniversityGuangzhouGuangdong510080P. R. China
| | - Li Lin
- Department of Radiation OncologyState Key Laboratory of Oncology in South ChinaCollaborative Innovation Center for Cancer MedicineGuangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and TherapySun Yat‐sen University Cancer CenterGuangzhouGuangdong510060P. R. China
| | - Chen‐Fei Wu
- Department of Radiation OncologyState Key Laboratory of Oncology in South ChinaCollaborative Innovation Center for Cancer MedicineGuangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and TherapySun Yat‐sen University Cancer CenterGuangzhouGuangdong510060P. R. China
| | - Chao‐Feng Li
- Artificial Intelligence LaboratoryState Key Laboratory of Oncology in South ChinaCollaborative Innovation Center for Cancer MedicineGuangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and TherapySun Yat‐sen University Cancer CenterGuangzhouGuangdong510060P. R. China
| | - Rui‐Hua Xu
- Department of Medical OncologyState Key Laboratory of Oncology in South ChinaCollaborative Innovation Center for Cancer MedicineGuangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and TherapySun Yat‐sen University Cancer CenterGuangzhouGuangdong510060P. R. China
| | - Ying Sun
- Department of Radiation OncologyState Key Laboratory of Oncology in South ChinaCollaborative Innovation Center for Cancer MedicineGuangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and TherapySun Yat‐sen University Cancer CenterGuangzhouGuangdong510060P. R. China
| |
Collapse
|
36
|
Kröner PT, Engels MML, Glicksberg BS, Johnson KW, Mzaik O, van Hooft JE, Wallace MB, El-Serag HB, Krittanawong C. Artificial intelligence in gastroenterology: A state-of-the-art review. World J Gastroenterol 2021; 27:6794-6824. [PMID: 34790008 PMCID: PMC8567482 DOI: 10.3748/wjg.v27.i40.6794] [Citation(s) in RCA: 52] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/11/2021] [Revised: 06/15/2021] [Accepted: 09/16/2021] [Indexed: 02/06/2023] Open
Abstract
The development of artificial intelligence (AI) has increased dramatically in the last 20 years, with clinical applications progressively being explored for most of the medical specialties. The field of gastroenterology and hepatology, substantially reliant on vast amounts of imaging studies, is not an exception. The clinical applications of AI systems in this field include the identification of premalignant or malignant lesions (e.g., identification of dysplasia or esophageal adenocarcinoma in Barrett’s esophagus, pancreatic malignancies), detection of lesions (e.g., polyp identification and classification, small-bowel bleeding lesion on capsule endoscopy, pancreatic cystic lesions), development of objective scoring systems for risk stratification, predicting disease prognosis or treatment response [e.g., determining survival in patients post-resection of hepatocellular carcinoma), determining which patients with inflammatory bowel disease (IBD) will benefit from biologic therapy], or evaluation of metrics such as bowel preparation score or quality of endoscopic examination. The objective of this comprehensive review is to analyze the available AI-related studies pertaining to the entirety of the gastrointestinal tract, including the upper, middle and lower tracts; IBD; the hepatobiliary system; and the pancreas, discussing the findings and clinical applications, as well as outlining the current limitations and future directions in this field.
Collapse
Affiliation(s)
- Paul T Kröner
- Division of Gastroenterology and Hepatology, Mayo Clinic, Jacksonville, FL 32224, United States
| | - Megan ML Engels
- Division of Gastroenterology and Hepatology, Mayo Clinic, Jacksonville, FL 32224, United States
- Cancer Center Amsterdam, Department of Gastroenterology and Hepatology, Amsterdam UMC, Location AMC, Amsterdam 1105, The Netherlands
| | - Benjamin S Glicksberg
- The Hasso Plattner Institute for Digital Health, Icahn School of Medicine at Mount Sinai, New York, NY 10029, United States
| | - Kipp W Johnson
- The Hasso Plattner Institute for Digital Health, Icahn School of Medicine at Mount Sinai, New York, NY 10029, United States
| | - Obaie Mzaik
- Division of Gastroenterology and Hepatology, Mayo Clinic, Jacksonville, FL 32224, United States
| | - Jeanin E van Hooft
- Department of Gastroenterology and Hepatology, Leiden University Medical Center, Amsterdam 2300, The Netherlands
| | - Michael B Wallace
- Division of Gastroenterology and Hepatology, Mayo Clinic, Jacksonville, FL 32224, United States
- Division of Gastroenterology and Hepatology, Sheikh Shakhbout Medical City, Abu Dhabi 11001, United Arab Emirates
| | - Hashem B El-Serag
- Section of Gastroenterology and Hepatology, Michael E. DeBakey VA Medical Center and Baylor College of Medicine, Houston, TX 77030, United States
- Section of Health Services Research, Michael E. DeBakey VA Medical Center and Baylor College of Medicine, Houston, TX 77030, United States
| | - Chayakrit Krittanawong
- Section of Health Services Research, Michael E. DeBakey VA Medical Center and Baylor College of Medicine, Houston, TX 77030, United States
- Section of Cardiology, Michael E. DeBakey VA Medical Center, Houston, TX 77030, United States
| |
Collapse
|
37
|
Artificial intelligence-enhanced white-light colonoscopy with attention guidance predicts colorectal cancer invasion depth. Gastrointest Endosc 2021; 94:627-638.e1. [PMID: 33852902 DOI: 10.1016/j.gie.2021.03.936] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/27/2020] [Accepted: 03/30/2021] [Indexed: 02/07/2023]
Abstract
BACKGROUND AND AIMS Endoscopic submucosal dissection (ESD) and EMR are applied in treating superficial colorectal neoplasms but are contraindicated by deeply invasive colorectal cancer (CRC). The invasion depth of neoplasms can be examined by an automated artificial intelligence (AI) system to determine the applicability of ESD and EMR. METHODS A deep convolutional neural network with a tumor localization branch to guide invasion depth classification was constructed on the GoogLeNet architecture. The model was trained using 7734 nonmagnified white-light colonoscopy (WLC) images supplemented by image augmentation from 657 lesions labeled with histopathologic analysis of invasion depth. An independent testing dataset consisting of 1634 WLC images from 156 lesions was used to validate the model. RESULTS For predicting noninvasive and superficially invasive neoplasms, the model achieved an overall accuracy of 91.1% (95% confidence interval [CI], 89.6%-92.4%), with 91.2% sensitivity (95% CI, 88.8%-93.3%) and 91.0% specificity (95% CI, 89.0%-92.7%) at an optimal cutoff of .41 and the area under the receiver operating characteristic (AUROC) curve of .970 (95% CI, .962-.978). Inclusion of the advanced CRC data significantly increased the sensitivity in differentiating superficial neoplasms from deeply invasive early CRC to 65.3% (95% CI, 61.9%-68.8%) with an AUROC curve of .729 (95% CI, .699-.759), similar to experienced endoscopists (.691; 95% CI, .624-.758). CONCLUSIONS We have developed an AI-enhanced attention-guided WLC system that differentiates noninvasive or superficially submucosal invasive neoplasms from deeply invasive CRC with high accuracy, sensitivity, and specificity.
Collapse
|
38
|
Yang H, Hu B. Early gastrointestinal cancer: The application of artificial intelligence. Artif Intell Gastrointest Endosc 2021; 2:185-197. [DOI: 10.37126/aige.v2.i4.185] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/11/2021] [Revised: 06/25/2021] [Accepted: 08/18/2021] [Indexed: 02/06/2023] Open
Abstract
Early gastrointestinal (GI) cancer has been the core of clinical endoscopic work. Its early detection and treatment are tightly associated with patients’ prognoses. As a novel technology, artificial intelligence has been improved and applied in the field of endoscopy. Studies on detection, diagnosis, risk, and prognosis evaluation of diseases in the GI tract have been in development, including precancerous lesions, adenoma, early GI cancers, and advanced GI cancers. In this review, research on esophagus, stomach, and colon was concluded, and associated with the process from precancerous lesions to early GI cancer, such as from Barrett’s esophagus to early esophageal cancer, from dysplasia to early gastric cancer, and from adenoma to early colonic cancer. A status quo of research on early GI cancers and artificial intelligence was provided.
Collapse
Affiliation(s)
- Hang Yang
- Department of Gastroenterology, West China Hospital, Sichuan University, Chengdu 610041, Sichuan Province, China
| | - Bing Hu
- Department of Gastroenterology, West China Hospital, Sichuan University, Chengdu 610041, Sichuan Province, China
| |
Collapse
|
39
|
Song YQ, Mao XL, Zhou XB, He SQ, Chen YH, Zhang LH, Xu SW, Yan LL, Tang SP, Ye LP, Li SW. Use of Artificial Intelligence to Improve the Quality Control of Gastrointestinal Endoscopy. Front Med (Lausanne) 2021; 8:709347. [PMID: 34368199 PMCID: PMC8339701 DOI: 10.3389/fmed.2021.709347] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2021] [Accepted: 06/29/2021] [Indexed: 12/04/2022] Open
Abstract
With the rapid development of science and technology, artificial intelligence (AI) systems are becoming ubiquitous, and their utility in gastroenteroscopy is beginning to be recognized. Digestive endoscopy is a conventional and reliable method of examining and diagnosing digestive tract diseases. However, with the increase in the number and types of endoscopy, problems such as a lack of skilled endoscopists and difference in the professional skill of doctors with different degrees of experience have become increasingly apparent. Most studies thus far have focused on using computers to detect and diagnose lesions, but improving the quality of endoscopic examination process itself is the basis for improving the detection rate and correctly diagnosing diseases. In the present study, we mainly reviewed the role of AI in monitoring systems, mainly through the endoscopic examination time, reducing the blind spot rate, improving the success rate for detecting high-risk lesions, evaluating intestinal preparation, increasing the detection rate of polyps, automatically collecting maps and writing reports. AI can even perform quality control evaluations for endoscopists, improve the detection rate of endoscopic lesions and reduce the burden on endoscopists.
Collapse
Affiliation(s)
- Ya-Qi Song
- Taizhou Hospital, Zhejiang University, Linhai, China
| | - Xin-Li Mao
- Key Laboratory of Minimally Invasive Techniques and Rapid Rehabilitation of Digestive System Tumor of Zhejiang Province, Taizhou Hospital Affiliated to Wenzhou Medical University, Linhai, China.,Department of Gastroenterology, Taizhou Hospital of Zhejiang Province Affiliated to Wenzhou Medical University, Linhai, China
| | - Xian-Bin Zhou
- Key Laboratory of Minimally Invasive Techniques and Rapid Rehabilitation of Digestive System Tumor of Zhejiang Province, Taizhou Hospital Affiliated to Wenzhou Medical University, Linhai, China.,Department of Gastroenterology, Taizhou Hospital of Zhejiang Province Affiliated to Wenzhou Medical University, Linhai, China
| | - Sai-Qin He
- Key Laboratory of Minimally Invasive Techniques and Rapid Rehabilitation of Digestive System Tumor of Zhejiang Province, Taizhou Hospital Affiliated to Wenzhou Medical University, Linhai, China.,Department of Gastroenterology, Taizhou Hospital of Zhejiang Province Affiliated to Wenzhou Medical University, Linhai, China
| | - Ya-Hong Chen
- Health Management Center, Taizhou Hospital of Zhejiang Province Affiliated to Wenzhou Medical University, Linhai, China
| | - Li-Hui Zhang
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
| | - Shi-Wen Xu
- Taizhou Hospital of Zhejiang Province Affiliated to Wenzhou Medical University, Linhai, China
| | - Ling-Ling Yan
- Key Laboratory of Minimally Invasive Techniques and Rapid Rehabilitation of Digestive System Tumor of Zhejiang Province, Taizhou Hospital Affiliated to Wenzhou Medical University, Linhai, China.,Department of Gastroenterology, Taizhou Hospital of Zhejiang Province Affiliated to Wenzhou Medical University, Linhai, China
| | - Shen-Ping Tang
- Taizhou Hospital of Zhejiang Province Affiliated to Wenzhou Medical University, Linhai, China
| | - Li-Ping Ye
- Taizhou Hospital, Zhejiang University, Linhai, China.,Key Laboratory of Minimally Invasive Techniques and Rapid Rehabilitation of Digestive System Tumor of Zhejiang Province, Taizhou Hospital Affiliated to Wenzhou Medical University, Linhai, China.,Department of Gastroenterology, Taizhou Hospital of Zhejiang Province Affiliated to Wenzhou Medical University, Linhai, China.,Institute of Digestive Disease, Taizhou Hospital of Zhejiang Province Affiliated to Wenzhou Medical University, Linhai, China
| | - Shao-Wei Li
- Key Laboratory of Minimally Invasive Techniques and Rapid Rehabilitation of Digestive System Tumor of Zhejiang Province, Taizhou Hospital Affiliated to Wenzhou Medical University, Linhai, China.,Department of Gastroenterology, Taizhou Hospital of Zhejiang Province Affiliated to Wenzhou Medical University, Linhai, China.,Institute of Digestive Disease, Taizhou Hospital of Zhejiang Province Affiliated to Wenzhou Medical University, Linhai, China
| |
Collapse
|
40
|
Chakraborty D, Ivan C, Amero P, Khan M, Rodriguez-Aguayo C, Başağaoğlu H, Lopez-Berestein G. Explainable Artificial Intelligence Reveals Novel Insight into Tumor Microenvironment Conditions Linked with Better Prognosis in Patients with Breast Cancer. Cancers (Basel) 2021; 13:3450. [PMID: 34298668 PMCID: PMC8303703 DOI: 10.3390/cancers13143450] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Revised: 07/06/2021] [Accepted: 07/06/2021] [Indexed: 12/29/2022] Open
Abstract
We investigated the data-driven relationship between immune cell composition in the tumor microenvironment (TME) and the ≥5-year survival rates of breast cancer patients using explainable artificial intelligence (XAI) models. We acquired TCGA breast invasive carcinoma data from the cbioPortal and retrieved immune cell composition estimates from bulk RNA sequencing data from TIMER2.0 based on EPIC, CIBERSORT, TIMER, and xCell computational methods. Novel insights derived from our XAI model showed that B cells, CD8+ T cells, M0 macrophages, and NK T cells are the most critical TME features for enhanced prognosis of breast cancer patients. Our XAI model also revealed the inflection points of these critical TME features, above or below which ≥5-year survival rates improve. Subsequently, we ascertained the conditional probabilities of ≥5-year survival under specific conditions inferred from the inflection points. In particular, the XAI models revealed that the B cell fraction (relative to all cells in a sample) exceeding 0.025, M0 macrophage fraction (relative to the total immune cell content) below 0.05, and NK T cell and CD8+ T cell fractions (based on cancer type-specific arbitrary units) above 0.075 and 0.25, respectively, in the TME could enhance the ≥5-year survival in breast cancer patients. The findings could lead to accurate clinical predictions and enhanced immunotherapies, and to the design of innovative strategies to reprogram the breast TME.
Collapse
Affiliation(s)
- Debaditya Chakraborty
- Department of Construction Science, The University of Texas at San Antonio, San Antonio, TX 78249, USA
| | - Cristina Ivan
- Department of Experimental Therapeutics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (C.I.); (P.A.); (C.R.-A.); (G.L.-B.)
- Center for RNA Interference and Non-Coding RNA, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Paola Amero
- Department of Experimental Therapeutics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (C.I.); (P.A.); (C.R.-A.); (G.L.-B.)
| | - Maliha Khan
- Department of Lymphoma and Myeloma, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA;
| | - Cristian Rodriguez-Aguayo
- Department of Experimental Therapeutics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (C.I.); (P.A.); (C.R.-A.); (G.L.-B.)
- Center for RNA Interference and Non-Coding RNA, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | | | - Gabriel Lopez-Berestein
- Department of Experimental Therapeutics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (C.I.); (P.A.); (C.R.-A.); (G.L.-B.)
- Center for RNA Interference and Non-Coding RNA, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| |
Collapse
|
41
|
Zhang L, Zhang Y, Wang L, Wang J, Liu Y. Diagnosis of gastric lesions through a deep convolutional neural network. Dig Endosc 2021; 33:788-796. [PMID: 32961597 DOI: 10.1111/den.13844] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/10/2020] [Revised: 08/27/2020] [Accepted: 09/14/2020] [Indexed: 12/13/2022]
Abstract
BACKGROUND AND AIMS A deep convolutional neural network (CNN) was used to achieve fast and accurate artificial intelligence (AI)-assisted diagnosis of early gastric cancer (GC) and other gastric lesions based on endoscopic images. METHODS A CNN-based diagnostic system based on a ResNet34 residual network structure and a DeepLabv3 structure was constructed and trained using 21,217 gastroendoscopic images of five gastric conditions, peptic ulcer (PU), early gastric cancer (EGC) and high-grade intraepithelial neoplasia (HGIN), advanced gastric cancer (AGC), gastric submucosal tumors (SMTs), and normal gastric mucosa without lesions. The trained CNN was evaluated using a test dataset of 1091 images. The accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the CNN were calculated. The CNN diagnosis was compared with those of 10 endoscopists with over 8 years of experience in endoscopic diagnosis. RESULTS The diagnostic specificity and PPV of the CNN were higher than that of the endoscopists for the EGC and HGIN images (specificity: 91.2% vs. 86.7%, by 4.5%, 95% CI 2.8-7.2%; PPV: 55.4% vs. 41.7%, by 13.7%, 95% CI 11.2-16.8%) and the diagnostic accuracy of the CNN was close to those of the endoscopists for the lesion-free, EGC and HGIN, PU, AGC, and SMTs images. The CNN had image recognition time of 42 s for all the test set images. CONCLUSION The constructed CNN system could be used as a rapid auxiliary diagnostic instrument to detect EGC and HGIN, as well as other gastric lesions, to reduce the workload of endoscopists.
Collapse
Affiliation(s)
- Liming Zhang
- Department of Gastroenterology, Peking University People's Hospital, Beijing, China
| | - Yang Zhang
- Internet Medical Department of Love Life Insurance Company, Beijing, China
| | - Li Wang
- Department of Gastroenterology, Peking University People's Hospital, Beijing, China
| | - Jiangyuan Wang
- Department of Gastroenterology, Peking University People's Hospital, Beijing, China
| | - Yulan Liu
- Department of Gastroenterology, Peking University People's Hospital, Beijing, China
| |
Collapse
|
42
|
Cao JS, Lu ZY, Chen MY, Zhang B, Juengpanich S, Hu JH, Li SJ, Topatana W, Zhou XY, Feng X, Shen JL, Liu Y, Cai XJ. Artificial intelligence in gastroenterology and hepatology: Status and challenges. World J Gastroenterol 2021; 27:1664-1690. [PMID: 33967550 PMCID: PMC8072192 DOI: 10.3748/wjg.v27.i16.1664] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/15/2021] [Revised: 02/11/2021] [Accepted: 03/17/2021] [Indexed: 02/06/2023] Open
Abstract
Originally proposed by John McCarthy in 1955, artificial intelligence (AI) has achieved a breakthrough and revolutionized the processing methods of clinical medicine with the increasing workloads of medical records and digital images. Doctors are paying attention to AI technologies for various diseases in the fields of gastroenterology and hepatology. This review will illustrate AI technology procedures for medical image analysis, including data processing, model establishment, and model validation. Furthermore, we will summarize AI applications in endoscopy, radiology, and pathology, such as detecting and evaluating lesions, facilitating treatment, and predicting treatment response and prognosis with excellent model performance. The current challenges for AI in clinical application include potential inherent bias in retrospective studies that requires larger samples for validation, ethics and legal concerns, and the incomprehensibility of the output results. Therefore, doctors and researchers should cooperate to address the current challenges and carry out further investigations to develop more accurate AI tools for improved clinical applications.
Collapse
Affiliation(s)
- Jia-Sheng Cao
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University, Hangzhou 310016, Zhejiang Province, China
| | - Zi-Yi Lu
- Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310058, Zhejiang Province, China
| | - Ming-Yu Chen
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University, Hangzhou 310016, Zhejiang Province, China
| | - Bin Zhang
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University, Hangzhou 310016, Zhejiang Province, China
| | - Sarun Juengpanich
- Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310058, Zhejiang Province, China
| | - Jia-Hao Hu
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University, Hangzhou 310016, Zhejiang Province, China
| | - Shi-Jie Li
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University, Hangzhou 310016, Zhejiang Province, China
| | - Win Topatana
- Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310058, Zhejiang Province, China
| | - Xue-Yin Zhou
- School of Medicine, Wenzhou Medical University, Wenzhou 325035, Zhejiang Province, China
| | - Xu Feng
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University, Hangzhou 310016, Zhejiang Province, China
| | - Ji-Liang Shen
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University, Hangzhou 310016, Zhejiang Province, China
| | - Yu Liu
- College of Life Sciences, Zhejiang University, Hangzhou 310058, Zhejiang Province, China
| | - Xiu-Jun Cai
- Department of General Surgery, Sir Run-Run Shaw Hospital, Zhejiang University, Hangzhou 310016, Zhejiang Province, China
| |
Collapse
|
43
|
Alloro R, Sinagra E. Artificial intelligence and colorectal cancer: How far can you go? Artif Intell Cancer 2021; 2:7-11. [DOI: 10.35713/aic.v2.i2.7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/24/2021] [Revised: 04/01/2021] [Accepted: 04/20/2021] [Indexed: 02/06/2023] Open
Abstract
Artificial intelligence is an emerging technology whose application is rapidly increasing in several medical fields. The numerous applications of artificial intelligence in gastroenterology have shown promising results, especially in the setting of gastrointestinal oncology. Therefore, we would like to highlight and summarize the research progress and clinical application value of artificial intelligence in the diagnosis, treatment, and prognosis of colorectal cancer to provide evidence for its use as a promising diagnostic and therapeutic tool in this setting.
Collapse
Affiliation(s)
- Rita Alloro
- Department of Surgical, Oncological and Oral Sciences (Di.Chir.On.S.), Unit of General and Oncological Surgery, Paolo Giaccone University Hospital, University of Palermo, Palermo 90127, Italy
| | - Emanuele Sinagra
- Gastroenterology and Endoscopy Unit, Fondazione Istituto G. Giglio, Palermo 90015, Italy
| |
Collapse
|
44
|
Computer-aided diagnosis system using only white-light endoscopy for the prediction of invasion depth in colorectal cancer. Gastrointest Endosc 2021; 93:647-653. [PMID: 32735946 DOI: 10.1016/j.gie.2020.07.053] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/23/2020] [Accepted: 07/24/2020] [Indexed: 02/06/2023]
Abstract
BACKGROUND AND AIMS Endoscopic treatment is recommended for low-grade dysplasia (LGD), high-grade dysplasia (HGD), and colorectal cancer (CRC) with submucosal (SM) invasion <1000 μm. However, diagnosis of invasion depth requires experience and is often difficult. This study developed and evaluated a novel computer-aided diagnosis (CAD) system to determine whether endoscopic treatment is appropriate for colorectal lesions using only white-light endoscopy (WLE). METHODS We extracted 3442 images from 1035 consecutive colorectal lesions (105 LGDs, 377 HGDs, 107 CRCs with SM <1000 μm, 146 CRCs with SM ≥1000 μm, and 300 advanced CRCs). All images were WLE, nonmagnified, and nonstained. We developed a novel CAD system using 2751 images; the remaining 691 images were evaluated by the CAD system as a test set. The capability of the CAD system to distinguish endoscopically treatable lesions and untreatable lesions was assessed and compared with the results from 2 trainees and 2 experts. RESULTS The CAD system distinguished endoscopically treatable from untreatable lesions with 96.7% sensitivity, 75.0% specificity, and 90.3% accuracy. These values were significantly higher than those from trainees (92.1%, 67.6%, and 84.9%; P < .01, <.01, and <.01, respectively) and were comparable with those from experts (96.5%, 72.5%, and 89.4%, respectively). Trainees assisted by the CAD system demonstrated a diagnostic capability comparable with that of experts. CONCLUSIONS The CAD system had good diagnostic capability for making treatment decisions for colorectal lesions. This system may enable a more convenient and accurate diagnosis using only WLE.
Collapse
|
45
|
Mohapatra S, Swarnkar T, Mishra M, Al-Dabass D, Mascella R. Deep learning in gastroenterology. HANDBOOK OF COMPUTATIONAL INTELLIGENCE IN BIOMEDICAL ENGINEERING AND HEALTHCARE 2021:121-149. [DOI: 10.1016/b978-0-12-822260-7.00001-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/07/2025]
|
46
|
Pannala R, Krishnan K, Melson J, Parsi MA, Schulman AR, Sullivan S, Trikudanathan G, Trindade AJ, Watson RR, Maple JT, Lichtenstein DR. Artificial intelligence in gastrointestinal endoscopy. VIDEOGIE : AN OFFICIAL VIDEO JOURNAL OF THE AMERICAN SOCIETY FOR GASTROINTESTINAL ENDOSCOPY 2020; 5:598-613. [PMID: 33319126 PMCID: PMC7732722 DOI: 10.1016/j.vgie.2020.08.013] [Citation(s) in RCA: 38] [Impact Index Per Article: 7.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 02/08/2023]
Abstract
BACKGROUND AND AIMS Artificial intelligence (AI)-based applications have transformed several industries and are widely used in various consumer products and services. In medicine, AI is primarily being used for image classification and natural language processing and has great potential to affect image-based specialties such as radiology, pathology, and gastroenterology (GE). This document reviews the reported applications of AI in GE, focusing on endoscopic image analysis. METHODS The MEDLINE database was searched through May 2020 for relevant articles by using key words such as machine learning, deep learning, artificial intelligence, computer-aided diagnosis, convolutional neural networks, GI endoscopy, and endoscopic image analysis. References and citations of the retrieved articles were also evaluated to identify pertinent studies. The manuscript was drafted by 2 authors and reviewed in person by members of the American Society for Gastrointestinal Endoscopy Technology Committee and subsequently by the American Society for Gastrointestinal Endoscopy Governing Board. RESULTS Deep learning techniques such as convolutional neural networks have been used in several areas of GI endoscopy, including colorectal polyp detection and classification, analysis of endoscopic images for diagnosis of Helicobacter pylori infection, detection and depth assessment of early gastric cancer, dysplasia in Barrett's esophagus, and detection of various abnormalities in wireless capsule endoscopy images. CONCLUSIONS The implementation of AI technologies across multiple GI endoscopic applications has the potential to transform clinical practice favorably and improve the efficiency and accuracy of current diagnostic methods.
Collapse
Key Words
- ADR, adenoma detection rate
- AI, artificial intelligence
- AMR, adenoma miss rate
- ANN, artificial neural network
- BE, Barrett’s esophagus
- CAD, computer-aided diagnosis
- CADe, CAD studies for colon polyp detection
- CADx, CAD studies for colon polyp classification
- CI, confidence interval
- CNN, convolutional neural network
- CRC, colorectal cancer
- DL, deep learning
- GI, gastroenterology
- HD-WLE, high-definition white light endoscopy
- HDWL, high-definition white light
- ML, machine learning
- NBI, narrow-band imaging
- NPV, negative predictive value
- PIVI, preservation and Incorporation of Valuable Endoscopic Innovations
- SVM, support vector machine
- VLE, volumetric laser endomicroscopy
- WCE, wireless capsule endoscopy
- WL, white light
Collapse
Affiliation(s)
- Rahul Pannala
- Department of Gastroenterology and Hepatology, Mayo Clinic, Scottsdale, Arizona
| | - Kumar Krishnan
- Division of Gastroenterology, Department of Internal Medicine, Harvard Medical School and Massachusetts General Hospital, Boston, Massachusetts
| | - Joshua Melson
- Division of Digestive Diseases, Department of Internal Medicine, Rush University Medical Center, Chicago, Illinois
| | - Mansour A Parsi
- Section for Gastroenterology and Hepatology, Tulane University Health Sciences Center, New Orleans, Louisiana
| | - Allison R Schulman
- Department of Gastroenterology, Michigan Medicine, University of Michigan, Ann Arbor, Michigan
| | - Shelby Sullivan
- Division of Gastroenterology and Hepatology, University of Colorado School of Medicine, Aurora, Colorado
| | - Guru Trikudanathan
- Department of Gastroenterology, Hepatology and Nutrition, University of Minnesota, Minneapolis, Minnesota
| | - Arvind J Trindade
- Department of Gastroenterology, Zucker School of Medicine at Hofstra/Northwell, Long Island Jewish Medical Center, New Hyde Park, New York
| | - Rabindra R Watson
- Department of Gastroenterology, Interventional Endoscopy Services, California Pacific Medical Center, San Francisco, California
| | - John T Maple
- Division of Digestive Diseases and Nutrition, University of Oklahoma Health Sciences Center, Oklahoma City, Oklahoma
| | - David R Lichtenstein
- Division of Gastroenterology, Boston Medical Center, Boston University School of Medicine, Boston, Massachusetts
| |
Collapse
|
47
|
Wang Y, Nie H, He X, Liao Z, Zhou Y, Zhou J, Ou C. The emerging role of super enhancer-derived noncoding RNAs in human cancer. Theranostics 2020; 10:11049-11062. [PMID: 33042269 PMCID: PMC7532672 DOI: 10.7150/thno.49168] [Citation(s) in RCA: 50] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2020] [Accepted: 08/23/2020] [Indexed: 02/06/2023] Open
Abstract
Super enhancers (SEs) are large clusters of adjacent enhancers that drive the expression of genes which regulate cellular identity; SE regions can be enriched with a high density of transcription factors, co-factors, and enhancer-associated epigenetic modifications. Through enhanced activation of their target genes, SEs play an important role in various diseases and conditions, including cancer. Recent studies have shown that SEs not only activate the transcriptional expression of coding genes to directly regulate biological functions, but also drive the transcriptional expression of non-coding RNAs (ncRNAs) to indirectly regulate biological functions. SE-derived ncRNAs play critical roles in tumorigenesis, including malignant proliferation, metastasis, drug resistance, and inflammatory response. Moreover, the abnormal expression of SE-derived ncRNAs is closely related to the clinical and pathological characterization of tumors. In this review, we summarize the functions and roles of SE-derived ncRNAs in tumorigenesis and discuss their prospective applications in tumor therapy. A deeper understanding of the potential mechanism underlying the action of SE-derived ncRNAs in tumorigenesis may provide new strategies for the early diagnosis of tumors and targeted therapy.
Collapse
|
48
|
Mohan BP, Khan SR, Kassab LL, Ponnada S, Dulai PS, Kochhar GS. Accuracy of convolutional neural network-based artificial intelligence in diagnosis of gastrointestinal lesions based on endoscopic images: A systematic review and meta-analysis. Endosc Int Open 2020; 8:E1584-E1594. [PMID: 33140014 PMCID: PMC7581460 DOI: 10.1055/a-1236-3007] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/22/2020] [Accepted: 07/08/2020] [Indexed: 12/29/2022] Open
Abstract
Background and study aims Recently, a growing body of evidence has been amassed on evaluation of artificial intelligence (AI) known as deep learning in computer-aided diagnosis of gastrointestinal lesions by means of convolutional neural networks (CNN). We conducted this meta-analysis to study pooled rates of performance for CNN-based AI in diagnosis of gastrointestinal neoplasia from endoscopic images. Methods Multiple databases were searched (from inception to November 2019) and studies that reported on the performance of AI by means of CNN in the diagnosis of gastrointestinal tumors were selected. A random effects model was used and pooled accuracy, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were calculated. Pooled rates were categorized based on the gastrointestinal location of lesion (esophagus, stomach and colorectum). Results Nineteen studies were included in our final analysis. The pooled accuracy of CNN in esophageal neoplasia was 87.2 % (76-93.6) and NPV was 92.1 % (85.9-95.7); the accuracy in lesions of stomach was 85.8 % (79.8-90.3) and NPV was 92.1 % (85.9-95.7); and in colorectal neoplasia the accuracy was 89.9 % (82-94.7) and NPV was 94.3 % (86.4-97.7). Conclusions Based on our meta-analysis, CNN-based AI achieved high accuracy in diagnosis of lesions in esophagus, stomach, and colorectum.
Collapse
Affiliation(s)
- Babu P Mohan
- Gastroenterology & Hepatology, University of Utah Health, Salt Lake City, Utah, United States
| | - Shahab R Khan
- Gastroenterology, Rush University Medical Center, Chicago, Illinois, United States
| | - Lena L Kassab
- Internal Medicine, Mayo Clinic, Rochester, Minnesota, United States
| | - Suresh Ponnada
- Internal Medicine, Roanoke Medical Center, Roanoke, Virginia, United States
| | - Parambir S Dulai
- Gastroenterology and Hepatology, University of California, San Diego, California, United States
| | - Gursimran S Kochhar
- Division of Gastroenterology and Hepatology, Allegheny Health Network, Pittsburgh, Pennsylvania, United States
| |
Collapse
|
49
|
Nakajima Y, Zhu X, Nemoto D, Li Q, Guo Z, Katsuki S, Hayashi Y, Utano K, Aizawa M, Takezawa T, Sagara Y, Shibukawa G, Yamamoto H, Lefor AK, Togashi K. Diagnostic performance of artificial intelligence to identify deeply invasive colorectal cancer on non-magnified plain endoscopic images. Endosc Int Open 2020; 8:E1341-E1348. [PMID: 33015336 PMCID: PMC7508661 DOI: 10.1055/a-1220-6596] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/23/2020] [Accepted: 06/24/2020] [Indexed: 02/07/2023] Open
Abstract
Background and study aims Colorectal cancers (CRC) with deep submucosal invasion (T1b) could be metastatic lesions. However, endoscopic images of T1b CRC resemble those of mucosal CRCs (Tis) or with superficial invasion (T1a). The aim of this study was to develop an automatic computer-aided diagnosis (CAD) system to identify T1b CRC based on plain endoscopic images. Patients and methods In two hospitals, 1839 non-magnified plain endoscopic images from 313 CRCs (Tis 134, T1a 46, T1b 56, beyond T1b 37) with sessile morphology were extracted for training. A CAD system was trained with the data augmented by rotation, saturation, resizing and exposure adjustment. Diagnostic performance was assessed using another dataset including 44 CRCs (Tis 23, T1b 21) from a third hospital. CAD generated a probability level for T1b diagnosis for each image, and > 95 % of probability level was defined as T1b. Lesions with at least one image with a probability level > 0.95 were regarded as T1b. Primary outcome is specificity. Six physicians separately read the same testing dataset. Results Specificity was 87 % (95 % confidence interval: 66-97) for CAD, 100 % (85-100) for Expert 1, 96 % (78-100) for Expert 2, 61 % (39-80) for both gastroenterology trainees, 48 % (27-69) for Novice 1 and 22 % (7-44) for Novice 2. Significant differences were observed between CAD and both novices ( P = 0.013, P = 0.0003). Other diagnostic values of CAD were slightly lower than of the two experts. Conclusions Specificity of CAD was superior to novices and possibly to gastroenterology trainees but slightly inferior to experts.
Collapse
Affiliation(s)
- Yuki Nakajima
- Coloproctology & Gastroenterology, Aizu Medical Center, Fukushima Medical University, Japan
| | - Xin Zhu
- Biomedical Information Engineering Lab, the University of Aizu, Japan
| | - Daiki Nemoto
- Coloproctology & Gastroenterology, Aizu Medical Center, Fukushima Medical University, Japan
| | - Qin Li
- Biomedical Information Engineering Lab, the University of Aizu, Japan
| | - Zhe Guo
- Biomedical Information Engineering Lab, the University of Aizu, Japan
| | | | | | - Kenichi Utano
- Coloproctology & Gastroenterology, Aizu Medical Center, Fukushima Medical University, Japan
| | - Masato Aizawa
- Coloproctology & Gastroenterology, Aizu Medical Center, Fukushima Medical University, Japan
| | | | | | - Goro Shibukawa
- Coloproctology & Gastroenterology, Aizu Medical Center, Fukushima Medical University, Japan
| | | | | | - Kazutomo Togashi
- Coloproctology & Gastroenterology, Aizu Medical Center, Fukushima Medical University, Japan
| |
Collapse
|
50
|
Kaul V, Enslin S, Gross SA. History of artificial intelligence in medicine. Gastrointest Endosc 2020; 92:807-812. [PMID: 32565184 DOI: 10.1016/j.gie.2020.06.040] [Citation(s) in RCA: 254] [Impact Index Per Article: 50.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/30/2020] [Accepted: 06/11/2020] [Indexed: 02/06/2023]
Abstract
Artificial intelligence (AI) was first described in 1950; however, several limitations in early models prevented widespread acceptance and application to medicine. In the early 2000s, many of these limitations were overcome by the advent of deep learning. Now that AI systems are capable of analyzing complex algorithms and self-learning, we enter a new age in medicine where AI can be applied to clinical practice through risk assessment models, improving diagnostic accuracy and workflow efficiency. This article presents a brief historical perspective on the evolution of AI over the last several decades and the introduction and development of AI in medicine in recent years. A brief summary of the major applications of AI in gastroenterology and endoscopy are also presented, which are reviewed in further detail by several other articles in this issue of Gastrointestinal Endoscopy.
Collapse
Affiliation(s)
- Vivek Kaul
- Division of Gastroenterology & Hepatology, University of Rochester Medical Center, Rochester, New York, USA
| | - Sarah Enslin
- Division of Gastroenterology & Hepatology, University of Rochester Medical Center, Rochester, New York, USA
| | - Seth A Gross
- Division of Gastroenterology & Hepatology, NYU Langone Health System, New York, New York, USA
| |
Collapse
|