|
1
|
Cai Y, Chen X, Chen J, Liao J, Han M, Lin D, Hong X, Hu H, Hu J. Deep learning-assisted colonoscopy images for prediction of mismatch repair deficiency in colorectal cancer. Surg Endosc 2025; 39:859-867. [PMID: 39623175 DOI: 10.1007/s00464-024-11426-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2024] [Accepted: 11/12/2024] [Indexed: 02/06/2025]
Abstract
BACKGROUND Deficient mismatch repair or microsatellite instability is a major predictive biomarker for the efficacy of immune checkpoint inhibitors of colorectal cancer. However, routine testing has not been uniformly implemented due to cost and resource constraints. METHODS We developed and validated a deep learning-based classifiers to detect mismatch repair-deficient status from routine colonoscopy images. We obtained the colonoscopy images from the imaging database at Endoscopic Center of the Sixth Affiliated Hospital, Sun Yat-sen University. Colonoscopy images from a prospective trial (Neoadjuvant PD-1 blockade by toripalimab with or without celecoxib in mismatch repair-deficient or microsatellite instability-high locally advanced colorectal cancer) were used to test the model. RESULTS A total of 5226 eligible images from 892 tumors from the consecutive patients were utilized to develop and validate the deep learning model. 2105 colorectal cancer images from 306 tumors were randomly selected to form model development dataset with a class-balanced approach. 3121 images of 488 proficient mismatch repair tumors and 98 deficient mismatch repair tumors were used to form the independent dataset. The model achieved an AUROC of 0.948 (95% CI 0.919-0.977) on the test dataset. On the independent validation dataset, the AUROC was 0.807 (0.760-0.854), and the NPV in was 94.2% (95% CI 0.918-0.967). On the prospective trial dataset, the model identified 29 tumors among the 33 deficient mismatch repair tumors (87.88%). CONCLUSIONS The model achieved a high NPV in detecting deficient mismatch repair colorectal cancers. This model might serve as an automatic screening tool.
Collapse
Affiliation(s)
- Yue Cai
- Department of Medical Oncology, The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China
- Guangdong Provincial Key Laboratory of Colorectal and Pelvic Floor Disease, Guangdong Research Institute of Gastroenterology, The Sixth Affiliated Hospital, Sun Yat-sen University, 26 Yuancun Erheng Road, Guangzhou, Guangdong, China
| | - Xijie Chen
- Guangdong Provincial Key Laboratory of Colorectal and Pelvic Floor Disease, Guangdong Research Institute of Gastroenterology, The Sixth Affiliated Hospital, Sun Yat-sen University, 26 Yuancun Erheng Road, Guangzhou, Guangdong, China
- Biomedical Innovation Center, The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China
- Department of General Surgery (Gastric Surgery), The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Junguo Chen
- Department of Thoracic Surgery, The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - James Liao
- Guangzhou Aptiligent Technology Co. Ltd., Guangzhou, Guangdong, China
| | - Ming Han
- Guangzhou Aptiligent Technology Co. Ltd., Guangzhou, Guangdong, China
| | - Dezheng Lin
- Guangdong Provincial Key Laboratory of Colorectal and Pelvic Floor Disease, Guangdong Research Institute of Gastroenterology, The Sixth Affiliated Hospital, Sun Yat-sen University, 26 Yuancun Erheng Road, Guangzhou, Guangdong, China
- Biomedical Innovation Center, The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China
- Department of General Surgery (Endoscopic Surgery), The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Xiaoling Hong
- Guangdong Provincial Key Laboratory of Colorectal and Pelvic Floor Disease, Guangdong Research Institute of Gastroenterology, The Sixth Affiliated Hospital, Sun Yat-sen University, 26 Yuancun Erheng Road, Guangzhou, Guangdong, China
- Biomedical Innovation Center, The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China
- Department of General Surgery (Endoscopic Surgery), The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China
| | - Huabin Hu
- Department of Medical Oncology, The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China.
- Guangdong Provincial Key Laboratory of Colorectal and Pelvic Floor Disease, Guangdong Research Institute of Gastroenterology, The Sixth Affiliated Hospital, Sun Yat-sen University, 26 Yuancun Erheng Road, Guangzhou, Guangdong, China.
| | - Jiancong Hu
- Guangdong Provincial Key Laboratory of Colorectal and Pelvic Floor Disease, Guangdong Research Institute of Gastroenterology, The Sixth Affiliated Hospital, Sun Yat-sen University, 26 Yuancun Erheng Road, Guangzhou, Guangdong, China.
- Biomedical Innovation Center, The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China.
- Department of General Surgery (Endoscopic Surgery), The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China.
| |
Collapse
|
|
2
|
Nakajima Y, Nemoto D, Guo Z, Boyuan P, Ruiyao Z, Katsuki S, Takezawa T, Maemoto R, Kawasaki K, Inoue K, Akutagawa T, Tanaka H, Sato K, Omori T, Hayashi Y, Miyakura Y, Matsumoto T, Yoshida N, Esaki M, Uraoka T, Kato H, Inoue Y, Yamamoto H, Zhu X, Togashi K. Differences in regions of interest to identify deeply invasive colorectal cancers: Computer-aided diagnosis vs expert endoscopists. Endosc Int Open 2024; 12:E1260-E1266. [PMID: 39524197 PMCID: PMC11543284 DOI: 10.1055/a-2401-6611] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Accepted: 08/23/2024] [Indexed: 11/16/2024] Open
Abstract
Background and study aims Diagnostic performance of a computer-aided diagnosis (CAD) system for deep submucosally invasive (T1b) colorectal cancer was excellent, but the "regions of interest" (ROI) within images are not obvious. Class activation mapping (CAM) enables identification of the ROI that CAD utilizes for diagnosis. The purpose of this study was a quantitative investigation of the difference between CAD and endoscopists. Patients and methods Endoscopic images collected for validation of a previous study were used, including histologically proven T1b colorectal cancers (n = 82; morphology: flat 36, polypoid 46; median maximum diameter 20 mm, interquartile range 15-25 mm; histological subtype: papillary 5, well 51, moderate 24, poor 2; location: proximal colon 26, distal colon 27, rectum 29). Application of CAM was limited to one white light endoscopic image (per lesion) to demonstrate findings of T1b cancers. The CAM images were generated from the weights of the previously fine-tuned ResNet50. Two expert endoscopists depicted the ROI in identical images. Concordance of the ROI was rated by intersection over union (IoU) analysis. Results Pixel counts of ROIs were significantly lower using 165K[x103] [108K-227K] than by endoscopists (300K [208K-440K]; P < 0.0001) and median [interquartile] of the IoU was 0.198 [0.024-0.349]. IoU was significantly higher in correctly identified lesions (n = 54, 0.213 [0.116-0.364]) than incorrect ones (n=28, 0.070 [0.000-0.2750, P = 0.033). Concusions IoU was larger in correctly diagnosed T1b colorectal cancers. Optimal annotation of the ROI may be the key to improving diagnostic sensitivity of CAD for T1b colorectal cancers.
Collapse
Affiliation(s)
- Yuki Nakajima
- Department of Gastroenterology, Aizu Medical Center, Fukushima Medical University, Aizuwakamatsu, Japan
| | - Daiki Nemoto
- Department of Coloproctology, Aizu Medical Center, Fukushima Medical University, Aizuwakamatsu, Japan
| | - Zhe Guo
- Biomedical Information Engineering Lab, The University of Aizu, Aizuwakamatsu, Japan
| | - Peng Boyuan
- Biomedical Information Engineering Lab, The University of Aizu, Aizuwakamatsu, Japan
| | - Zhang Ruiyao
- Biomedical Information Engineering Lab, The University of Aizu, Aizuwakamatsu, Japan
| | - Shinichi Katsuki
- Department of Gastroenterology, Otaru Ekisaikai Hospital, Otaru, Japan
| | - Takahito Takezawa
- Department of Medicine, Division of Gastroenterology, Jichi Medical University, Shimotsuke, Japan
| | - Ryo Maemoto
- Department of Surgery, Saitama Medical Center, Jichi Medical University, Saitama, Japan
| | - Keisuke Kawasaki
- Department of Gastroenterology, Iwate Medical University, Morioka, Japan
| | - Ken Inoue
- Department of Molecular Gastroenterology and Hepatology, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Takashi Akutagawa
- Division of Gastroenterology, Department of Internal Medicine, Faculty of Medicine, Saga University, Saga, Japan
| | - Hirohito Tanaka
- Department of Gastroenterology and Hepatology, Gunma University Graduate School of Medicine, Maebashi, Japan
| | - Koichiro Sato
- Department of Clinical Laboratory and Endoscopy, Tokyo Women's Medical University Medical Center East, Tokyo, Japan
| | - Teppei Omori
- Institute of Gastroenterology, Tokyo Women's Medical University, Tokyo, Japan
| | - Yoshikazu Hayashi
- Department of Medicine, Division of Gastroenterology, Jichi Medical University, Shimotsuke, Japan
| | - Yasuyuki Miyakura
- Department of Surgery, Saitama Medical Center, Jichi Medical University, Saitama, Japan
| | - Takayuki Matsumoto
- Department of Gastroenterology, Iwate Medical University, Morioka, Japan
| | - Naohisa Yoshida
- Department of Molecular Gastroenterology and Hepatology, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Motohiro Esaki
- Division of Gastroenterology, Department of Internal Medicine, Faculty of Medicine, Saga University, Saga, Japan
| | - Toshio Uraoka
- Department of Gastroenterology and Hepatology, Gunma University Graduate School of Medicine, Maebashi, Japan
| | - Hiroyuki Kato
- Department of Clinical Laboratory and Endoscopy, Tokyo Women's Medical University Medical Center East, Tokyo, Japan
| | - Yuji Inoue
- Institute of Gastroenterology, Tokyo Women's Medical University, Tokyo, Japan
| | - Hironori Yamamoto
- Department of Medicine, Division of Gastroenterology, Jichi Medical University, Shimotsuke, Japan
| | - Xin Zhu
- Biomedical Information Engineering Lab, The University of Aizu, Aizuwakamatsu, Japan
| | - Kazutomo Togashi
- Department of Coloproctology, Aizu Medical Center, Fukushima Medical University, Aizuwakamatsu, Japan
| |
Collapse
|
|
3
|
Thijssen A, Schreuder RM, Dehghani N, Schor M, de With PH, van der Sommen F, Boonstra JJ, Moons LM, Schoon EJ. Improving the endoscopic recognition of early colorectal carcinoma using artificial intelligence: current evidence and future directions. Endosc Int Open 2024; 12:E1102-E1117. [PMID: 39398448 PMCID: PMC11466514 DOI: 10.1055/a-2403-3103] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/22/2024] [Accepted: 08/21/2024] [Indexed: 10/15/2024] Open
Abstract
Background and study aims Artificial intelligence (AI) has great potential to improve endoscopic recognition of early stage colorectal carcinoma (CRC). This scoping review aimed to summarize current evidence on this topic, provide an overview of the methodologies currently used, and guide future research. Methods A systematic search was performed following the PRISMA-Scr guideline. PubMed (including Medline), Scopus, Embase, IEEE Xplore, and ACM Digital Library were searched up to January 2024. Studies were eligible for inclusion when using AI for distinguishing CRC from colorectal polyps on endoscopic imaging, using histopathology as gold standard, reporting sensitivity, specificity, or accuracy as outcomes. Results Of 5024 screened articles, 26 were included. Computer-aided diagnosis (CADx) system classification categories ranged from two categories, such as lesions suitable or unsuitable for endoscopic resection, to five categories, such as hyperplastic polyp, sessile serrated lesion, adenoma, cancer, and other. The number of images used in testing databases varied from 69 to 84,585. Diagnostic performances were divergent, with sensitivities varying from 55.0% to 99.2%, specificities from 67.5% to 100% and accuracies from 74.4% to 94.4%. Conclusions This review highlights that using AI to improve endoscopic recognition of early stage CRC is an upcoming research field. We introduced a suggestions list of essential subjects to report in research regarding the development of endoscopy CADx systems, aiming to facilitate more complete reporting and better comparability between studies. There is a knowledge gap regarding real-time CADx system performance during multicenter external validation. Future research should focus on development of CADx systems that can differentiate CRC from premalignant lesions, while providing an indication of invasion depth.
Collapse
Affiliation(s)
- Ayla Thijssen
- GROW Research Institute for Oncology and Reproduction, Maastricht University, Maastricht, Netherlands
- Department of Gastroenterology and Hepatology, Maastricht Universitair Medisch Centrum+, Maastricht, Netherlands
| | - Ramon-Michel Schreuder
- GROW Research Institute for Oncology and Reproduction, Maastricht University, Maastricht, Netherlands
- Department of Gastroenterology and Hepatology, Catharina Hospital, Eindhoven, Netherlands
| | - Nikoo Dehghani
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
| | - Marieke Schor
- University Library, Department of Education and Support, Maastricht University, Maastricht, Netherlands
| | - Peter H.N. de With
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
| | - Fons van der Sommen
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
| | - Jurjen J. Boonstra
- Department of Gastroenterology and Hepatology, Leids Universitair Medisch Centrum, Leiden, Netherlands
| | - Leon M.G. Moons
- Department of Gastroenterology and Hepatology, University Medical Center Utrecht, Utrecht, Netherlands
| | - Erik J. Schoon
- GROW Research Institute for Oncology and Reproduction, Maastricht University, Maastricht, Netherlands
- Department of Gastroenterology and Hepatology, Catharina Hospital, Eindhoven, Netherlands
| |
Collapse
|
|
4
|
Tudela Y, Majó M, de la Fuente N, Galdran A, Krenzer A, Puppe F, Yamlahi A, Tran TN, Matuszewski BJ, Fitzgerald K, Bian C, Pan J, Liu S, Fernández-Esparrach G, Histace A, Bernal J. A complete benchmark for polyp detection, segmentation and classification in colonoscopy images. Front Oncol 2024; 14:1417862. [PMID: 39381041 PMCID: PMC11458519 DOI: 10.3389/fonc.2024.1417862] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2024] [Accepted: 07/11/2024] [Indexed: 10/10/2024] Open
Abstract
Introduction Colorectal cancer (CRC) is one of the main causes of deaths worldwide. Early detection and diagnosis of its precursor lesion, the polyp, is key to reduce its mortality and to improve procedure efficiency. During the last two decades, several computational methods have been proposed to assist clinicians in detection, segmentation and classification tasks but the lack of a common public validation framework makes it difficult to determine which of them is ready to be deployed in the exploration room. Methods This study presents a complete validation framework and we compare several methodologies for each of the polyp characterization tasks. Results Results show that the majority of the approaches are able to provide good performance for the detection and segmentation task, but that there is room for improvement regarding polyp classification. Discussion While studied show promising results in the assistance of polyp detection and segmentation tasks, further research should be done in classification task to obtain reliable results to assist the clinicians during the procedure. The presented framework provides a standarized method for evaluating and comparing different approaches, which could facilitate the identification of clinically prepared assisting methods.
Collapse
Affiliation(s)
- Yael Tudela
- Computer Vision Center and Computer Science Department, Universitat Autònoma de Cerdanyola del Valles, Barcelona, Spain
| | - Mireia Majó
- Computer Vision Center and Computer Science Department, Universitat Autònoma de Cerdanyola del Valles, Barcelona, Spain
| | - Neil de la Fuente
- Computer Vision Center and Computer Science Department, Universitat Autònoma de Cerdanyola del Valles, Barcelona, Spain
| | - Adrian Galdran
- Department of Information and Communication Technologies, SymBioSys Research Group, BCNMedTech, Barcelona, Spain
| | - Adrian Krenzer
- Artificial Intelligence and Knowledge Systems, Institute for Computer Science, Julius-Maximilians University of Würzburg, Würzburg, Germany
| | - Frank Puppe
- Artificial Intelligence and Knowledge Systems, Institute for Computer Science, Julius-Maximilians University of Würzburg, Würzburg, Germany
| | - Amine Yamlahi
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Thuy Nuong Tran
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Bogdan J. Matuszewski
- Computer Vision and Machine Learning (CVML) Research Group, University of Central Lancashir (UCLan), Preston, United Kingdom
| | - Kerr Fitzgerald
- Computer Vision and Machine Learning (CVML) Research Group, University of Central Lancashir (UCLan), Preston, United Kingdom
| | - Cheng Bian
- Hebei University of Technology, Baoding, China
| | | | - Shijle Liu
- Hebei University of Technology, Baoding, China
| | | | - Aymeric Histace
- ETIS UMR 8051, École Nationale Supérieure de l'Électronique et de ses Applications (ENSEA), Centre national de la recherche scientifique (CNRS), CY Paris Cergy University, Cergy, France
| | - Jorge Bernal
- Computer Vision Center and Computer Science Department, Universitat Autònoma de Cerdanyola del Valles, Barcelona, Spain
| |
Collapse
|
|
5
|
Mandarino FV, Danese S, Uraoka T, Parra-Blanco A, Maeda Y, Saito Y, Kudo SE, Bourke MJ, Iacucci M. Precision endoscopy in colorectal polyps' characterization and planning of endoscopic therapy. Dig Endosc 2024; 36:761-777. [PMID: 37988279 DOI: 10.1111/den.14727] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Accepted: 11/19/2023] [Indexed: 11/23/2023]
Abstract
Precision endoscopy in the management of colorectal polyps and early colorectal cancer has emerged as the standard of care. It includes optical characterization of polyps and estimation of submucosal invasion depth of large nonpedunculated colorectal polyps to select the appropriate endoscopic resection modality. Over time, several imaging modalities have been implemented in endoscopic practice to improve optical performance. Among these, image-enhanced endoscopy systems and magnification endoscopy represent now well-established tools. New advanced technologies, such as endocytoscopy and confocal laser endomicroscopy, have recently shown promising results in predicting the histology of colorectal polyps. In recent years, artificial intelligence has continued to enhance endoscopic performance in the characterization of colorectal polyps, overcoming the limitations of other imaging modes. In this review we retrace the path of precision endoscopy, analyzing the yield of various endoscopic imaging techniques in personalizing management of colorectal polyps and early colorectal cancer.
Collapse
Affiliation(s)
- Francesco Vito Mandarino
- Department of Gastroenterology and Gastrointestinal Endoscopy, San Raffaele Hospital IRCSS, Milan, Italy
- Department of Gastrointestinal Endoscopy, Westmead Hospital, Sydney, NSW, Australia
| | - Silvio Danese
- Department of Gastroenterology and Gastrointestinal Endoscopy, San Raffaele Hospital IRCSS, Milan, Italy
| | - Toshio Uraoka
- Department of Gastroenterology and Hepatology, Gunma University Graduate School of Medicine, Gumma, Japan
| | - Adolfo Parra-Blanco
- NIHR Nottingham Biomedical Research Centre, Nottingham University Hospitals NHS Trust and the University of Nottingham, Nottingham, UK
| | - Yasuharu Maeda
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Kanagawa, Japan
| | - Yutaka Saito
- Endoscopy Division, National Cancer Center Hospital, Tokyo, Japan
| | - Shin-Ei Kudo
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Kanagawa, Japan
| | - Michael J Bourke
- Department of Gastrointestinal Endoscopy, Westmead Hospital, Sydney, NSW, Australia
| | - Marietta Iacucci
- Department of Gastroenterology, University College Cork, Cork, Ireland
| |
Collapse
|
|
6
|
Li JW, Wang LM, Ichimasa K, Lin KW, Ngu JCY, Ang TL. Use of artificial intelligence in the management of T1 colorectal cancer: a new tool in the arsenal or is deep learning out of its depth? Clin Endosc 2024; 57:24-35. [PMID: 37743068 PMCID: PMC10834280 DOI: 10.5946/ce.2023.036] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Accepted: 05/11/2023] [Indexed: 09/26/2023] Open
Abstract
The field of artificial intelligence is rapidly evolving, and there has been an interest in its use to predict the risk of lymph node metastasis in T1 colorectal cancer. Accurately predicting lymph node invasion may result in fewer patients undergoing unnecessary surgeries; conversely, inadequate assessments will result in suboptimal oncological outcomes. This narrative review aims to summarize the current literature on deep learning for predicting the probability of lymph node metastasis in T1 colorectal cancer, highlighting areas of potential application and barriers that may limit its generalizability and clinical utility.
Collapse
Affiliation(s)
- James Weiquan Li
- Department of Gastroenterology and Hepatology, Changi General Hospital, Singapore Health Services, Singapore
- Academic Medicine Center, Duke-NUS Medical School, Singapore
| | - Lai Mun Wang
- Department of Laboratory Medicine, Changi General Hospital, Singapore Health Services, Singapore
| | - Katsuro Ichimasa
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan
- Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Kenneth Weicong Lin
- Department of Gastroenterology and Hepatology, Changi General Hospital, Singapore Health Services, Singapore
- Academic Medicine Center, Duke-NUS Medical School, Singapore
| | - James Chi-Yong Ngu
- Department of General Surgery, Changi General Hospital, Singapore Health Services, Singapore
| | - Tiing Leong Ang
- Department of Gastroenterology and Hepatology, Changi General Hospital, Singapore Health Services, Singapore
- Academic Medicine Center, Duke-NUS Medical School, Singapore
| |
Collapse
|
|
7
|
Wang K, Zhuang S, Miao J, Chen Y, Hua J, Zhou GQ, He X, Li S. Adaptive Frequency Learning Network With Anti-Aliasing Complex Convolutions for Colon Diseases Subtypes. IEEE J Biomed Health Inform 2023; 27:4816-4827. [PMID: 37796719 DOI: 10.1109/jbhi.2023.3300288] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/07/2023]
Abstract
The automatic and dependable identification of colonic disease subtypes by colonoscopy is crucial. Once successful, it will facilitate clinically more in-depth disease staging analysis and the formulation of more tailored treatment plans. However, inter-class confusion and brightness imbalance are major obstacles to colon disease subtyping. Notably, the Fourier-based image spectrum, with its distinctive frequency features and brightness insensitivity, offers a potential solution. To effectively leverage its advantages to address the existing challenges, this article proposes a framework capable of thorough learning in the frequency domain based on four core designs: the position consistency module, the high-frequency self-supervised module, the complex number arithmetic model, and the feature anti-aliasing module. The position consistency module enables the generation of spectra that preserve local and positional information while compressing the spectral data range to improve training stability. Through band masking and supervision, the high-frequency autoencoder module guides the network to learn useful frequency features selectively. The proposed complex number arithmetic model allows direct spectral training while avoiding the loss of phase information caused by current general-purpose real-valued operations. The feature anti-aliasing module embeds filters in the model to prevent spectral aliasing caused by down-sampling and improve performance. Experiments are performed on the collected five-class dataset, which contains 4591 colorectal endoscopic images. The outcomes show that our proposed method produces state-of-the-art results with an accuracy rate of 89.82%.
Collapse
|
|
8
|
Bai J, Liu K, Gao L, Zhao X, Zhu S, Han Y, Liu Z. Computer-aided diagnosis in predicting the invasion depth of early colorectal cancer: a systematic review and meta-analysis of diagnostic test accuracy. Surg Endosc 2023; 37:6627-6639. [PMID: 37430125 DOI: 10.1007/s00464-023-10223-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2023] [Accepted: 06/16/2023] [Indexed: 07/12/2023]
Abstract
BACKGROUND Endoscopic resection (ER) is widely applied to treat early colorectal cancer (CRC). Predicting the invasion depth of early CRC is critical in determining treatment strategies. The use of computer-aided diagnosis (CAD) algorithms could theoretically make accurate and objective predictions regarding the suitability of lesions for ER indication based on invasion depth. This study aimed to assess diagnostic test accuracy of CAD algorithms in predicting the invasion depth of early CRC and to compare the performance between the CAD algorithms and endoscopists. METHODS Multiple databases were searched until June 30, 2022 for studies that evaluated the diagnostic performance of CAD algorithms for invasion depth of CRC. Meta-analysis of diagnostic test accuracy using a bivariate mixed-effects model was performed. RESULTS Ten studies consisting of 13 arms (13,918 images from 1472 lesions) were included. Due to significant heterogeneity, studies were stratified into Japan/Korea-based or China-based studies. For the former, the area under the curve (AUC), sensitivity, and specificity of the CAD algorithms were 0.89 (95% CI 0.86-0.91), 62% (95% CI 50-72%), and 96% (95% CI 93-98%), respectively. For the latter, AUC, sensitivity, and specificity were 0.94 (95% CI 0.92-0.96), 88% (95% CI 78-94%), and 88% (95% CI 80-93%), respectively. The performance of the CAD algorithms in Japan/Korea-based studies was not significantly different from that of all endoscopists (0.88 vs. 0.91, P = 0.10) but was inferior to that of expert endoscopists (0.88 vs. 0.92, P = 0.03). The performance of the CAD algorithms in China-based studies was better than that of all endoscopists (0.94 vs. 0.90, P = 0.01). CONCLUSION The CAD algorithms showed comparable accuracy for prediction of invasion depth of early CRC compared to all endoscopists, which was still lower than expert endoscopists in diagnostic accuracy; more improvements should be achieved before it can be extensively applied to clinical practice.
Collapse
Affiliation(s)
- Jiawei Bai
- Xijing Hospital of Digestive Diseases, Air Force Medical University (Fourth Military Medical University), 127 Changle West Road, Xi'an, 710032, Shaanxi, China
- School of Medicine, Yan'an University, Yan'an, China
| | - Kai Liu
- Xijing Hospital of Digestive Diseases, Air Force Medical University (Fourth Military Medical University), 127 Changle West Road, Xi'an, 710032, Shaanxi, China
| | - Li Gao
- Xijing Hospital of Digestive Diseases, Air Force Medical University (Fourth Military Medical University), 127 Changle West Road, Xi'an, 710032, Shaanxi, China
| | - Xin Zhao
- Xijing Hospital of Digestive Diseases, Air Force Medical University (Fourth Military Medical University), 127 Changle West Road, Xi'an, 710032, Shaanxi, China
| | - Shaohua Zhu
- Xijing Hospital of Digestive Diseases, Air Force Medical University (Fourth Military Medical University), 127 Changle West Road, Xi'an, 710032, Shaanxi, China
| | - Ying Han
- Xijing Hospital of Digestive Diseases, Air Force Medical University (Fourth Military Medical University), 127 Changle West Road, Xi'an, 710032, Shaanxi, China.
| | - Zhiguo Liu
- Xijing Hospital of Digestive Diseases, Air Force Medical University (Fourth Military Medical University), 127 Changle West Road, Xi'an, 710032, Shaanxi, China.
| |
Collapse
|
|
9
|
van Bokhorst QNE, Houwen BBSL, Hazewinkel Y, Fockens P, Dekker E. Advances in artificial intelligence and computer science for computer-aided diagnosis of colorectal polyps: current status. Endosc Int Open 2023; 11:E752-E767. [PMID: 37593158 PMCID: PMC10431975 DOI: 10.1055/a-2098-1999] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/04/2023] [Accepted: 05/08/2023] [Indexed: 08/19/2023] Open
Affiliation(s)
- Querijn N E van Bokhorst
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, location Academic Medical Center, Amsterdam, the Netherlands
- Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam, the Netherlands
| | - Britt B S L Houwen
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, location Academic Medical Center, Amsterdam, the Netherlands
- Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam, the Netherlands
| | - Yark Hazewinkel
- Department of Gastroenterology and Hepatology, Tergooi Medical Center, Hilversum, the Netherlands
| | - Paul Fockens
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, location Academic Medical Center, Amsterdam, the Netherlands
- Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam, the Netherlands
| | - Evelien Dekker
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, location Academic Medical Center, Amsterdam, the Netherlands
- Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam, the Netherlands
| |
Collapse
|
|
10
|
Nemoto D, Guo Z, Katsuki S, Takezawa T, Maemoto R, Kawasaki K, Inoue K, Akutagawa T, Tanaka H, Sato K, Omori T, Takanashi K, Hayashi Y, Nakajima Y, Miyakura Y, Matsumoto T, Yoshida N, Esaki M, Uraoka T, Kato H, Inoue Y, Peng B, Zhang R, Hisabe T, Matsuda T, Yamamoto H, Tanaka N, Lefor AK, Zhu X, Togashi K. Computer-aided diagnosis of early-stage colorectal cancer using nonmagnified endoscopic white-light images (with videos). Gastrointest Endosc 2023; 98:90-99.e4. [PMID: 36738793 DOI: 10.1016/j.gie.2023.01.050] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Revised: 01/05/2023] [Accepted: 01/25/2023] [Indexed: 02/06/2023]
Abstract
BACKGROUND AND AIMS Differentiation of colorectal cancers (CRCs) with deep submucosal invasion (T1b) from CRCs with superficial invasion (T1a) or no invasion (Tis) is not straightforward. This study aimed to develop a computer-aided diagnosis (CADx) system to establish the diagnosis of early-stage cancers using nonmagnified endoscopic white-light images alone. METHODS From 5108 images, 1513 lesions (Tis, 1074; T1a, 145; T1b, 294) were collected from 1470 patients at 10 academic hospitals and assigned to training and testing datasets (3:1). The ResNet-50 network was used as the backbone to extract features from images. Oversampling and focal loss were used to compensate class imbalance of the invasive stage. Diagnostic performance was assessed using the testing dataset including 403 CRCs with 1392 images. Two experts and 2 trainees read the identical testing dataset. RESULTS At a 90% cutoff for the per-lesion score, CADx showed the highest specificity of 94.4% (95% confidence interval [CI], 91.3-96.6), with 59.8% (95% CI, 48.3-70.4) sensitivity and 87.3% (95% CI, 83.7-90.4) accuracy. The area under the characteristic curve was 85.1% (95% CI, 79.9-90.4) for CADx, 88.2% (95% CI, 83.7-92.8) for expert 1, 85.9% (95% CI, 80.9-90.9) for expert 2, 77.0% (95% CI, 71.5-82.4) for trainee 1 (vs CADx; P = .0076), and 66.2% (95% CI, 60.6-71.9) for trainee 2 (P < .0001). The function was also confirmed on 9 short videos. CONCLUSIONS A CADx system developed with endoscopic white-light images showed excellent per-lesion specificity and accuracy for T1b lesion diagnosis, equivalent to experts and superior to trainees. (Clinical trial registration number: UMIN000037053.).
Collapse
Affiliation(s)
- Daiki Nemoto
- Department of Coloproctology, Aizu Medical Center Fukushima Medical University, Aizuwakamatsu, Japan
| | - Zhe Guo
- Biomedical Information Engineering Lab, The University of Aizu, Aizuwakamatsu, Japan; Department of Laboratory Medicine, The Second Xiangya Hospital, Central South University, Changsha, China
| | - Shinichi Katsuki
- Department of Gastroenterology, Otaru Ekisaikai Hospital, Otaru, Japan
| | - Takahito Takezawa
- Department of Medicine, Division of Gastroenterology, Jichi Medical University, Shimotsuke, Japan
| | - Ryo Maemoto
- Department of Surgery, Saitama Medical Center, Jichi Medical University, Saitama, Japan
| | - Keisuke Kawasaki
- Department of Gastroenterology, Iwate Medical University, Morioka, Japan
| | - Ken Inoue
- Department of Molecular Gastroenterology and Hepatology, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Takashi Akutagawa
- Division of Gastroenterology, Department of Internal Medicine, Faculty of Medicine, Saga University, Saga, Japan
| | - Hirohito Tanaka
- Department of Gastroenterology and Hepatology, Gunma University Graduate School of Medicine, Maebashi, Japan
| | - Koichiro Sato
- Department of Clinical Laboratory and Endoscopy, Tokyo Women's Medical University Medical Center East, Tokyo, Japan
| | - Teppei Omori
- Institute of Gastroenterology, Tokyo Women's Medical University, Tokyo, Japan
| | | | - Yoshikazu Hayashi
- Department of Medicine, Division of Gastroenterology, Jichi Medical University, Shimotsuke, Japan
| | - Yuki Nakajima
- Department of Coloproctology, Aizu Medical Center Fukushima Medical University, Aizuwakamatsu, Japan
| | - Yasuyuki Miyakura
- Department of Surgery, Saitama Medical Center, Jichi Medical University, Saitama, Japan
| | - Takayuki Matsumoto
- Department of Gastroenterology, Iwate Medical University, Morioka, Japan
| | - Naohisa Yoshida
- Department of Molecular Gastroenterology and Hepatology, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Motohiro Esaki
- Division of Gastroenterology, Department of Internal Medicine, Faculty of Medicine, Saga University, Saga, Japan
| | - Toshio Uraoka
- Department of Gastroenterology and Hepatology, Gunma University Graduate School of Medicine, Maebashi, Japan
| | - Hiroyuki Kato
- Department of Clinical Laboratory and Endoscopy, Tokyo Women's Medical University Medical Center East, Tokyo, Japan
| | - Yuji Inoue
- Institute of Gastroenterology, Tokyo Women's Medical University, Tokyo, Japan
| | - Boyuan Peng
- Biomedical Information Engineering Lab, The University of Aizu, Aizuwakamatsu, Japan
| | - Ruiyao Zhang
- Biomedical Information Engineering Lab, The University of Aizu, Aizuwakamatsu, Japan
| | - Takashi Hisabe
- Department of Gastroenterology, Fukuoka University Chikushi Hospital, Fukuoka, Japan
| | - Tomoki Matsuda
- Department of Gastroenterology, Sendai Kosei Hospital, Sendai, Japan
| | - Hironori Yamamoto
- Department of Medicine, Division of Gastroenterology, Jichi Medical University, Shimotsuke, Japan
| | - Noriko Tanaka
- Health Data Science Research Section, Tokyo Metropolitan Institute of Gerontology, Tokyo, Japan
| | | | - Xin Zhu
- Biomedical Information Engineering Lab, The University of Aizu, Aizuwakamatsu, Japan
| | - Kazutomo Togashi
- Department of Coloproctology, Aizu Medical Center Fukushima Medical University, Aizuwakamatsu, Japan
| |
Collapse
|
|
11
|
Shimizu T, Sasaki Y, Ito K, Matsuzaka M, Sakuraba H, Fukuda S. A trial deep learning-based model for four-class histologic classification of colonic tumor from narrow band imaging. Sci Rep 2023; 13:7510. [PMID: 37161081 PMCID: PMC10169849 DOI: 10.1038/s41598-023-34750-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Accepted: 05/06/2023] [Indexed: 05/11/2023] Open
Abstract
Narrow band imaging (NBI) has been extensively utilized as a diagnostic tool for colorectal neoplastic lesions. This study aimed to develop a trial deep learning (DL) based four-class classification model for low-grade dysplasia (LGD); high-grade dysplasia or mucosal carcinoma (HGD); superficially invasive submucosal carcinoma (SMs) and deeply invasive submucosal carcinomas (SMd) and evaluate its potential as a diagnostic tool. We collected a total of 1,390 NBI images as the dataset, including 53 LGD, 120 HGD, 20 SMs and 17 SMd. A total of 598,801 patches were trimmed from the lesion and background. A patch-based classification model was built by employing a residual convolutional neural network (CNN) and validated by three-fold cross-validation. The patch-based validation accuracy was 0.876, 0.957, 0.907 and 0.929 in LGD, HGD, SMs and SMd, respectively. The image-level classification algorithm was derived from the patch-based mapping across the entire image domain, attaining accuracies of 0.983, 0.990, 0.964, and 0.992 in LGD, HGD, SMs, and SMd, respectively. Our CNN-based model demonstrated high performance for categorizing the histological grade of dysplasia as well as the depth of invasion in routine colonoscopy, suggesting a potential diagnostic tool with minimal human inputs.
Collapse
Affiliation(s)
- Takeshi Shimizu
- Department of Gastroenterology, Sendai City Medical Center Sendai Open Hospital, 5-22-1 Tsurugaya, Miyagino-ku, Sendai, 983-0824, Japan
| | - Yoshihiro Sasaki
- Department of Medical Informatics, Hirosaki University Hospital, 53 Hon-cho, Hirosaki, 036-8563, Japan.
| | - Kei Ito
- Department of Gastroenterology, Sendai City Medical Center Sendai Open Hospital, 5-22-1 Tsurugaya, Miyagino-ku, Sendai, 983-0824, Japan
| | - Masashi Matsuzaka
- Department of Medical Informatics, Hirosaki University Hospital, 53 Hon-cho, Hirosaki, 036-8563, Japan
| | - Hirotake Sakuraba
- Department of Gastroenterology and Hematology, Hirosaki University Graduate School of Medicine, 5 Zaifu-cho, Hirosaki, 036-8562, Japan
| | - Shinsaku Fukuda
- Department of Community Medical Support, Hirosaki University Graduate School of Medicine, 5 Zaifu-cho, Hirosaki, 036-8562, Japan
| |
Collapse
|
|
12
|
Krenzer A, Heil S, Fitting D, Matti S, Zoller WG, Hann A, Puppe F. Automated classification of polyps using deep learning architectures and few-shot learning. BMC Med Imaging 2023; 23:59. [PMID: 37081495 PMCID: PMC10120204 DOI: 10.1186/s12880-023-01007-4] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2022] [Accepted: 03/24/2023] [Indexed: 04/22/2023] Open
Abstract
BACKGROUND Colorectal cancer is a leading cause of cancer-related deaths worldwide. The best method to prevent CRC is a colonoscopy. However, not all colon polyps have the risk of becoming cancerous. Therefore, polyps are classified using different classification systems. After the classification, further treatment and procedures are based on the classification of the polyp. Nevertheless, classification is not easy. Therefore, we suggest two novel automated classifications system assisting gastroenterologists in classifying polyps based on the NICE and Paris classification. METHODS We build two classification systems. One is classifying polyps based on their shape (Paris). The other classifies polyps based on their texture and surface patterns (NICE). A two-step process for the Paris classification is introduced: First, detecting and cropping the polyp on the image, and secondly, classifying the polyp based on the cropped area with a transformer network. For the NICE classification, we design a few-shot learning algorithm based on the Deep Metric Learning approach. The algorithm creates an embedding space for polyps, which allows classification from a few examples to account for the data scarcity of NICE annotated images in our database. RESULTS For the Paris classification, we achieve an accuracy of 89.35 %, surpassing all papers in the literature and establishing a new state-of-the-art and baseline accuracy for other publications on a public data set. For the NICE classification, we achieve a competitive accuracy of 81.13 % and demonstrate thereby the viability of the few-shot learning paradigm in polyp classification in data-scarce environments. Additionally, we show different ablations of the algorithms. Finally, we further elaborate on the explainability of the system by showing heat maps of the neural network explaining neural activations. CONCLUSION Overall we introduce two polyp classification systems to assist gastroenterologists. We achieve state-of-the-art performance in the Paris classification and demonstrate the viability of the few-shot learning paradigm in the NICE classification, addressing the prevalent data scarcity issues faced in medical machine learning.
Collapse
Affiliation(s)
- Adrian Krenzer
- Department of Artificial Intelligence and Knowledge Systems, Julius-Maximilians University of Würzburg, Sanderring 2, 97070, Würzburg, Germany.
- Interventional and Experimental Endoscopy (InExEn), Department of Internal Medicine II, University Hospital Würzburg, Oberdürrbacher Straße 6, 97080, Würzburg, Germany.
| | - Stefan Heil
- Department of Artificial Intelligence and Knowledge Systems, Julius-Maximilians University of Würzburg, Sanderring 2, 97070, Würzburg, Germany
| | - Daniel Fitting
- Interventional and Experimental Endoscopy (InExEn), Department of Internal Medicine II, University Hospital Würzburg, Oberdürrbacher Straße 6, 97080, Würzburg, Germany
| | - Safa Matti
- Department of Artificial Intelligence and Knowledge Systems, Julius-Maximilians University of Würzburg, Sanderring 2, 97070, Würzburg, Germany
| | - Wolfram G Zoller
- Department of Internal Medicine and Gastroenterology, Katharinenhospital, Kriegsbergstrasse 60, 70174, Stuttgart, Germany
| | - Alexander Hann
- Interventional and Experimental Endoscopy (InExEn), Department of Internal Medicine II, University Hospital Würzburg, Oberdürrbacher Straße 6, 97080, Würzburg, Germany
| | - Frank Puppe
- Department of Artificial Intelligence and Knowledge Systems, Julius-Maximilians University of Würzburg, Sanderring 2, 97070, Würzburg, Germany
| |
Collapse
|
|
13
|
Gimeno-García AZ, Hernández-Pérez A, Nicolás-Pérez D, Hernández-Guerra M. Artificial Intelligence Applied to Colonoscopy: Is It Time to Take a Step Forward? Cancers (Basel) 2023; 15:cancers15082193. [PMID: 37190122 DOI: 10.3390/cancers15082193] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Revised: 04/04/2023] [Accepted: 04/05/2023] [Indexed: 05/17/2023] Open
Abstract
Growing evidence indicates that artificial intelligence (AI) applied to medicine is here to stay. In gastroenterology, AI computer vision applications have been stated as a research priority. The two main AI system categories are computer-aided polyp detection (CADe) and computer-assisted diagnosis (CADx). However, other fields of expansion are those related to colonoscopy quality, such as methods to objectively assess colon cleansing during the colonoscopy, as well as devices to automatically predict and improve bowel cleansing before the examination, predict deep submucosal invasion, obtain a reliable measurement of colorectal polyps and accurately locate colorectal lesions in the colon. Although growing evidence indicates that AI systems could improve some of these quality metrics, there are concerns regarding cost-effectiveness, and large and multicentric randomized studies with strong outcomes, such as post-colonoscopy colorectal cancer incidence and mortality, are lacking. The integration of all these tasks into one quality-improvement device could facilitate the incorporation of AI systems in clinical practice. In this manuscript, the current status of the role of AI in colonoscopy is reviewed, as well as its current applications, drawbacks and areas for improvement.
Collapse
Affiliation(s)
- Antonio Z Gimeno-García
- Gastroenterology Department, Hospital Universitario de Canarias, 38200 San Cristóbal de La Laguna, Tenerife, Spain
- Instituto Universitario de Tecnologías Biomédicas (ITB) & Centro de Investigación Biomédica de Canarias (CIBICAN), Internal Medicine Department, Universidad de La Laguna, 38200 San Cristóbal de La Laguna, Tenerife, Spain
| | - Anjara Hernández-Pérez
- Gastroenterology Department, Hospital Universitario de Canarias, 38200 San Cristóbal de La Laguna, Tenerife, Spain
- Instituto Universitario de Tecnologías Biomédicas (ITB) & Centro de Investigación Biomédica de Canarias (CIBICAN), Internal Medicine Department, Universidad de La Laguna, 38200 San Cristóbal de La Laguna, Tenerife, Spain
| | - David Nicolás-Pérez
- Gastroenterology Department, Hospital Universitario de Canarias, 38200 San Cristóbal de La Laguna, Tenerife, Spain
- Instituto Universitario de Tecnologías Biomédicas (ITB) & Centro de Investigación Biomédica de Canarias (CIBICAN), Internal Medicine Department, Universidad de La Laguna, 38200 San Cristóbal de La Laguna, Tenerife, Spain
| | - Manuel Hernández-Guerra
- Gastroenterology Department, Hospital Universitario de Canarias, 38200 San Cristóbal de La Laguna, Tenerife, Spain
- Instituto Universitario de Tecnologías Biomédicas (ITB) & Centro de Investigación Biomédica de Canarias (CIBICAN), Internal Medicine Department, Universidad de La Laguna, 38200 San Cristóbal de La Laguna, Tenerife, Spain
| |
Collapse
|
|
14
|
Houwen BBSL, Nass KJ, Vleugels JLA, Fockens P, Hazewinkel Y, Dekker E. Comprehensive review of publicly available colonoscopic imaging databases for artificial intelligence research: availability, accessibility, and usability. Gastrointest Endosc 2023; 97:184-199.e16. [PMID: 36084720 DOI: 10.1016/j.gie.2022.08.043] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Revised: 08/24/2022] [Accepted: 08/30/2022] [Indexed: 01/28/2023]
Abstract
BACKGROUND AND AIMS Publicly available databases containing colonoscopic imaging data are valuable resources for artificial intelligence (AI) research. Currently, little is known regarding the available number and content of these databases. This review aimed to describe the availability, accessibility, and usability of publicly available colonoscopic imaging databases, focusing on polyp detection, polyp characterization, and quality of colonoscopy. METHODS A systematic literature search was performed in MEDLINE and Embase to identify AI studies describing publicly available colonoscopic imaging databases published after 2010. Second, a targeted search using Google's Dataset Search, Google Search, GitHub, and Figshare was done to identify databases directly. Databases were included if they contained data about polyp detection, polyp characterization, or quality of colonoscopy. To assess accessibility of databases, the following categories were defined: open access, open access with barriers, and regulated access. To assess the potential usability of the included databases, essential details of each database were extracted using a checklist derived from the Checklist for Artificial Intelligence in Medical Imaging. RESULTS We identified 22 databases with open access, 3 databases with open access with barriers, and 15 databases with regulated access. The 22 open access databases contained 19,463 images and 952 videos. Nineteen of these databases focused on polyp detection, localization, and/or segmentation; 6 on polyp characterization, and 3 on quality of colonoscopy. Only half of these databases have been used by other researcher to develop, train, or benchmark their AI system. Although technical details were in general well reported, important details such as polyp and patient demographics and the annotation process were under-reported in almost all databases. CONCLUSIONS This review provides greater insight on public availability of colonoscopic imaging databases for AI research. Incomplete reporting of important details limits the ability of researchers to assess the usability of current databases.
Collapse
Affiliation(s)
- Britt B S L Houwen
- Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam University Medical Centres, location Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands
| | - Karlijn J Nass
- Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam University Medical Centres, location Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands
| | - Jasper L A Vleugels
- Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam University Medical Centres, location Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands
| | - Paul Fockens
- Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam University Medical Centres, location Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands
| | - Yark Hazewinkel
- Department of Gastroenterology and Hepatology, Radboud University Nijmegen Medical Center, Radboud University of Nijmegen, Nijmegen, the Netherlands
| | - Evelien Dekker
- Department of Gastroenterology and Hepatology, Amsterdam Gastroenterology Endocrinology Metabolism, Amsterdam University Medical Centres, location Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands
| |
Collapse
|
|
15
|
Dilmaghani S, Coelho-Prabhu N. Role of Artificial Intelligence in Colonoscopy: A Literature Review of the Past, Present, and Future Directions. TECHNIQUES AND INNOVATIONS IN GASTROINTESTINAL ENDOSCOPY 2023; 25:399-412. [DOI: 10.1016/j.tige.2023.03.002] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/04/2025]
|
|
16
|
Young EJ, Rajandran A, Philpott HL, Sathananthan D, Hoile SF, Singh R. Mucosal imaging in colon polyps: New advances and what the future may hold. World J Gastroenterol 2022; 28:6632-6661. [PMID: 36620337 PMCID: PMC9813932 DOI: 10.3748/wjg.v28.i47.6632] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/03/2022] [Revised: 10/23/2022] [Accepted: 11/23/2022] [Indexed: 12/19/2022] Open
Abstract
An expanding range of advanced mucosal imaging technologies have been developed with the goal of improving the detection and characterization of lesions in the gastrointestinal tract. Many technologies have targeted colorectal neoplasia given the potential for intervention prior to the development of invasive cancer in the setting of widespread surveillance programs. Improvement in adenoma detection reduces miss rates and prevents interval cancer development. Advanced imaging technologies aim to enhance detection without significantly increasing procedural time. Accurate polyp characterisation guides resection techniques for larger polyps, as well as providing the platform for the “resect and discard” and “do not resect” strategies for small and diminutive polyps. This review aims to collate and summarise the evidence regarding these technologies to guide colonoscopic practice in both interventional and non-interventional endoscopists.
Collapse
Affiliation(s)
- Edward John Young
- Department of Gastroenterology, Lyell McEwin Hospital, Northern Adelaide Local Health Network, Elizabeth Vale 5031, South Australia, Australia
- Faculty of Health and Medical Sciences, University of Adelaide, Adelaide 5000, South Australia, Australia
| | - Arvinf Rajandran
- Department of Gastroenterology, Lyell McEwin Hospital, Northern Adelaide Local Health Network, Elizabeth Vale 5031, South Australia, Australia
| | - Hamish Lachlan Philpott
- Department of Gastroenterology, Lyell McEwin Hospital, Northern Adelaide Local Health Network, Elizabeth Vale 5031, South Australia, Australia
- Faculty of Health and Medical Sciences, University of Adelaide, Adelaide 5000, South Australia, Australia
| | - Dharshan Sathananthan
- Department of Gastroenterology, Lyell McEwin Hospital, Northern Adelaide Local Health Network, Elizabeth Vale 5031, South Australia, Australia
- Faculty of Health and Medical Sciences, University of Adelaide, Adelaide 5000, South Australia, Australia
| | - Sophie Fenella Hoile
- Department of Gastroenterology, Lyell McEwin Hospital, Northern Adelaide Local Health Network, Elizabeth Vale 5031, South Australia, Australia
- Faculty of Health and Medical Sciences, University of Adelaide, Adelaide 5000, South Australia, Australia
| | - Rajvinder Singh
- Department of Gastroenterology, Lyell McEwin Hospital, Northern Adelaide Local Health Network, Elizabeth Vale 5031, South Australia, Australia
- Faculty of Health and Medical Sciences, University of Adelaide, Adelaide 5000, South Australia, Australia
| |
Collapse
|
|
17
|
Yao L, Lu Z, Yang G, Zhou W, Xu Y, Guo M, Huang X, He C, Zhou R, Deng Y, Wu H, Chen B, Gong R, Zhang L, Zhang M, Gong W, Yu H. Development and validation of an artificial intelligence-based system for predicting colorectal cancer invasion depth using multi-modal data. Dig Endosc 2022. [PMID: 36478234 DOI: 10.1111/den.14493] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Accepted: 12/05/2022] [Indexed: 01/20/2023]
Abstract
OBJECTIVES Accurate endoscopic optical prediction of the depth of cancer invasion is critical for guiding an optimal treatment approach of large sessile colorectal polyps but was hindered by insufficient endoscopists expertise and inter-observer variability. We aimed to construct a clinically applicable artificial intelligence (AI) system for the identification of presence of cancer invasion in large sessile colorectal polyps. METHODS A deep learning-based colorectal cancer invasion calculation (CCIC) system was constructed. Multi-modal data including clinical information, white light (WL) and image-enhanced endoscopy (IEE) were included for training. The system was trained using 339 lesions and tested on 198 lesions across three hospitals. Man-machine contest, reader study and video validation were further conducted to evaluate the performance of CCIC. RESULTS The overall accuracy of CCIC system using image and video validation was 90.4% and 89.7%, respectively. In comparison with 14 endoscopists, the accuracy of CCIC was comparable with expert endoscopists but superior to all the participating senior and junior endoscopists in both image and video validation set. With CCIC augmentation, the average accuracy of junior endoscopists improved significantly from 75.4% to 85.3% (P = 0.002). CONCLUSIONS This deep learning-based CCIC system may play an important role in predicting the depth of cancer invasion in colorectal polyps, thus determining treatment strategies for these large sessile colorectal polyps.
Collapse
Affiliation(s)
- Liwen Yao
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Zihua Lu
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Genhua Yang
- Department of Gastroenterology, Shenzhen Hospital of Southern Medical University, Shenzhen, China
| | - Wei Zhou
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Youming Xu
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Mingwen Guo
- Department of Gastroenterology, The First Hospital of Yichang, Yichang, China
| | - Xu Huang
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Chunping He
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Rui Zhou
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Yunchao Deng
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Huiling Wu
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Boru Chen
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Rongrong Gong
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Lihui Zhang
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Mengjiao Zhang
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Wei Gong
- Department of Gastroenterology, Shenzhen Hospital of Southern Medical University, Shenzhen, China
| | - Honggang Yu
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China.,Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.,Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| |
Collapse
|
|
18
|
Komanduri S, Dominitz JA, Rabeneck L, Kahi C, Ladabaum U, Imperiale TF, Byrne MF, Lee JK, Lieberman D, Wang AY, Sultan S, Shaukat A, Pohl H, Muthusamy VR. AGA White Paper: Challenges and Gaps in Innovation for the Performance of Colonoscopy for Screening and Surveillance of Colorectal Cancer. Clin Gastroenterol Hepatol 2022; 20:2198-2209.e3. [PMID: 35688352 DOI: 10.1016/j.cgh.2022.03.051] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/04/2021] [Revised: 02/23/2022] [Accepted: 03/17/2022] [Indexed: 02/07/2023]
Abstract
In 2018, the American Gastroenterological Association's Center for GI Innovation and Technology convened a consensus conference, entitled "Colorectal Cancer Screening and Surveillance: Role of Emerging Technology and Innovation to Improve Outcomes." The conference participants, which included more than 60 experts in colorectal cancer, considered recent improvements in colorectal cancer screening rates and polyp detection, persistent barriers to colonoscopy uptake, and opportunities for performance improvement and innovation. This white paper originates from that conference. It aims to summarize current patient- and physician-centered gaps and challenges in colonoscopy, diagnostic and therapeutic challenges affecting colonoscopy uptake, and the potential use of emerging technologies and quality metrics to improve patient outcomes.
Collapse
Affiliation(s)
- Srinadh Komanduri
- Department of Department of Gastroenterology and Hepatology, Northwestern University, Chicago, Illinois
| | - Jason A Dominitz
- Veterans Affairs Puget Sound Health Care System and the Division of Gastroenterology, Department of Medicine, University of Washington School of Medicine, Seattle, Washington
| | - Linda Rabeneck
- Department of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Charles Kahi
- Indiana University School of Medicine, Richard L. Roudebush Veterans Affairs Medical Center, Indianapolis, Indiana
| | - Uri Ladabaum
- Division of Gastroenterology and Hepatology, Stanford University School of Medicine, Stanford, California
| | - Thomas F Imperiale
- Department of Medicine, Indiana University School of Medicine, the Regenstrief Institute, the Simon Cancer Center, and the Center for Innovation at Roudebush Veterans Affairs Medical Center, Indianapolis, Indiana
| | - Michael F Byrne
- Division of Gastroenterology, Vancouver General Hospital/University of British Columbia, Vancouver, British Columbia, Canada
| | - Jeffrey K Lee
- Collaborative Health Outcomes Research in Digestive Diseases (CHORD) Group, Kaiser Permanente Division of Research, Kaiser Permanente San Francisco, San Francisco, California
| | - David Lieberman
- Division of Gastroenterology and Hepatology, Oregon Health and Science University, Portland, Oregon
| | - Andrew Y Wang
- Division of Gastroenterology and Hepatology, University of Virginia, Charlottesville, Virginia
| | - Shahnaz Sultan
- Division of Gastroenterology, Hepatology and Nutrition, School of Medicine, University of Minnesota, Minneapolis, Minnesota
| | - Aasma Shaukat
- Division of Gastroenterology, Minneapolis Veterans Affairs Health Care System and Department of Medicine, School of Medicine, University of Minnesota, Minneapolis, Minnesota
| | - Heiko Pohl
- Veterans Affairs Medical Center White River Junction, Vermont; Dartmouth Geisel School of Medicine, Hanover, New Hampshire
| | - V Raman Muthusamy
- Vatche and Tamar Manoukian Division of Digestive Diseases, Department of Medicine, University of California Los Angeles, Los Angeles, California.
| |
Collapse
|
|
19
|
Akbulut S, Hargura AS, Garzali IU, Aloun A, Colak C. Clinical presentation, management, screening and surveillance for colorectal cancer during the COVID-19 pandemic. World J Clin Cases 2022; 10:9228-9240. [PMID: 36159422 PMCID: PMC9477669 DOI: 10.12998/wjcc.v10.i26.9228] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/01/2022] [Revised: 05/29/2022] [Accepted: 08/05/2022] [Indexed: 02/05/2023] Open
Abstract
Management of colorectal cancer (CRC) was severely affected by the changes implemented during the pandemic, and this resulted in delayed elective presentation, increased emergency presentation, reduced screening and delayed definitive therapy. This review was conducted to analyze the impact of the coronavirus disease 2019 (COVID-19) pandemic on management of CRC and to identify the changes made in order to adapt to the pandemic. We performed a literature search in PubMed, Medline, Index Medicus, EMBASE, SCOPUS, Reference Citation Analysis (https://www.referencecitationanalysis.com/) and Google Scholar using the following keywords in various combinations: Colorectal cancer, elective surgery, emergency surgery, stage upgrading, screening, surveillance and the COVID-19 pandemic. Only studies published in English were included. To curtail the spread of COVID-19 infection, there were modifications made in the management of CRC. Screening was limited to high risk individuals, and the screening tests of choice during the pandemic were fecal occult blood test, fecal immunochemical test and stool DNA testing. The use of capsule colonoscopy and open access colonoscopy was also encouraged. Blood-based tests like serum methylated septin 9 were also encouraged for screening of CRC during the pandemic. The presentation of CRC was also affected by the pandemic with more patients presenting with emergencies like obstruction and perforation. Stage migration was also observed during the pandemic with more patients presenting with more advanced tumors. The operative therapy of CRC was altered by the pandemic as more emergencies surgeries were done, which may require exteriorization by stoma. This was to reduce the morbidity associated with anastomosis and encourage early discharge from the hospital. There was also an initial reduction in laparoscopic surgical procedures due to the fear of aerosols and COVID-19 infection. As we gradually come out of the pandemic, we should remember the lessons learned and continue to apply them even after the pandemic passes.
Collapse
Affiliation(s)
- Sami Akbulut
- Department of Surgery, Inonu University Faculty of Medicine, Malatya 44280, Turkey
- Biostatistics and Medical Informatics, Inonu University Faculty of Medicine, Malatya 44280, Turkey
| | - Abdirahman Sakulen Hargura
- Department of Surgery, Inonu University Faculty of Medicine, Malatya 44280, Turkey
- Department of Surgery, Kenyatta University Teaching, Referral and Research Hospital, Nairobi 00100, Kenya
| | - Ibrahim Umar Garzali
- Department of Surgery, Inonu University Faculty of Medicine, Malatya 44280, Turkey
- Department of Surgery, Aminu Kano Teaching Hospital, Kano 700101, Nigeria
| | - Ali Aloun
- Department of Surgery, King Hussein Medical Center, Amman 11855, Jordan
| | - Cemil Colak
- Biostatistics and Medical Informatics, Inonu University Faculty of Medicine, Malatya 44280, Turkey
| |
Collapse
|
|
20
|
Lui TKL, Cheung KS, Leung WK. Machine learning models in the prediction of 1-year mortality in patients with advanced hepatocellular cancer on immunotherapy: a proof-of-concept study. Hepatol Int 2022; 16:879-891. [PMID: 35779202 DOI: 10.1007/s12072-022-10370-3] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/02/2021] [Accepted: 05/22/2022] [Indexed: 11/28/2022]
Abstract
INTRODUCTION Immunotherapy is a new promising treatment for patients with advanced hepatocellular carcinoma (HCC), but is costly and potentially associated with considerable side effects. This study aimed to evaluate the role of machine learning (ML) models in predicting the 1-year cancer-related mortality in advanced HCC patients treated with immunotherapy. METHOD 395 HCC patients who had received immunotherapy (including nivolumab, pembrolizumab or ipilimumab) between 2014 and 2019 in Hong Kong were included. The whole data sets were randomly divided into training (n = 316) and internal validation (n = 79) set. The data set, including 47 clinical variables, was used to construct six different ML models in predicting the risk of 1-year mortality. The performances of ML models were measured by the area under receiver operating characteristic curve (AUC) and their performances were compared with C-Reactive protein and Alpha Fetoprotein in ImmunoTherapY score (CRAFITY) and albumin-bilirubin (ALBI) score. The ML models were further validated with an external cohort between 2020 and 2021. RESULTS The 1-year cancer-related mortality was 51.1%. Of the six ML models, the random forest (RF) has the highest AUC of 0.92 (95% CI 0.87-0.98), which was better than logistic regression (0.82, p = 0.01) as well as the CRAFITY (0.68, p < 0.01) and ALBI score (0.84, p = 0.04). RF had the lowest false positive (2.0%) and false negative rate (5.2%), and performed better than CRAFITY score in the external validation cohort (0.91 vs 0.66, p < 0.01). High baseline AFP, bilirubin and alkaline phosphatase were three common risk factors identified by all ML models. CONCLUSION ML models could predict 1-year cancer-related mortality in HCC patients treated with immunotherapy, which may help to select patients who would benefit from this treatment.
Collapse
Affiliation(s)
- Thomas Ka Luen Lui
- Department of Medicine, University of Hong Kong, 4/F, Professorial Block, Queen Mary Hospital, 102 Pokfulam Road, Hong Kong, China
| | - Ka Shing Cheung
- Department of Medicine, University of Hong Kong, 4/F, Professorial Block, Queen Mary Hospital, 102 Pokfulam Road, Hong Kong, China
| | - Wai Keung Leung
- Department of Medicine, University of Hong Kong, 4/F, Professorial Block, Queen Mary Hospital, 102 Pokfulam Road, Hong Kong, China.
| |
Collapse
|
|
21
|
Lu Z, Xu Y, Yao L, Zhou W, Gong W, Yang G, Guo M, Zhang B, Huang X, He C, Zhou R, Deng Y, Yu H. Real-time automated diagnosis of colorectal cancer invasion depth using a deep learning model with multimodal data (with video). Gastrointest Endosc 2022; 95:1186-1194.e3. [PMID: 34919941 DOI: 10.1016/j.gie.2021.11.049] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Accepted: 11/30/2021] [Indexed: 12/24/2022]
Abstract
BACKGROUND AND AIMS The optical diagnosis of colorectal cancer (CRC) invasion depth with white light (WL) and image-enhanced endoscopy (IEE) remains challenging. We aimed to construct and validate a 2-modal deep learning-based system, incorporated with both WL and IEE images (named Endo-CRC) in estimating the invasion depth of CRC. METHODS Samples were retrospectively obtained from 3 hospitals in China. We combined WL and IEE images into image pairs. Altogether, 337,278 image pairs from 268 noninvasive and superficial CRC and 181,934 image pairs from 82 deep CRC were used for training. A total of 296,644 and 4528 image pairs were used for internal and external tests and for comparison with endoscopists. Thirty-five videos were used for evaluating the real-time performance of the Endo-CRC system. Two deep learning models, solely using either WL (model W) or IEE images (model I), were constructed to compare with Endo-CRC. RESULTS The accuracies of Endo-CRC in internal image tests with and without advanced CRC were 91.61% and 93.78%, respectively, and 88.65% in the external test, which did not include advanced CRC. In an endoscopist-machine competition, Endo-CRC achieved an expert comparable accuracy of 88.11% and the highest sensitivity compared with all endoscopists. In a video test, Endo-CRC achieved an accuracy of 100.00%. Compared with model W and model I, Endo-CRC had a higher accuracy (per image pair: 91.61% vs 88.27% compared with model I and 91.61% vs 81.32% compared with model W). CONCLUSIONS The Endo-CRC system has great potential for assisting in CRC invasion depth diagnosis and may be well applied in clinical practice.
Collapse
Affiliation(s)
- Zihua Lu
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China; Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China; Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Youming Xu
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China; Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China; Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Liwen Yao
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China; Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China; Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Wei Zhou
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China; Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China; Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Wei Gong
- Department of Gastroenterology, Shenzhen Hospital of Southern Medical University, Shenzhen, China
| | - Genhua Yang
- Department of Gastroenterology, Shenzhen Hospital of Southern Medical University, Shenzhen, China
| | - Mingwen Guo
- Department of Gastroenterology, The First Hospital of Yichang, Yichang, China
| | - Beiping Zhang
- Department of Gastroenterology, Guangdong Province Traditional Chinese Medical Hospital, Guangzhou, China
| | - Xu Huang
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China; Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China; Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Chunping He
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China; Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China; Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Rui Zhou
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China; Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China; Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Yunchao Deng
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China; Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China; Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| | - Honggang Yu
- Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China; Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China; Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
| |
Collapse
|
|
22
|
Abstract
Artificial intelligence (AI) is rapidly developing in various medical fields, and there is an increase in research performed in the field of gastrointestinal (GI) endoscopy. In particular, the advent of convolutional neural network, which is a class of deep learning method, has the potential to revolutionize the field of GI endoscopy, including esophagogastroduodenoscopy (EGD), capsule endoscopy (CE), and colonoscopy. A total of 149 original articles pertaining to AI (27 articles in esophagus, 30 articles in stomach, 29 articles in CE, and 63 articles in colon) were identified in this review. The main focuses of AI in EGD are cancer detection, identifying the depth of cancer invasion, prediction of pathological diagnosis, and prediction of Helicobacter pylori infection. In the field of CE, automated detection of bleeding sites, ulcers, tumors, and various small bowel diseases is being investigated. AI in colonoscopy has advanced with several patient-based prospective studies being conducted on the automated detection and classification of colon polyps. Furthermore, research on inflammatory bowel disease has also been recently reported. Most studies of AI in the field of GI endoscopy are still in the preclinical stages because of the retrospective design using still images. Video-based prospective studies are needed to advance the field. However, AI will continue to develop and be used in daily clinical practice in the near future. In this review, we have highlighted the published literature along with providing current status and insights into the future of AI in GI endoscopy.
Collapse
Affiliation(s)
- Yutaka Okagawa
- Endoscopy Division, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan.,Department of Gastroenterology, Tonan Hospital, Sapporo, Japan
| | - Seiichiro Abe
- Endoscopy Division, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan.
| | - Masayoshi Yamada
- Endoscopy Division, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
| | - Ichiro Oda
- Endoscopy Division, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
| | - Yutaka Saito
- Endoscopy Division, National Cancer Center Hospital, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
| |
Collapse
|
|
23
|
Kandel P, Wallace MB. Advanced Imaging Techniques and In vivo Histology: Current Status and Future Perspectives (Lower G.I.). GASTROINTESTINAL AND PANCREATICO-BILIARY DISEASES: ADVANCED DIAGNOSTIC AND THERAPEUTIC ENDOSCOPY 2022:291-310. [DOI: 10.1007/978-3-030-56993-8_110] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/03/2025]
|
|
24
|
Okamoto Y, Yoshida S, Izakura S, Katayama D, Michida R, Koide T, Tamaki T, Kamigaichi Y, Tamari H, Shimohara Y, Nishimura T, Inagaki K, Tanaka H, Yamashita K, Sumimoto K, Oka S, Tanaka S. Development of multi-class computer-aided diagnostic systems using the NICE/JNET classifications for colorectal lesions. J Gastroenterol Hepatol 2022; 37:104-110. [PMID: 34478167 DOI: 10.1111/jgh.15682] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/22/2021] [Revised: 07/22/2021] [Accepted: 08/30/2021] [Indexed: 12/15/2022]
Abstract
BACKGROUND AND AIM Diagnostic support using artificial intelligence may contribute to the equalization of endoscopic diagnosis of colorectal lesions. We developed computer-aided diagnosis (CADx) support system for diagnosing colorectal lesions using the NBI International Colorectal Endoscopic (NICE) classification and the Japan NBI Expert Team (JNET) classification. METHODS Using Residual Network as the classifier and NBI images as training images, we developed a CADx based on the NICE classification (CADx-N) and a CADx based on the JNET classification (CADx-J). For validation, 480 non-magnifying and magnifying NBI images were used for the CADx-N and 320 magnifying NBI images were used for the CADx-J. The diagnostic performance of the CADx-N was evaluated using the magnification rate. RESULTS The accuracy of the CADx-N for Types 1, 2, and 3 was 97.5%, 91.2%, and 93.8%, respectively. The diagnostic performance for each magnification level was good (no statistically significant difference). The sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of the CADx-J were 100%, 96.3%, 82.8%, 100%, and 96.9% for Type 1; 80.3%, 93.7%, 94.1%, 79.2%, and 86.3% for Type 2A; 80.4%, 84.7%, 46.8%, 96.3%, and 84.1% for Type 2B; and 62.5%, 99.6%, 96.8%, 93.8%, and 94.1% for Type 3, respectively. CONCLUSIONS The multi-class CADx systems had good diagnostic performance with both the NICE and JNET classifications and may aid in educating non-expert endoscopists and assist in diagnosing colorectal lesions.
Collapse
Affiliation(s)
- Yuki Okamoto
- Department of Gastroenterology and Metabolism, Hiroshima University Hospital, Hiroshima, Japan
| | - Shigeto Yoshida
- Department of Gastroenterology, JR Hiroshima Hospital, Hiroshima, Japan
| | - Seiji Izakura
- Research Institute for Nanodevice and Bio Systems, Hiroshima University, Hiroshima, Japan
| | - Daisuke Katayama
- Research Institute for Nanodevice and Bio Systems, Hiroshima University, Hiroshima, Japan
| | - Ryuichi Michida
- Research Institute for Nanodevice and Bio Systems, Hiroshima University, Hiroshima, Japan
| | - Tetsushi Koide
- Research Institute for Nanodevice and Bio Systems, Hiroshima University, Hiroshima, Japan
| | - Toru Tamaki
- Department of Computer Science, Nagoya Institute of Technology, Nagoya, Japan
| | - Yuki Kamigaichi
- Department of Gastroenterology and Metabolism, Hiroshima University Hospital, Hiroshima, Japan
| | - Hirosato Tamari
- Department of Gastroenterology and Metabolism, Hiroshima University Hospital, Hiroshima, Japan
| | - Yasutsugu Shimohara
- Department of Gastroenterology and Metabolism, Hiroshima University Hospital, Hiroshima, Japan
| | - Tomoyuki Nishimura
- Department of Gastroenterology and Metabolism, Hiroshima University Hospital, Hiroshima, Japan
| | - Katsuaki Inagaki
- Department of Gastroenterology and Metabolism, Hiroshima University Hospital, Hiroshima, Japan
| | - Hidenori Tanaka
- Department of Endoscopy, Hiroshima University Hospital, Hiroshima, Japan
| | - Ken Yamashita
- Department of Endoscopy, Hiroshima University Hospital, Hiroshima, Japan
| | - Kyoku Sumimoto
- Department of Endoscopy, Hiroshima University Hospital, Hiroshima, Japan
| | - Shiro Oka
- Department of Gastroenterology and Metabolism, Hiroshima University Hospital, Hiroshima, Japan
| | - Shinji Tanaka
- Department of Endoscopy, Hiroshima University Hospital, Hiroshima, Japan
| |
Collapse
|
|
25
|
Saraiva S, Rosa I, Fonseca R, Pereira AD. Colorectal malignant polyps: a modern approach. Ann Gastroenterol 2022; 35:17-27. [PMID: 34987284 PMCID: PMC8713339 DOI: 10.20524/aog.2021.0681] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/24/2021] [Accepted: 11/02/2021] [Indexed: 11/22/2022] Open
Abstract
Colorectal malignant polyps (MP) are polyps with invasive cancer into the submucosa harboring a variable risk of lymph node involvement, which can be estimated through evaluation of morphological, endoscopic, and histologic features. The recent advances in imaging endoscopic techniques have led to the possibility of performing an optical diagnosis of T1 colorectal cancer, allowing the selection of the best therapeutic modality to optimize outcomes for the patient. When MP are diagnosed after endoscopic removal, their management can be challenging. Differentiating low- and high-risk histologic features that influence the possibility of residual tumor, the risk of recurrence and the risk of lymph node metastasis, is crucial to further optimize treatment and surveillance plans. While the presence of high-risk features indicates a need for surgery in the majority of cases, location, comorbidities and the patient’s preference should be taken in account when making the final decision. This is a particularly important issue in the management of low rectal MP presenting with high-risk features, where chemoradiotherapy followed by a watch-and-wait strategy has demonstrated promising results. In this review we discuss the important prognostic features of MP and the most modern approaches regarding their management.
Collapse
Affiliation(s)
- Sofia Saraiva
- Gastroenterology Department (Sofia Saraiva, Isadora Rosa, António Dias Pereira)
| | - Isadora Rosa
- Gastroenterology Department (Sofia Saraiva, Isadora Rosa, António Dias Pereira)
| | - Ricardo Fonseca
- Pathology Department (Ricardo Fonseca), Instituto Português de Oncologia de Lisboa Francisco Gentil, Lisboa, Portugal
| | | |
Collapse
|
|
26
|
Viscaino M, Torres Bustos J, Muñoz P, Auat Cheein C, Cheein FA. Artificial intelligence for the early detection of colorectal cancer: A comprehensive review of its advantages and misconceptions. World J Gastroenterol 2021; 27:6399-6414. [PMID: 34720530 PMCID: PMC8517786 DOI: 10.3748/wjg.v27.i38.6399] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/28/2021] [Revised: 04/26/2021] [Accepted: 09/14/2021] [Indexed: 02/06/2023] Open
Abstract
Colorectal cancer (CRC) was the second-ranked worldwide type of cancer during 2020 due to the crude mortality rate of 12.0 per 100000 inhabitants. It can be prevented if glandular tissue (adenomatous polyps) is detected early. Colonoscopy has been strongly recommended as a screening test for both early cancer and adenomatous polyps. However, it has some limitations that include the high polyp miss rate for smaller (< 10 mm) or flat polyps, which are easily missed during visual inspection. Due to the rapid advancement of technology, artificial intelligence (AI) has been a thriving area in different fields, including medicine. Particularly, in gastroenterology AI software has been included in computer-aided systems for diagnosis and to improve the assertiveness of automatic polyp detection and its classification as a preventive method for CRC. This article provides an overview of recent research focusing on AI tools and their applications in the early detection of CRC and adenomatous polyps, as well as an insightful analysis of the main advantages and misconceptions in the field.
Collapse
Affiliation(s)
- Michelle Viscaino
- Department of Electronic Engineering, Universidad Tecnica Federico Santa Maria, Valpaiso 2340000, Chile
| | - Javier Torres Bustos
- Department of Electronic Engineering, Universidad Tecnica Federico Santa Maria, Valpaiso 2340000, Chile
| | - Pablo Muñoz
- Hospital Clinico, University of Chile, Santiago 8380456, Chile
| | - Cecilia Auat Cheein
- Facultad de Medicina, Universidad Nacional de Santiago del Estero, Santiago del Estero 4200, Argentina
| | - Fernando Auat Cheein
- Department of Electronic Engineering, Universidad Técnica Federico Santa María, Valparaiso 2340000, Chile
| |
Collapse
|
|
27
|
Artificial intelligence-enhanced white-light colonoscopy with attention guidance predicts colorectal cancer invasion depth. Gastrointest Endosc 2021; 94:627-638.e1. [PMID: 33852902 DOI: 10.1016/j.gie.2021.03.936] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/27/2020] [Accepted: 03/30/2021] [Indexed: 02/07/2023]
Abstract
BACKGROUND AND AIMS Endoscopic submucosal dissection (ESD) and EMR are applied in treating superficial colorectal neoplasms but are contraindicated by deeply invasive colorectal cancer (CRC). The invasion depth of neoplasms can be examined by an automated artificial intelligence (AI) system to determine the applicability of ESD and EMR. METHODS A deep convolutional neural network with a tumor localization branch to guide invasion depth classification was constructed on the GoogLeNet architecture. The model was trained using 7734 nonmagnified white-light colonoscopy (WLC) images supplemented by image augmentation from 657 lesions labeled with histopathologic analysis of invasion depth. An independent testing dataset consisting of 1634 WLC images from 156 lesions was used to validate the model. RESULTS For predicting noninvasive and superficially invasive neoplasms, the model achieved an overall accuracy of 91.1% (95% confidence interval [CI], 89.6%-92.4%), with 91.2% sensitivity (95% CI, 88.8%-93.3%) and 91.0% specificity (95% CI, 89.0%-92.7%) at an optimal cutoff of .41 and the area under the receiver operating characteristic (AUROC) curve of .970 (95% CI, .962-.978). Inclusion of the advanced CRC data significantly increased the sensitivity in differentiating superficial neoplasms from deeply invasive early CRC to 65.3% (95% CI, 61.9%-68.8%) with an AUROC curve of .729 (95% CI, .699-.759), similar to experienced endoscopists (.691; 95% CI, .624-.758). CONCLUSIONS We have developed an AI-enhanced attention-guided WLC system that differentiates noninvasive or superficially submucosal invasive neoplasms from deeply invasive CRC with high accuracy, sensitivity, and specificity.
Collapse
|
|
28
|
Ebigbo A, Mendel R, Rückert T, Schuster L, Probst A, Manzeneder J, Prinz F, Mende M, Steinbrück I, Faiss S, Rauber D, de Souza LA, Papa JP, Deprez PH, Oyama T, Takahashi A, Seewald S, Sharma P, Byrne MF, Palm C, Messmann H. Endoscopic prediction of submucosal invasion in Barrett's cancer with the use of artificial intelligence: a pilot study. Endoscopy 2021; 53:878-883. [PMID: 33197942 DOI: 10.1055/a-1311-8570] [Citation(s) in RCA: 30] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
BACKGROUND The accurate differentiation between T1a and T1b Barrett's-related cancer has both therapeutic and prognostic implications but is challenging even for experienced physicians. We trained an artificial intelligence (AI) system on the basis of deep artificial neural networks (deep learning) to differentiate between T1a and T1b Barrett's cancer on white-light images. METHODS Endoscopic images from three tertiary care centers in Germany were collected retrospectively. A deep learning system was trained and tested using the principles of cross validation. A total of 230 white-light endoscopic images (108 T1a and 122 T1b) were evaluated using the AI system. For comparison, the images were also classified by experts specialized in endoscopic diagnosis and treatment of Barrett's cancer. RESULTS The sensitivity, specificity, F1 score, and accuracy of the AI system in the differentiation between T1a and T1b cancer lesions was 0.77, 0.64, 0.74, and 0.71, respectively. There was no statistically significant difference between the performance of the AI system and that of experts, who showed sensitivity, specificity, F1, and accuracy of 0.63, 0.78, 0.67, and 0.70, respectively. CONCLUSION This pilot study demonstrates the first multicenter application of an AI-based system in the prediction of submucosal invasion in endoscopic images of Barrett's cancer. AI scored equally to international experts in the field, but more work is necessary to improve the system and apply it to video sequences and real-life settings. Nevertheless, the correct prediction of submucosal invasion in Barrett's cancer remains challenging for both experts and AI.
Collapse
Affiliation(s)
- Alanna Ebigbo
- III Medizinische Klinik, Universitätsklinikum Augsburg, Augsburg Germany
| | - Robert Mendel
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Regensburg Germany.,Regensburg Center of Health Sciences and Technology (RCHST), OTH Regensburg, Regensburg, Germany
| | - Tobias Rückert
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Regensburg Germany
| | - Laurin Schuster
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Regensburg Germany
| | - Andreas Probst
- III Medizinische Klinik, Universitätsklinikum Augsburg, Augsburg Germany
| | | | - Friederike Prinz
- III Medizinische Klinik, Universitätsklinikum Augsburg, Augsburg Germany
| | - Matthias Mende
- Gastroenterology, Sana Klinikum Lichtenberg, Berlin, Germany
| | - Ingo Steinbrück
- Department of Gastroenterology, Hepatology and Interventional Endoscopy, Asklepios Klinik Barmbek, Hamburg, Germany
| | - Siegbert Faiss
- Gastroenterology, Sana Klinikum Lichtenberg, Berlin, Germany
| | - David Rauber
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Regensburg Germany.,Regensburg Center of Biomedical Engineering (RCBE), OTH Regensburg and Regensburg University, Regensburg, Germany
| | - Luis A de Souza
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Regensburg Germany.,Department of Computing, São Paulo State University, São Paulo, Brazil
| | - João P Papa
- Department of Computing, São Paulo State University, São Paulo, Brazil
| | - Pierre H Deprez
- Cliniques Universitaires St-Luc, Université Catholique de Louvain, Brussels, Belgium
| | - Tsuneo Oyama
- Saku Central Hospital Advanced Care Center, Nagano, Japan
| | | | | | - Prateek Sharma
- Department of Gastroenterology and Hepatology, Veterans Affairs Medical Center and University of Kansas School of Medicine, Kansas City, Missouri, United States
| | - Michael F Byrne
- Division of Gastroenterology, Vancouver General Hospital, University of British Columbia, Vancouver, British Columbia, Canada
| | - Christoph Palm
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Regensburg Germany.,Regensburg Center of Health Sciences and Technology (RCHST), OTH Regensburg, Regensburg, Germany.,Regensburg Center of Biomedical Engineering (RCBE), OTH Regensburg and Regensburg University, Regensburg, Germany
| | - Helmut Messmann
- III Medizinische Klinik, Universitätsklinikum Augsburg, Augsburg Germany
| |
Collapse
|
|
29
|
Shah N, Jyala A, Patel H, Makker J. Utility of artificial intelligence in colonoscopy. Artif Intell Gastrointest Endosc 2021; 2:79-88. [DOI: 10.37126/aige.v2.i3.79] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/02/2021] [Revised: 06/20/2021] [Accepted: 06/28/2021] [Indexed: 02/06/2023] Open
Abstract
Colorectal cancer is one of the major causes of death worldwide. Colonoscopy is the most important tool that can identify neoplastic lesion in early stages and resect it in a timely manner which helps in reducing mortality related to colorectal cancer. However, the quality of colonoscopy findings depends on the expertise of the endoscopist and thus the rate of missed adenoma or polyp cannot be controlled. It is desirable to standardize the quality of colonoscopy by reducing the number of missed adenoma/polyps. Introduction of artificial intelligence (AI) in the field of medicine has become popular among physicians nowadays. The application of AI in colonoscopy can help in reducing miss rate and increasing colorectal cancer detection rate as per recent studies. Moreover, AI assistance during colonoscopy has also been utilized in patients with inflammatory bowel disease to improve diagnostic accuracy, assessing disease severity and predicting clinical outcomes. We conducted a literature review on the available evidence on use of AI in colonoscopy. In this review article, we discuss about the principles, application, limitations, and future aspects of AI in colonoscopy.
Collapse
Affiliation(s)
- Niel Shah
- Department of Internal Medicine, BronxCare Hospital Center, Bronx, NY 10457, United States
| | - Abhilasha Jyala
- Department of Internal Medicine, BronxCare Hospital Center, Bronx, NY 10457, United States
| | - Harish Patel
- Department of Internal Medicine, Gastroenterology, BronxCare Hospital Center, Bronx, NY 10457, United States
| | - Jasbir Makker
- Department of Internal Medicine, Gastroenterology, BronxCare Hospital Center, Bronx, NY 10457, United States
| |
Collapse
|
|
30
|
Shah N, Jyala A, Patel H, Makker J. Utility of artificial intelligence in colonoscopy. Artif Intell Gastrointest Endosc 2021. [DOI: 10.37126/aige.v2.i3.78] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/19/2022] Open
|
|
31
|
Komeda Y, Handa H, Matsui R, Hatori S, Yamamoto R, Sakurai T, Takenaka M, Hagiwara S, Nishida N, Kashida H, Watanabe T, Kudo M. Artificial intelligence-based endoscopic diagnosis of colorectal polyps using residual networks. PLoS One 2021; 16:e0253585. [PMID: 34157030 PMCID: PMC8219125 DOI: 10.1371/journal.pone.0253585] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2020] [Accepted: 06/09/2021] [Indexed: 01/03/2023] Open
Abstract
Convolutional neural networks (CNNs) are widely used for artificial intelligence (AI)-based image classification. Residual network (ResNet) is a new technology that facilitates the accuracy of image classification by CNN-based AI. In this study, we developed a novel AI model combined with ResNet to diagnose colorectal polyps. In total, 127,610 images consisting of 62,510 images with adenomatous polyps, 30,443 with non-adenomatous hyperplastic polyps, and 34,657 with healthy colorectal normal mucosa were subjected to deep learning after annotation. Each validation process was performed using 12,761 stored images of colorectal polyps by a 10-fold cross validation. The efficacy of the ResNet system was evaluated by sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and diagnostic accuracy. The sensitivity, specificity, PPV, NPV, and diagnostic accuracy for adenomatous polyps at WLIs were 98.8%, 94.3%, 90.5%, 87.4%, and 92.8%, respectively. Similar results were obtained for adenomatous polyps at narrow-band imagings (NBIs) and chromoendoscopy images (CEIs) (NBIs vs. CEIs: sensitivity, 94.9% vs. 98.2%; specificity, 93.9% vs. 85.8%; PPV, 92.5% vs. 81.7%; NPV, 93.5% vs. 99.9%; and overall accuracy, 91.5% vs. 90.1%). The ResNet model is a powerful tool that can be used for AI-based accurate diagnosis of colorectal polyps.
Collapse
Affiliation(s)
- Yoriaki Komeda
- Department of Gastroenterology and Hepatology, Kindai University Faculty of Medicine, Osaka, Japan
- * E-mail:
| | - Hisashi Handa
- Faculty of Science and Engineering, Kindai University, Osaka, Japan
- Research Institute for Science and Technology, Kindai University, Osaka, Japan
- Cyber Informatics Research Institute, Kindai University, Osaka, Japan
| | - Ryoma Matsui
- Faculty of Science and Engineering, Kindai University, Osaka, Japan
| | - Shohei Hatori
- Faculty of Science and Engineering, Kindai University, Osaka, Japan
| | - Riku Yamamoto
- Faculty of Science and Engineering, Kindai University, Osaka, Japan
| | - Toshiharu Sakurai
- Department of Gastroenterology and Hepatology, Kindai University Faculty of Medicine, Osaka, Japan
| | - Mamoru Takenaka
- Department of Gastroenterology and Hepatology, Kindai University Faculty of Medicine, Osaka, Japan
| | - Satoru Hagiwara
- Department of Gastroenterology and Hepatology, Kindai University Faculty of Medicine, Osaka, Japan
| | - Naoshi Nishida
- Department of Gastroenterology and Hepatology, Kindai University Faculty of Medicine, Osaka, Japan
| | - Hiroshi Kashida
- Department of Gastroenterology and Hepatology, Kindai University Faculty of Medicine, Osaka, Japan
| | - Tomohiro Watanabe
- Department of Gastroenterology and Hepatology, Kindai University Faculty of Medicine, Osaka, Japan
| | - Masatoshi Kudo
- Department of Gastroenterology and Hepatology, Kindai University Faculty of Medicine, Osaka, Japan
| |
Collapse
|
|
32
|
Soletti RC, de Britto MAP, Borges HL, Machado JC. Detection of Mice Colorectal Tumors by Endoluminal Ultrasound Biomicroscopic Images and Quantification of Image Augmented Gray Values Following Injection of VEGFR-2 Targeted Contrast Agent. Acad Radiol 2021; 28:808-816. [PMID: 32067837 DOI: 10.1016/j.acra.2020.01.013] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2019] [Revised: 01/08/2020] [Accepted: 01/09/2020] [Indexed: 02/07/2023]
Abstract
RATIONALE AND OBJECTIVES Ultrasound biomicroscopy (UBM) is a noninvasive imaging technique that can be applied in detecting colonic tumors and, once associated with an ultrasound contrast agent (UCA), can identify the molecular expression of cancer-related biomarkers, such as the vascular endothelial growth factor receptor 2 (VEGFR-2). The present work aimed to detect colonic tumors and quantify augmented gray values of endoluminal UBM (eUBM) images from colonic tumors following the injection of VEGFR-2 targeted UCA (VEGFR2-UCA) into a mouse model of colorectal cancer. MATERIAL AND METHODS A 40 MHz miniprobe catheter inserted through the biopsy channel of a pediatric flexible bronchofiberscope was used to obtain colonoscopic and B-mode eUBM images simultaneously. Seventeen tumor-bearing mice had their colons inspected and six of them were subjected to a VEGFR2-UCA injection to predict VEGFR-2 expression. RESULTS All animals developed distal colon tumors and eUBM was able to detect all of them and also to characterize the tumors, with 71.4% being in situ lesions and 28.6% being tumors invading the mucosa + muscularis mucosae + submucosa layers, as confirmed by histopathology. After VEGFR2-UCA injection, gray values from the eUBM tumoral images increased significantly (p < 0.01). Tumor sites with increased eUBM image gray values corresponded to areas with increased VEGFR-2 expression, as confirmed by immunohistochemistry. CONCLUSION The results confirm eUBM as a powerful noninvasive and real-time tool for detecting colon tumor and its invasiveness and once associated with VEGFR2-UCA may become a tool for the detection of VEGFR-2 expression in colonic tumors.
Collapse
|
|
33
|
Mitsala A, Tsalikidis C, Pitiakoudis M, Simopoulos C, Tsaroucha AK. Artificial Intelligence in Colorectal Cancer Screening, Diagnosis and Treatment. A New Era. ACTA ACUST UNITED AC 2021; 28:1581-1607. [PMID: 33922402 PMCID: PMC8161764 DOI: 10.3390/curroncol28030149] [Citation(s) in RCA: 98] [Impact Index Per Article: 24.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2021] [Revised: 04/09/2021] [Accepted: 04/20/2021] [Indexed: 12/24/2022]
Abstract
The development of artificial intelligence (AI) algorithms has permeated the medical field with great success. The widespread use of AI technology in diagnosing and treating several types of cancer, especially colorectal cancer (CRC), is now attracting substantial attention. CRC, which represents the third most commonly diagnosed malignancy in both men and women, is considered a leading cause of cancer-related deaths globally. Our review herein aims to provide in-depth knowledge and analysis of the AI applications in CRC screening, diagnosis, and treatment based on current literature. We also explore the role of recent advances in AI systems regarding medical diagnosis and therapy, with several promising results. CRC is a highly preventable disease, and AI-assisted techniques in routine screening represent a pivotal step in declining incidence rates of this malignancy. So far, computer-aided detection and characterization systems have been developed to increase the detection rate of adenomas. Furthermore, CRC treatment enters a new era with robotic surgery and novel computer-assisted drug delivery techniques. At the same time, healthcare is rapidly moving toward precision or personalized medicine. Machine learning models have the potential to contribute to individual-based cancer care and transform the future of medicine.
Collapse
Affiliation(s)
- Athanasia Mitsala
- Second Department of Surgery, University General Hospital of Alexandroupolis, Democritus University of Thrace Medical School, Dragana, 68100 Alexandroupolis, Greece; (C.T.); (M.P.); (C.S.)
- Correspondence: ; Tel.: +30-6986423707
| | - Christos Tsalikidis
- Second Department of Surgery, University General Hospital of Alexandroupolis, Democritus University of Thrace Medical School, Dragana, 68100 Alexandroupolis, Greece; (C.T.); (M.P.); (C.S.)
| | - Michail Pitiakoudis
- Second Department of Surgery, University General Hospital of Alexandroupolis, Democritus University of Thrace Medical School, Dragana, 68100 Alexandroupolis, Greece; (C.T.); (M.P.); (C.S.)
| | - Constantinos Simopoulos
- Second Department of Surgery, University General Hospital of Alexandroupolis, Democritus University of Thrace Medical School, Dragana, 68100 Alexandroupolis, Greece; (C.T.); (M.P.); (C.S.)
| | - Alexandra K. Tsaroucha
- Laboratory of Experimental Surgery & Surgical Research, Democritus University of Thrace Medical School, Dragana, 68100 Alexandroupolis, Greece;
| |
Collapse
|
|
34
|
Computer-aided diagnosis system using only white-light endoscopy for the prediction of invasion depth in colorectal cancer. Gastrointest Endosc 2021; 93:647-653. [PMID: 32735946 DOI: 10.1016/j.gie.2020.07.053] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/23/2020] [Accepted: 07/24/2020] [Indexed: 02/06/2023]
Abstract
BACKGROUND AND AIMS Endoscopic treatment is recommended for low-grade dysplasia (LGD), high-grade dysplasia (HGD), and colorectal cancer (CRC) with submucosal (SM) invasion <1000 μm. However, diagnosis of invasion depth requires experience and is often difficult. This study developed and evaluated a novel computer-aided diagnosis (CAD) system to determine whether endoscopic treatment is appropriate for colorectal lesions using only white-light endoscopy (WLE). METHODS We extracted 3442 images from 1035 consecutive colorectal lesions (105 LGDs, 377 HGDs, 107 CRCs with SM <1000 μm, 146 CRCs with SM ≥1000 μm, and 300 advanced CRCs). All images were WLE, nonmagnified, and nonstained. We developed a novel CAD system using 2751 images; the remaining 691 images were evaluated by the CAD system as a test set. The capability of the CAD system to distinguish endoscopically treatable lesions and untreatable lesions was assessed and compared with the results from 2 trainees and 2 experts. RESULTS The CAD system distinguished endoscopically treatable from untreatable lesions with 96.7% sensitivity, 75.0% specificity, and 90.3% accuracy. These values were significantly higher than those from trainees (92.1%, 67.6%, and 84.9%; P < .01, <.01, and <.01, respectively) and were comparable with those from experts (96.5%, 72.5%, and 89.4%, respectively). Trainees assisted by the CAD system demonstrated a diagnostic capability comparable with that of experts. CONCLUSIONS The CAD system had good diagnostic capability for making treatment decisions for colorectal lesions. This system may enable a more convenient and accurate diagnosis using only WLE.
Collapse
|
|
35
|
|
|
36
|
Sumiyama K, Futakuchi T, Kamba S, Matsui H, Tamai N. Artificial intelligence in endoscopy: Present and future perspectives. Dig Endosc 2021; 33:218-230. [PMID: 32935376 DOI: 10.1111/den.13837] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/11/2020] [Accepted: 09/04/2020] [Indexed: 02/08/2023]
Abstract
Artificial intelligence (AI) has been attracting considerable attention as an important scientific topic in the field of medicine. Deep-leaning (DL) technologies have been applied more dominantly than other traditional machine-learning methods. They have demonstrated excellent capability to retract visual features of objectives, even unnoticeable ones for humans, and analyze huge amounts of information within short periods. The amount of research applying DL-based models to real-time computer-aided diagnosis (CAD) systems has been increasing steadily in the GI endoscopy field. An array of published data has already demonstrated the advantages of DL-based CAD models in the detection and characterization of various neoplastic lesions, regardless of the level of the GI tract. Although the diagnostic performances and study designs vary widely, owing to a lack of academic standards to assess the capability of AI for GI endoscopic diagnosis fairly, the superiority of CAD models has been demonstrated for almost all applications studied so far. Most of the challenges associated with AI in the endoscopy field are general problems for AI models used in the real world outside of medical fields. Solutions have been explored seriously and some solutions have been tested in the endoscopy field. Given that AI has become the basic technology to make machines react to the environment, AI would be a major technological paradigm shift, for not only diagnosis but also treatment. In the near future, autonomous endoscopic diagnosis might no longer be just a dream, as we are witnessing with the advent of autonomously driven electric vehicles.
Collapse
Affiliation(s)
- Kazuki Sumiyama
- Department of Endoscopy, The Jikei University School of Medicine, Tokyo, Japan
| | - Toshiki Futakuchi
- Department of Endoscopy, The Jikei University School of Medicine, Tokyo, Japan
| | - Shunsuke Kamba
- Department of Endoscopy, The Jikei University School of Medicine, Tokyo, Japan
| | - Hiroaki Matsui
- Department of Endoscopy, The Jikei University School of Medicine, Tokyo, Japan
| | - Naoto Tamai
- Department of Endoscopy, The Jikei University School of Medicine, Tokyo, Japan
| |
Collapse
|
|
37
|
Golhar M, Bobrow TL, Khoshknab MP, Jit S, Ngamruengphong S, Durr NJ. Improving Colonoscopy Lesion Classification Using Semi-Supervised Deep Learning. IEEE ACCESS : PRACTICAL INNOVATIONS, OPEN SOLUTIONS 2021; 9:631-640. [PMID: 33747680 PMCID: PMC7978231 DOI: 10.1109/access.2020.3047544] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/07/2023]
Abstract
While data-driven approaches excel at many image analysis tasks, the performance of these approaches is often limited by a shortage of annotated data available for training. Recent work in semi-supervised learning has shown that meaningful representations of images can be obtained from training with large quantities of unlabeled data, and that these representations can improve the performance of supervised tasks. Here, we demonstrate that an unsupervised jigsaw learning task, in combination with supervised training, results in up to a 9.8% improvement in correctly classifying lesions in colonoscopy images when compared to a fully-supervised baseline. We additionally benchmark improvements in domain adaptation and out-of-distribution detection, and demonstrate that semi-supervised learning outperforms supervised learning in both cases. In colonoscopy applications, these metrics are important given the skill required for endoscopic assessment of lesions, the wide variety of endoscopy systems in use, and the homogeneity that is typical of labeled datasets.
Collapse
Affiliation(s)
- Mayank Golhar
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, USA
| | - Taylor L Bobrow
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD 21218, USA
| | | | - Simran Jit
- Division of Gastroenterology and Hepatology, Johns Hopkins Hospital, Baltimore, MD 21287, USA
| | - Saowanee Ngamruengphong
- Division of Gastroenterology and Hepatology, Johns Hopkins Hospital, Baltimore, MD 21287, USA
| | - Nicholas J Durr
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, USA
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD 21218, USA
| |
Collapse
|
|
38
|
Misawa M, Kudo SE, Mori Y, Maeda Y, Ogawa Y, Ichimasa K, Kudo T, Wakamura K, Hayashi T, Miyachi H, Baba T, Ishida F, Itoh H, Oda M, Mori K. Current status and future perspective on artificial intelligence for lower endoscopy. Dig Endosc 2021; 33:273-284. [PMID: 32969051 DOI: 10.1111/den.13847] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/11/2020] [Revised: 09/03/2020] [Accepted: 09/16/2020] [Indexed: 12/23/2022]
Abstract
The global incidence and mortality rate of colorectal cancer remains high. Colonoscopy is regarded as the gold standard examination for detecting and eradicating neoplastic lesions. However, there are some uncertainties in colonoscopy practice that are related to limitations in human performance. First, approximately one-fourth of colorectal neoplasms are missed on a single colonoscopy. Second, it is still difficult for non-experts to perform adequately regarding optical biopsy. Third, recording of some quality indicators (e.g. cecal intubation, bowel preparation, and withdrawal speed) which are related to adenoma detection rate, is sometimes incomplete. With recent improvements in machine learning techniques and advances in computer performance, artificial intelligence-assisted computer-aided diagnosis is being increasingly utilized by endoscopists. In particular, the emergence of deep-learning, data-driven machine learning techniques have made the development of computer-aided systems easier than that of conventional machine learning techniques, the former currently being considered the standard artificial intelligence engine of computer-aided diagnosis by colonoscopy. To date, computer-aided detection systems seem to have improved the rate of detection of neoplasms. Additionally, computer-aided characterization systems may have the potential to improve diagnostic accuracy in real-time clinical practice. Furthermore, some artificial intelligence-assisted systems that aim to improve the quality of colonoscopy have been reported. The implementation of computer-aided system clinical practice may provide additional benefits such as helping in educational poorly performing endoscopists and supporting real-time clinical decision-making. In this review, we have focused on computer-aided diagnosis during colonoscopy reported by gastroenterologists and discussed its status, limitations, and future prospects.
Collapse
Affiliation(s)
- Masashi Misawa
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Kanagawa, Japan
| | - Shin-Ei Kudo
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Kanagawa, Japan
| | - Yuichi Mori
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Kanagawa, Japan.,Clinical Effectiveness Research Group, Institute of Heath and Society, University of Oslo, Oslo, Norway
| | - Yasuharu Maeda
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Kanagawa, Japan
| | - Yushi Ogawa
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Kanagawa, Japan
| | - Katsuro Ichimasa
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Kanagawa, Japan
| | - Toyoki Kudo
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Kanagawa, Japan
| | - Kunihiko Wakamura
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Kanagawa, Japan
| | - Takemasa Hayashi
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Kanagawa, Japan
| | - Hideyuki Miyachi
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Kanagawa, Japan
| | - Toshiyuki Baba
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Kanagawa, Japan
| | - Fumio Ishida
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Kanagawa, Japan
| | - Hayato Itoh
- Graduate School of Informatics, Nagoya University, Aichi, Japan
| | - Masahiro Oda
- Graduate School of Informatics, Nagoya University, Aichi, Japan
| | - Kensaku Mori
- Graduate School of Informatics, Nagoya University, Aichi, Japan
| |
Collapse
|
|
39
|
Peshevska-Sekulovska M, Velikova TV, Peruhova M. Artificial intelligence assisted endocytoscopy: A novel eye in endoscopy. Artif Intell Gastrointest Endosc 2020; 1:44-52. [DOI: 10.37126/aige.v1.i3.44] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/01/2020] [Revised: 11/29/2020] [Accepted: 12/06/2020] [Indexed: 02/06/2023] Open
Abstract
Over the past few years, emerging new approaches in endoscopic imaging technologies facilitate a high-quality assessment of lesions found in the gastrointestinal (GI) tract. Endocytoscopy (EC), as a novel tool in endoscopy, aids the more accurate evaluation of superficial mucosal surface. This review article aims to represent the most relevant information related to the latest EC technology and its clinical application in the lower GI tract diagnostic. We discuss EC-computer-aided diagnosis capability to differentiate between non-neoplastic and neoplastic lesion that offers a closer look to in-vivo assessment and diagnosis of cancerous tissue. Nevertheless, artificial-assisted EC diagnostics could also be employed with benefits in patients with inflammatory bowel disease (IBD) by accurately highlighting the presence of mucosal injury. In our review we included those studies comprising data about colonoscopy with narrow banding imaging and computer-aided diagnosis, as well as EC. Last but not least, artificial-assisted EC facilitates in-vivo diagnosis of the lower GI tract and may, in the future, remodel the field of in-vivo endoscopic diagnosis of colorectal lesions, representing another step towards the so-called optical biopsy.
Collapse
Affiliation(s)
| | - Tsvetelina Veselinova Velikova
- Department of Clinical Immunology, University Hospital Lozenetz, Sofia 1407, Bulgaria
- Medical Faculty, Sofia University, St. Kliment Ohridski, Sofia 1407, Bulgaria
| | - Milena Peruhova
- Department of Gastroenterology, University Hospital Lozenetz, Sofia 1407, Bulgaria
- Medical Faculty, Sofia University, St. Kliment Ohridski, Sofia 1407, Bulgaria
| |
Collapse
|
|
40
|
Mashimo H, Gordon SR, Singh SK. Advanced endoscopic imaging for detecting and guiding therapy of early neoplasias of the esophagus. Ann N Y Acad Sci 2020; 1482:61-76. [PMID: 33184872 DOI: 10.1111/nyas.14523] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2020] [Revised: 10/08/2020] [Accepted: 10/13/2020] [Indexed: 12/16/2022]
Abstract
Esophageal cancers, largely adenocarcinoma in Western countries and squamous cell cancer in Asia, present a significant burden of disease and remain one of the most lethal of cancers. Key to improving survival is the development and adoption of new imaging modalities to identify early neoplastic lesions, which may be small, multifocal, subsurface, and difficult to detect by standard endoscopy. Such advanced imaging is particularly relevant with the emergence of ablative techniques that often require multiple endoscopic sessions and may be complicated by bleeding, pain, strictures, and recurrences. Assessing the specific location, depth of involvement, and features correlated with neoplastic progression or incomplete treatment may optimize treatments. While not comprehensive of all endoscopic imaging modalities, we review here some of the recent advances in endoscopic luminal imaging, particularly with surface contrast enhancement using virtual chromoendoscopy, highly magnified subsurface imaging with confocal endomicroscopy, optical coherence tomography, elastic scattering spectroscopy, angle-resolved low-coherence interferometry, and light scattering spectroscopy. While there is no single ideal imaging modality, various multimodal instruments are also being investigated. The future of combining computer-aided assessments, molecular markers, and improved imaging technologies to help localize and ablate early neoplastic lesions shed hope for improved disease outcome.
Collapse
Affiliation(s)
- Hiroshi Mashimo
- VA Boston Healthcare System, Boston, Massachusetts.,Harvard Medical School, Boston, Massachusetts
| | - Stuart R Gordon
- Dartmouth-Hitchcock Medical Center, Dartmouth University, Lebanon, New Hampshire
| | - Satish K Singh
- VA Boston Healthcare System, Boston, Massachusetts.,Boston University School of Medicine, Boston, Massachusetts
| |
Collapse
|
|
41
|
Nakajima Y, Zhu X, Nemoto D, Li Q, Guo Z, Katsuki S, Hayashi Y, Utano K, Aizawa M, Takezawa T, Sagara Y, Shibukawa G, Yamamoto H, Lefor AK, Togashi K. Diagnostic performance of artificial intelligence to identify deeply invasive colorectal cancer on non-magnified plain endoscopic images. Endosc Int Open 2020; 8:E1341-E1348. [PMID: 33015336 PMCID: PMC7508661 DOI: 10.1055/a-1220-6596] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/23/2020] [Accepted: 06/24/2020] [Indexed: 02/07/2023] Open
Abstract
Background and study aims Colorectal cancers (CRC) with deep submucosal invasion (T1b) could be metastatic lesions. However, endoscopic images of T1b CRC resemble those of mucosal CRCs (Tis) or with superficial invasion (T1a). The aim of this study was to develop an automatic computer-aided diagnosis (CAD) system to identify T1b CRC based on plain endoscopic images. Patients and methods In two hospitals, 1839 non-magnified plain endoscopic images from 313 CRCs (Tis 134, T1a 46, T1b 56, beyond T1b 37) with sessile morphology were extracted for training. A CAD system was trained with the data augmented by rotation, saturation, resizing and exposure adjustment. Diagnostic performance was assessed using another dataset including 44 CRCs (Tis 23, T1b 21) from a third hospital. CAD generated a probability level for T1b diagnosis for each image, and > 95 % of probability level was defined as T1b. Lesions with at least one image with a probability level > 0.95 were regarded as T1b. Primary outcome is specificity. Six physicians separately read the same testing dataset. Results Specificity was 87 % (95 % confidence interval: 66-97) for CAD, 100 % (85-100) for Expert 1, 96 % (78-100) for Expert 2, 61 % (39-80) for both gastroenterology trainees, 48 % (27-69) for Novice 1 and 22 % (7-44) for Novice 2. Significant differences were observed between CAD and both novices ( P = 0.013, P = 0.0003). Other diagnostic values of CAD were slightly lower than of the two experts. Conclusions Specificity of CAD was superior to novices and possibly to gastroenterology trainees but slightly inferior to experts.
Collapse
Affiliation(s)
- Yuki Nakajima
- Coloproctology & Gastroenterology, Aizu Medical Center, Fukushima Medical University, Japan
| | - Xin Zhu
- Biomedical Information Engineering Lab, the University of Aizu, Japan
| | - Daiki Nemoto
- Coloproctology & Gastroenterology, Aizu Medical Center, Fukushima Medical University, Japan
| | - Qin Li
- Biomedical Information Engineering Lab, the University of Aizu, Japan
| | - Zhe Guo
- Biomedical Information Engineering Lab, the University of Aizu, Japan
| | | | | | - Kenichi Utano
- Coloproctology & Gastroenterology, Aizu Medical Center, Fukushima Medical University, Japan
| | - Masato Aizawa
- Coloproctology & Gastroenterology, Aizu Medical Center, Fukushima Medical University, Japan
| | | | | | - Goro Shibukawa
- Coloproctology & Gastroenterology, Aizu Medical Center, Fukushima Medical University, Japan
| | | | | | - Kazutomo Togashi
- Coloproctology & Gastroenterology, Aizu Medical Center, Fukushima Medical University, Japan
| |
Collapse
|
|
42
|
Morreale GC, Sinagra E, Vitello A, Shahini E, Shahini E, Maida M. Emerging artificial intelligence applications in gastroenterology: A review of the literature. Artif Intell Gastrointest Endosc 2020; 1:6-18. [DOI: 10.37126/aige.v1.i1.6] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/23/2020] [Revised: 07/07/2020] [Accepted: 07/16/2020] [Indexed: 02/06/2023] Open
Abstract
Artificial intelligence (AI) allows machines to provide disruptive value in several industries and applications. Applications of AI techniques, specifically machine learning and more recently deep learning, are arising in gastroenterology. Computer-aided diagnosis for upper gastrointestinal endoscopy has growing attention for automated and accurate identification of dysplasia in Barrett’s esophagus, as well as for the detection of early gastric cancers (GCs), therefore preventing esophageal and gastric malignancies. Besides, convoluted neural network technology can accurately assess Helicobacter pylori (H. pylori) infection during standard endoscopy without the need for biopsies, thus, reducing gastric cancer risk. AI can potentially be applied during colonoscopy to automatically discover colorectal polyps and differentiate between neoplastic and non-neoplastic ones, with the possible ability to improve adenoma detection rate, which changes broadly among endoscopists performing screening colonoscopies. In addition, AI permits to establish the feasibility of curative endoscopic resection of large colonic lesions based on the pit pattern characteristics. The aim of this review is to analyze current evidence from the literature, supporting recent technologies of AI both in upper and lower gastrointestinal diseases, including Barrett's esophagus, GC, H. pylori infection, colonic polyps and colon cancer.
Collapse
Affiliation(s)
| | - Emanuele Sinagra
- Gastroenterology and Endoscopy Unit, Fondazione Istituto G. Giglio, Cefalù 90015, Italy
| | - Alessandro Vitello
- Gastroenterology and Endoscopy Unit, S. Elia- M. Raimondi Hospital, Caltanissetta 93100, Italy
| | - Endrit Shahini
- Gastroenterology and Endoscopy Unit, Istituto di Candiolo, FPO-IRCCS, Candiolo (Torino) 93100, Italy
| | | | - Marcello Maida
- Gastroenterology and Endoscopy Unit, S. Elia- M. Raimondi Hospital, Caltanissetta 93100, Italy
| |
Collapse
|
|
43
|
Morreale GC, Sinagra E, Vitello A, Shahini E, Shahini E, Maida M. Emerging artificia intelligence applications in gastroenterology: A review of the literature. Artif Intell Gastrointest Endosc 2020. [DOI: 10.37126/wjem.v1.i1.19] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/06/2023] Open
|
|
44
|
Lui TKL, Guo CG, Leung WK. Accuracy of artificial intelligence on histology prediction and detection of colorectal polyps: a systematic review and meta-analysis. Gastrointest Endosc 2020; 92:11-22.e6. [PMID: 32119938 DOI: 10.1016/j.gie.2020.02.033] [Citation(s) in RCA: 63] [Impact Index Per Article: 12.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/15/2020] [Accepted: 02/17/2020] [Indexed: 02/08/2023]
Abstract
BACKGROUND AND AIMS We performed a meta-analysis of all published studies to determine the diagnostic accuracy of artificial intelligence (AI) on histology prediction and detection of colorectal polyps. METHOD We searched Embase, PubMed, Medline, Web of Science, and Cochrane library databases to identify studies using AI for colorectal polyp histology prediction and detection. The quality of included studies was measured by the Quality Assessment of Diagnostic Accuracy Studies tool. We used a bivariate meta-analysis following a random-effects model to summarize the data and plotted hierarchical summary receiver operating characteristic curves. The area under the hierarchical summary receiver operating characteristic curve (AUC) served as an indicator of the diagnostic accuracy and during head-to-head comparisons. RESULTS A total of 7680 images of colorectal polyps from 18 studies were included in the analysis of histology prediction. The accuracy of the AI (AUC) was .96 (95% confidence interval [CI], .95-.98), with a corresponding pooled sensitivity of 92.3% (95% CI, 88.8%-94.9%) and specificity of 89.8% (95% CI, 85.3%-93.0%). The AUC of AI using narrow-band imaging (NBI) was significantly higher than the AUC using non-NBI (.98 vs .84, P < .01). The performance of AI was superior to nonexpert endoscopists (.97 vs .90, P < .01). For characterization of diminutive polyps using a deep learning model with nonmagnifying NBI, the pooled negative predictive value was 95.1% (95% CI, 87.7%-98.1%). For polyp detection, the pooled AUC was .90 (95% CI, .67-1.00) with a sensitivity of 95.0% (95% CI, 91.0%-97.0%) and a specificity of 88.0% (95% CI, 58.0%-99.0%). CONCLUSIONS AI was accurate in histology prediction and detection of colorectal polyps, including diminutive polyps. The performance of AI was better under NBI and was superior to nonexpert endoscopists. Despite the difference in AI models and study designs, AI performances are rather consistent, which could serve as a reference for future AI studies.
Collapse
Affiliation(s)
- Thomas K L Lui
- Department of Medicine, Queen Mary Hospital, University of Hong Kong, Hong Kong
| | - Chuan-Guo Guo
- Department of Medicine, Queen Mary Hospital, University of Hong Kong, Hong Kong
| | - Wai K Leung
- Department of Medicine, Queen Mary Hospital, University of Hong Kong, Hong Kong.
| |
Collapse
|
|
45
|
Mori Y, Kudo SE, Misawa M, Takeda K, Kudo T, Itoh H, Oda M, Mori K. How Far Will Clinical Application of AI Applications Advance for Colorectal Cancer Diagnosis? JOURNAL OF THE ANUS RECTUM AND COLON 2020; 4:47-50. [PMID: 32346642 PMCID: PMC7186008 DOI: 10.23922/jarc.2019-045] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 12/30/2019] [Accepted: 01/22/2020] [Indexed: 12/16/2022]
Abstract
Integrating artificial intelligence (AI) applications into colonoscopy practice is being accelerated as deep learning technologies emerge. In this field, most of the preceding research has focused on polyp detection and characterization, which can mitigate inherent human errors accompanying colonoscopy procedures. On the other hand, more challenging research areas are currently capturing attention: the automated prediction of invasive cancers. Colorectal cancers (CRCs) harbor potential lymph node metastasis when they invade deeply into submucosal layers, which should be resected surgically rather than endoscopically. However, pretreatment discrimination of deeply invasive submucosal CRCs is considered difficult, according to previous prospective studies (e.g., <70% sensitivity), leading to an increased number of unnecessary surgeries for large adenomas or slightly invasive submucosal CRCs. AI is now expected to overcome this challenging hurdle because it is considered to provide better performance in predicting invasive cancer than non-expert endoscopists. In this review, we introduce five relevant publications in this area. Unfortunately, progress in this research area is in a very preliminary phase, compared to that of automated polyp detection and characterization, because of the lack of number of invasive CRCs used for machine learning. However, this issue will be overcome with more target images and cases. The research field of AI for invasive CRCs is just starting but could be a game changer of patient care in the near future, given rapidly growing technologies, and research will gradually increase.
Collapse
Affiliation(s)
- Yuichi Mori
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan
| | - Shin-Ei Kudo
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan
| | - Masashi Misawa
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan
| | - Kenichi Takeda
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan
| | - Toyoki Kudo
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan
| | - Hayato Itoh
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
| | - Masahiro Oda
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
| | - Kensaku Mori
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
| |
Collapse
|
|
46
|
Mori Y, Kudo SE, Misawa M, Takeda K, Kudo T, Itoh H, Oda M, Mori K. Artificial Intelligence for Colorectal Polyp Detection and Characterization. ACTA ACUST UNITED AC 2020. [DOI: 10.1007/s11938-020-00287-x] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
|
|
47
|
Lui TK, Wong KK, Mak LL, To EW, Tsui VW, Deng Z, Guo J, Ni L, Cheung MK, Leung WK. Feedback from artificial intelligence improved the learning of junior endoscopists on histology prediction of gastric lesions. Endosc Int Open 2020; 8:E139-E146. [PMID: 32010746 PMCID: PMC6976335 DOI: 10.1055/a-1036-6114] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/25/2019] [Accepted: 10/09/2019] [Indexed: 12/12/2022] Open
Abstract
Background and study aims Artificial intelligence (AI)-assisted image classification has been shown to have high accuracy on endoscopic diagnosis. We evaluated the potential effects of use of an AI-assisted image classifier on training of junior endoscopists for histological prediction of gastric lesions. Methods An AI image classifier was built on a convolutional neural network with five convolutional layers and three fully connected layers A Resnet backbone was trained by 2,000 non-magnified endoscopic gastric images. The independent validation set consisted of another 1,000 endoscopic images from 100 gastric lesions. The first part of the validation set was reviewed by six junior endoscopists and the prediction of AI was then disclosed to three of them (Group A) while the remaining three (Group B) were not provided this information. All endoscopists reviewed the second part of the validation set independently. Results The overall accuracy of AI was 91.0 % (95 % CI: 89.2-92.7 %) with 97.1 % sensitivity (95 % CI: 95.6-98.7%), 85.9 % specificity (95 % CI: 83.0-88.4 %) and 0.91 area under the ROC (AUROC) (95 % CI: 0.89-0.93). AI was superior to all junior endoscopists in accuracy and AUROC in both validation sets. The performance of Group A endoscopists but not Group B endoscopists improved on the second validation set (accuracy 69.3 % to 74.7 %; P = 0.003). Conclusion The trained AI image classifier can accurately predict presence of neoplastic component of gastric lesions. Feedback from the AI image classifier can also hasten the learning curve of junior endoscopists in predicting histology of gastric lesions.
Collapse
Affiliation(s)
- Thomas K.L. Lui
- Department of Medicine, Queen Mary Hospital, University of Hong Kong, Hong Kong, China
| | - Kenneth K.Y. Wong
- Department of Computer Science, University of Hong Kong, Hong Kong, China
| | - Loey L.Y. Mak
- Department of Medicine, Queen Mary Hospital, University of Hong Kong, Hong Kong, China
| | - Elvis W.P. To
- Department of Medicine, Queen Mary Hospital, University of Hong Kong, Hong Kong, China
| | - Vivien W.M. Tsui
- Department of Medicine, Queen Mary Hospital, University of Hong Kong, Hong Kong, China
| | - Zijie Deng
- Department of Medicine, University of Hong Kong-Shenzhen Hospital, Shenzhen, China
| | - Jiaqi Guo
- Department of Medicine, University of Hong Kong-Shenzhen Hospital, Shenzhen, China
| | - Li Ni
- Department of Medicine, University of Hong Kong-Shenzhen Hospital, Shenzhen, China
| | - Michael K.S. Cheung
- Department of Medicine, Queen Mary Hospital, University of Hong Kong, Hong Kong, China,Department of Medicine, University of Hong Kong-Shenzhen Hospital, Shenzhen, China
| | - Wai K. Leung
- Department of Medicine, Queen Mary Hospital, University of Hong Kong, Hong Kong, China,Corresponding author Wai K. Leung Department of MedicineQueen Mary HospitalUniversity of Hong KongHong KongChina+852 2816 2863
| |
Collapse
|