Artificial intelligence-assisted colonoscopy: A review of current state of practice and research

Table 1 Summary of the randomized controlled trials involving computer-aided detection for colonoscopy

Ref.	Year	Study design	Study aim	CADe system	Image modality	Number of patients in the CADe group	Number of patients in the control group	Number of polyps (CADe vs control group)	Adenoma detection rate (%) (CADe vs control group)	Polyp detection rate (%) (CADe vs control group)	Number of false-positive rate (%) (CADe vs control group)	Withdrawal time (CADe vs control group), min ± SD; minute
Wang et al[36]	2019	Non-blinded prospective randomised controlled study	To investigate whether a high-performance real-time CADe system can increase polyp and adenoma detection rates in the real clinical setting	The real-time automatic polyp detection system (Shanghai Wision AI Co., Ltd.) based on artificial neural network-SegNet architecture	Real-time Video stream	522	536	767 (498 vs 269)	29.1 vs 20.3; P < 0.001; 95%CI = 1.21-2.135	45.0 vs 29.1; P < 0.001; 95%CI = 1.532-2.544	39 vs 0	6.18 ± 1.38 vs 6.07 ± 1.11; P = 0.15
Wang et al[74]	2020	Double-blind Prospective randomised trial	To assess the effectiveness of a CADe system for improving detection of colon adenomas andpolyps; to analyse the characteristics ofpolyps missed by endoscopists	The real-time automatic polyp detection system (Shanghai Wision AI Co., Ltd.) based on artificial neural network-SegNet architecture	Real-time Video stream	484	478	809 (501 vs 308)	34.0 vs 28.0; P = 0.030; OR = 1.36, 95%CI = 1.03–1.79	52.0 vs 37.0; P < 0.0001; OR = 1.86, 95%CI = 1.44–2.41	48 in CADe group (control group not reported)	6.48 ± 1.32 vs 6.37 ± 1.09; P = 0.14
Su et al[75]	2020	Single-blind Prospective randomised trial	To develop an automatic quality control system; to investigate whether the system could increase the detection of polyps and adenomas in real clinical practice	Five deep learning convolutional neural networks (DCNNs) based on AlexNet, ZFNet, and YOLO V2	Real-time Video stream	308	315	273 (177 vs 96)	28.9 vs 16.5; P < 0.001; OR = 2.055, 95%CI = 1.397-3.024	38.3 vs 25.4; P = 0.00; OR = 1.824, 95%CI = 1.296-2.569	62 in CADe system (control group not reported)	7.03 ± 1.01 vs 5.6 ± 1.26; P < 0.001
Gong et al[76]	2020	Single-blind Prospective randomised trial	To evaluate whether the CADe system could improve polyp yield during colonoscopy	ENDOANGEL based on the deep neural networks and perceptual hash algorithms	Real-time video stream	355	349	302 (178 vs 124)	16 vs 8; P = 0.001; OR = 2.30, 95%CI = 1.40-3·77	47 vs 34; P = 0.0016; OR = 1.69, 95%CI = 1.22-2.34	For endoscope being inside = 0.8; For identification of the caecum = 2; for prediction of slipping = 0	6.38 ± 2·48 vs 4.76 ± 254; P < 0.0001
Liu et al[77]	2020	Double-blind Prospective randomised trial	To study the impact of CADe system on the detection rateof polyps and adenomas in colonoscopy	The convolutional threedimensional (3D) neural network	Real-time video stream	508	518	734 (486 vs 248)	39.1 vs 23.9; P < 0.001; OR = 1.637, 95%CI = 1.201‑2.220	43.7 vs 27.8; P < 0.001; OR = 1.57, 95%CI = 1.586‑2.483	36 in CADe system (control group not reported)	6.82 ± 1.78 vs 6.74 ± 1.62; P < 0.001
Luo et al[78]	2021	Non-blinded Prospective randomised trial	To explore whether CADe could improve the polyp detection rate in the actual clinical environment	A CNN algorithm based on a YOLO network architecture	Real-time Video stream	150	150	185 (105 vs 80)	38.7 vs 34.0; P < 0.001	-	52 in CADe system (control group not reported)	6.22 ± 0.55 vs 6.17 ± 0.52; P = 0.102
Repici et al[79]	2020	Singles-blind Prospective randomised trial	To assess the safety and efficacy of a CADe system for the detection of colorectal neoplasia	The CNN (GI-Genius; Medtronic)	Real-time Video stream	341	344	596 (353 vs 243)	54.8 vs 40.4; P < 0.001; RR = 1.30, 95%CI = 1.14-1.45	279/341 (82) 214/344 (62)	-	417 ± 101 seconds for the CADe group vs 435 ± 149 for controls; P = 0.1
Wang1 et al[80]	2020	Singles-blind Prospective randomised trial	To investigate the impact of CADe on adenoma miss and detection rate	The artificial neural network (EndoScreener, Shanghai Wision AI Co,Ltd, Shanghai, Chin)	Real-time Video stream	184 (CADe-routine group)2	185 (Routine-CADe group)3	529 (244 vs 285)	42.39 vs 35.68; P = 0.186; OR = 1.327, 95%CI = 0.872–2.018	63.59 vs 55.14; P = 0.09; OR = 1.421, 95%CI = 0.936–2.157	67 in CADe system (control group not reported)	6.55 (5.34–7.77) vs 6.51 (5.45–7.57); P = 0.7454

¹The total adenoma miss rate by computer-assisted detection system (CADe) [colonoscopy = 13.89%, 95% confidence interval (CI) = 8.24%–19.54%]; by routine colonoscopy = 40.00%, 95%CI=31.23%–48.77%, P < 0.0001. The total polyp miss rate by CADe colonoscopy = 12.98%, 95%CI = 9.08%–16.88%; by routine colonoscopy = 45.90%, 95%CI = 39.65%–52.15%, P < 0.0001). Visible adenoma miss rate: Routine-CADe group = 24.21% vs CADe-routine group = 1.59%, P < 0.001; Visible polyp miss rate: Routine-CADe group = 30.89% vs CADe-routine group = 2.36%; P < 0.001.

²It means that the colonoscopy was performed by the CADe system and then the conventional method.

³It means that the colonoscopy was performed by the conventional method and then the CADe system.

⁴Median (interquartile range).

CADe: Computer-assisted detection system; CNN: Convolutional neural network; DCNN: Deep learning convolutional neural network; SD: Standard deviation; OR: Odds ratio; RR: Relative risk; CI: Confidence interval.

Table 2 Summary of the non-controlled studies involving computer-aided detection for colonoscopy

Ref.	Year	Study design	System	Image modality	Number of patients/colonoscopies used for training/test datasets (total)	Number of colonoscopy/polyp images/videos used for training/test datasets	Diagnostic properties
Park and Sargent[81]	2016	Retrospective	CADe based on DCNN using a conditional random field model	Still images	35 (colonoscopy videos)	562/562 (colonoscopy still images)	Sensitivity = 86%; specificity = 85%; AUC = 0.8585
Fernández-Esparrach et al[73]	2016	Retrospective	CADe based on energy map	Still images	NA/24 colonoscopy videos containing 31 different polyps	NA/Experiment A: 612 polyp images from all 24 videos. Experiment B: 47886 frames from the 24 videos	Experiment A: accuracy = small vs all polyps = 77.5%, 95%CI = 71.5%–82.6% vs 66.2%, 95%CI = 61.4%–70.7%; P < 0.01. Experiment B: The AUC = high quality frames vs all Frames = 0.79, 95%CI = 0.70–0.87 vs 0.75, 95%CI = 0.66–0.83
Yu et al[82]	2017	Retrospective	CADe based on three-dimensional (3-D) deep learning integration framework by leveraging the 3-D fully CNN (3D-FCN)	Videos	20/18 (colonoscopy videos)	3799 frames with polyps in total	Sensitivity = 71%; PPV = 88%; precision = 88.1%
Billah et al[83]	2017	Retrospective	CADe based on CNN and color wavelet features using a linear support vector machine	Still images	100 (colonoscopy videos for combined training and test datasets)	14000 still images (combined for training and test datasets)	Accuracy = 98.65%; sensitivity = 98.79%; specificity = 98.52%
Zhang et al[84]	2017	Retrospective	CADe based on DCNN	Still images	NA	2262/150 random, 30 NBI (colonoscopy still images)	Accuracy = 85.9%; sensitivity = 98%; PPV = 99%; precision = 87.3%; recall rate = 87.6%; AUC = 1.0
Wang et al[85]	2018	Retrospective	CADe based on DNN	Still images	1290/1138 (2428) patients	27113/5545 (colonoscopy images)	Sensitivity = 94.38%, 95%CI = 93.80%-94.96% in images with polyp; AUC = 0.984
Misawa et al[34]	2018	Retrospective	CADe based on CNN	Videos	59/14 (73)	411/135 (colonoscopy videos containing 150 polyps)	Per-polyp sensitivity = 94%; per-frame sensitivity = 90%; specificity = 63.3%; accuracy = 76.5%; false positive rate = 60%; AUC = 0.87
Yamada et al[33]	2019	Retrospective	CADe based on DNN	Videos	NA/77 (number of videos)	13983/4840 (colonoscopy videos)	Sensitivity = 97.3%, 95%CI = 95.9%–98.4%; specificity = 99.0%, 95%CI = 98.6%–99.2%; AUC = 0.975, 95%CI = 0.964–0.986)
Urban et al[35]	2018	Retrospective	CADe based on deep learning CNN	Videos	Several training and validation sets: (1) Cross-validation on the 8641 images; (2) Training on the 8641 images and testing on the 9 videos, 11 videos, and independent dataset; and (3) Training on the 8641 images and 9 videos and testing on the 11 videos and independent dataset		Sensitivity = 96.9%; specificity: 95%; AUC = 0.991; accuracy = 96.4%; false positive rate = 7%
Klare et al[37]	2019	Prospective	Automated polyp detection software (“KoloPol,” Fraunhofer IIS, Erlangen, Germany) based on CNN	Live colonoscopy videos	NA	NA/55 (colonoscopy videos)	Per-polyp sensitivity = 75.3%, 95%CI = 62.3%-84.9%; PDR = 50.9%, 95%CI = 37.1%-64.4%; ADR = 29.1%, 95%CI = 17.6%-42.9%
Ozawa et al[86]	2020	Retrospective	CADe based on DCNN	Still images	12895 patients	16418/7077	Sensitivity = 92%; PPV = 86%; accuracy = 83%; identified adenomas = 97%

CADe: Computer-assisted detection system; CNN: Convolutional neural network; DCNN: Deep learning convolutional neural network; AUC: Area Under the Receiver Operating Characteristic curve; PPV: Positive predictive value; NPV: Negative predictive value; PDR: Polyp detection rate; ADR: Adenoma detection rate; CI: Confidence interval.

Table 3 Summary of the non-controlled studies involving computer-aided diagnosis for colonoscopy including studies with combined detection and diagnosis systems

Ref.	Year	Study design	Study aim	System	Number of patients/colonoscopies used for training/test datasets (total)	Number of colonoscopy/polyp images/videos used in training/test datasets	Diagnostic properties
Tischendorf et al[38]	2010	Prospective pilot	Distinguishing adenomas from non-adenomas	CADx based on SVMs	NA/128; Colonoscopy videos	NA/209 polyps containing 160 neoplastic and 49 non-neoplastic polyps in the test dataset	CADx: Sensitivity = 90%, specificity = 70%, correct classification rate = 85.3%. Consensus decision between the human. Observers: Sensitivity = 93.8%, specificity = 85.7%, correct classification rate = 91.9%. “Safe” decision, when there was interobserver discrepancy: Sensitivity = 96.9%, specificity = 71.4%, correct classification rate = 90.9%
Aihara et al[47]	2013	Prospective	Distinguishing neoplastic from non-neoplastic lesion	CADx based on numerical color analysis of autofluorescence endoscopy as an Adobe AIRapplication	NA/32 patients in the test dataset	NA/102 lesions containing 75 neoplastic lesions in the test dataset	Sensitivity = 94.2%; specificity = 88.8%; PPV = 95.6%; NPV = 85.2%
Mori et al[87]	2015	Retrospective pilot	Distinguishing small (≤ 10 mm) neoplastic from non-neoplastic lesion	CADx (EC-CAD) based on CNN	NA/152 patients in the test dataset	NA/176 small polyps in the test dataset containing 137 neoplastic and 39 non-neoplastic polyps for the test dataset	Accuracy = 89.2%, 95%CI = 83.7%-93.4%; Sensitivity = 92.0%, 95%CI = 86.1%-95.9%; specificity of 79.5%, 95%CI = 63.5%-90.7%
Kuiper et al[49]	2015	Retrospective	Distinguishing small (≤ 9 mm) neoplastic from non-neoplastic lesion	CADx (WavSTAT) based on CNN	NA/87 patients in the test dataset	NA/207 small lesions in the test dataset	Accuracy = 74.4%, 95%CI = 68.1%–79.9%; sensitivity = 85.3%, 95%CI = 0.78–0.90; specificity = 58.8%, 95%CI = 0.48–0.69; PPV = 74.8%, 95%CI = 0.67–0.81; NPV = 73.5%; accuracy of on-site recommended surveillance interval = 73.7%
Misawa et al[34]	2018	Retrospective	Distinguishing neoplastic from non-neoplastic lesion categorized	CADx based on SVMs	NA	979 images containing 381 non-neoplasms and 598 neoplasms in the training dataset/100 images containing 50 non-neoplasms and 50 neoplasms in the test dataset	Accuracy = 90.0%, 95%CI = 82.4–95.1; sensitivity = 84.5%, 95%CI = 72.6–92.7; specificity = 97.6%, 95%CI = 87.4–99.9; PPV = 98.0%, 95%CI = 89.4–99.9; NPV = 82.0%, 95%CI = 68.6–91.4
Byrne et al[51]	2018	Retrospective	Distinguishing neoplastic from non-neoplastic lesions	CADx + CADe based on an improved DCNN model using NBI	NA	NA/21804 unseen frames in the test dataset	Accuracy = 99.94%; sensitivity = 95.95%; specificity = 91.66%; NPV = 93.6%; prediction of polyp videos = 97.6%
Mori et al[48]	2018	Prospective	Distinguishing diminutive (≤ 5 mm) neoplastic from non-neoplastic lesions	CADx based on SVMs used with NBI and endocytoscope	NA/791 patients in the test dataset	61925/466 polyps from 325 patients in the test dataset	CADx-NBI: Sensitivity = 92.7%, 95%CI = 89.1–95.4; specificity = 89.8%, 95%CI = 84.4–93.9; PPV = 93.7%, 95%CI = 90.2–96.2; NPV = 88.3%, 95%CI = 82.7–92.6. CADx-endocytoscope: Sensitivity = 91.3%, 95%CI = 87.5–94.3; specificity = 88.7%, 95%CI = 83.1–93.0; PPV = 92.9%, 95%CI = 89.3–95.6; NPV = 86.3%, 95%CI = 80.4–90.9
Byrne et al[45]	2019	Retrospective	Distinguishing diminutive (≤ 5 mm) neoplastic from non-neoplastic lesions	CADx based on DCNN		Training dataset: 60089 frames from 223 polyp videos (29% NICE type 1, 53% NICE type 2 and 18% of normal mucosa with no polyp)/validation dataset: 40 videos (NICE type 1, NICE type 2 and two videos of normal mucosa)/test dataset: 125 consecutively identified diminutive polyps, comprising 51 hyperplastic polyps and 74 adenomas	Accuracy = 94%, 95%CI = 86%-97%; sensitivity = 98%, 95%CI = 92%-100%; Specificity = 83%, 95%CI = 67%-93%; NPV = 97%; PPV = 90%
Song et al[88]	2020	Retrospective	Distinguishing adenomas from SPs	CADx based on DCNN	NA	12480 image patches of 624 polyps/two test datasets of 545 polyp	Agreement between the true polyp histology CADx = 0.614–0.642; accuracy = 81.3%–82.4%; sensitivity = 82.1%; specificity = 93.7%; PPV = 78%; NPV = 95%; the AUC = 0.93–0.95, 0.86–0.89, and 0.89–0.91 for serrated polyps, benign adenoma/mucosal or superficial submucosal cancer, and deep submucosal cancer, respectively
Kudo et al[89]	2020	Retrospective	Distinguishing small (≤ 10 mm) neoplastic from non-neoplastic lesions	The EndoBRAIN system (CADx + CADe based on DCNN)	NA/89 patients test set	69,142 images taken at 520-fold magnification and 2,000 polyps/100 lesions (≤ 10 mm) in the test dataset	CADe: Accuracy = 98%, 95%CI = 97.3%–98.6%; sensitivity = 96.9%, 95%CI = 95.8%–97.8%; specificity = 100%, 95%CI = 99.6%–100%; PPV = 100%, 95%CI = 99.8%–100%; NPV = 94.6%, 95%CI = 92.7%–96.1%; CADx: Accuracy = 96%, 95%CI = 95.1%–96.8%; sensitivity = 96.9%, 95%CI = 95.8%–97.8%; specificity = 94.3%, 95%CI = 92.3%–95.9%; PPV = 96.9%, 95%CI = 95.8%–97.8%; NPV = 94.3%, 95%CI = 92.3%–95.9%

CADe: Computer-assisted detection system; CADx: Computer-assisted diagnosis system; CNN: Convolutional neural network; DCNN: Deep learning convolutional neural network; AUC: Area Under the Receiver Operating Characteristic curve; PPV: Positive predictive value; NPV: Negative predictive value; SVM: Support vector machine; SP: Serrated polyps; CI: Confidence interval.

Table 4 Commercially available computer-assisted colonoscopy tools that have cleared regulatory approval

Computer assissted system	Product	Manufacturer	Year of regulatory approval	Place of regulatory approval
CADx	EndoBRAIN	Cybernet System Corp./Olympus Corp.	2018	Japan
CADe	GI Genius	Medtronic Corp.	2019 in Europe; 2021 in United States	Europe/United States
CADe	ENDO-AID	Olympus Corp.	2020	Europe
CADe/CADx	CAD EYE	Fujifilm Corp.	2020	Europe/Japan
CADe	DISCOVERY	Pentax Corp.	2020	Europe
CADe	EndoBRAIN-EYE	Cybernet System Corp./Olympus Corp.	2020	Japan
CADe	EndoAngel	Wuhan EndoAngel Medical Technology Company	2020	China
CADe	EndoScreener	WISION A.I.	2020	China
CADx	EndoBRAIN-PLUS	Cybernet System Corp./Olympus Corp.	2020	Japan
CADx	EndoBRAIN-UC	Cybernet System Corp./Olympus Corp.	2020	Japan
CADe	WISE VISION	NEC Corp.	2021	Europe/Japan
CADe	ME-APDS	Magentiq Eye	2021	Europe
CADe	CADDIE	Odin Vision	2021	Europe

CADe: Computer-assisted detection system; CADx: Computer-assisted diagnosis system.