Deep learning driven colorectal lesion detection in gastrointestinal endoscopic and pathological imaging

Table 1 Summary of important studies of computer-aided endoscopic colorectal lesion detection

Ref.	Methods and data	Important results	Limitation and drawback
Karkanis et al[40], 2003	Endoscopic video tumor detection by color wavelet covariance, supported by linear discriminant analysis, 66 patients with 95 polyps	Specificity 90% and sensitivity 97%	It is not enough stable to classify different types of colorectal polyps
Misawa et al[41], 2018	An AI-assisted CADe system using 3D CNNs, 155 polyp-positive videos with 391 polyp-negative	Sensitivity 90.0%, specificity 63.3%, and accuracy 76.5%	Further machine and deep learning and prospective evaluations are mandatory
Urban et al[42], 2018	CNNs; 8641 hand-labeled images with 4088 unique polyps	AUC of 0.991 and accuracy of 96.4%	Unknown effects of CNNs on inspection behavior by colonoscopists, anonymous and unidentified natural or endoscopic videos
Mori et al[43], 2018	Retrospective analysis: An AI system by machine learning, 144 diminutive polyps (≤ 5 mm)	Sensitivity 98%, specificity 71%, accuracy 81%, positive 67%, and negative 98%	Insufficient endoscopic video image data
Yamada et al[44], 2020	Retrospective analysis: A deep learning driven system using a Single Shot Multibox Detector for capsule endoscopic colon lesions detection, 15933 training images and 4784 testing images	AUC 0.902, sensitivity 79.0%, specificity 87.0%, accuracy 83.9%, and at a probability cutoff of 0.348	It was a retrospective study that only used the selected images, while it also did not consider pathological diagnoses and the clinical utility of the AI model has not been evaluated

AI: Artificial intelligence; CNN: Convolutional neural network; AUC: Area under the curve.

Table 2 Important studies of computer-aided pathological prediction of colorectal lesions

Ref.	Methods and data	Important results	Limitation and drawback
Tamaki et al[47], 2013	A new combination of local features and sampling tested on 908 NBI images	A recognition rate of 96% for 10-fold cross validation and a rate of 93% for separate data	Without investigation on robustness, motion blur, focus, window size, color bleeding, and highlight areas
Ito et al[48], 2018	Use AlexNet to diagnose cT1b, 190 colorectal lesion images from 41 patient cases	Sensitivity 67.5%, specificity 89.0%, accuracy 81.2%, and AUC 0.871	Insufficient pathological images to build CNNs
Zachariah et al[50], 2020	A CNNs model using TensorFlow and ImageNet, 6223 images with 80% train and 20% test, processing at 77 frames per second	Negative 97% among diminutive rectum/ rectosigmoid polyps, surveillance interval 93%. In fresh validation, NPV 97% and surveillance interval 94%	Retrospective study using offline images
Shahidi et al[51], 2020	An established real-time AI clinical decision support solution to resolve endoscopic and pathologic discrepancies, 644 images with colorectal lesions ≤ 3 mm	CDSS was consistent with the endoscopic diagnosis in 577 (89.6%) lesions	Inevitable CDSS optimization, given the increasingly used deep learning for development of current AI platforms, manifesting in AI’s ability to adapt with increasing data exposure

AI: Artificial intelligence; CNN: Convolutional neural network; AUC: Area under the curve; CDSS: Clinical decision support system.