Artificial intelligence and early esophageal cancer

Abstract

The development of esophageal cancer (EC) from early to advanced stage results in a high mortality rate and poor prognosis. Advanced EC not only poses a serious threat to the life and health of patients but also places a heavy economic burden on their families and society. Endoscopy is of great value for the diagnosis of EC, especially in the screening of Barrett’s esophagus and early EC. However, at present, endoscopy has a low diagnostic rate for early tumors. In recent years, artificial intelligence (AI) has made remarkable progress in the diagnosis of digestive system tumors, providing a new model for clinicians to diagnose and treat these tumors. In this review, we aim to provide a comprehensive overview of how AI can help doctors diagnose early EC and precancerous lesions and make clinical decisions based on the predicted results. We analyze and summarize the recent research on AI and early EC. We find that based on deep learning (DL) and convolutional neural network methods, the current computer-aided diagnosis system has gradually developed from in vitro image analysis to real-time detection and diagnosis. Based on powerful computing and DL capabilities, the diagnostic accuracy of AI is close to or better than that of endoscopy specialists. We also analyze the shortcomings in the current AI research and corresponding improvement strategies. We believe that the application of AI-assisted endoscopy in the diagnosis of early EC and precancerous lesions will become possible after the further advancement of AI-related research.

Key Words: Artificial intelligence, Computer-aided diagnosis, Deep learning, Convolutional neural network, Barrett’s esophagus, Early esophageal cancer

Core Tip: The early diagnosis and early treatment of esophageal cancer (EC) have always been a hot spot in clinical medicine research and are of great importance to the prognosis of patients. With continuous improvements in computer technology and the arrival of the era of big data, the artificial intelligence (AI)-assisted endoscopic diagnosis of EC has also flourished. This review mainly introduces the research progress of AI-assisted endoscopy in the diagnosis of Barrett’s esophagus and early EC.

INTRODUCTION

Barrett’s esophagus (BE) is a premalignant condition characterized by the replacement of columnar epithelium with esophageal squamous epithelium. Esophageal cancer (EC) is the seventh most common cancer and the sixth leading cause of cancer-related mortality worldwide[1]. EC mainly consists of two histological types: Esophageal squamous cell carcinoma (ESCC) and esophageal adenocarcinoma (EAC). ESCC is the main pathological type in Asian countries, and the 5-year survival rate is less than 20%[2]. EAC is more common in Western countries, and its incidence has been on the rise globally in recent years[3]. The development of EC from early to advanced stage is accompanied by a high mortality rate and poor prognosis. Early detection and diagnosis greatly impact the prognosis of EC. The need for more efficient detection methods for early EC has led to in-depth research in the field of artificial intelligence (AI). The purpose of this review is to summarize the diagnostic value of AI for BE and early EC, which is conducive to the early treatment of patients and the reduction in mortality. In this review, we will discuss the following: (1) The utility of AI techniques in the endoscopic detection of BE; (2) the utility of AI techniques in the endoscopic detection of early EC; and (3) problems and prospects of AI-assisted endoscopic diagnosis.

ARTIFICIAL INTELLIGENCE

AI refers to the abilities of computers to imitate the cognitive function of the human mind and conduct autonomous learning. In recent years, AI has made great progress in various fields of medicine, such as radiological oncology, diabetic retinopathy, and skin cancer[4-6]. Machine learning (ML) for AI can be roughly divided into traditional learning and deep learning (DL). Traditional learning methods require artificial design, which is time-consuming and laborious. DL methods can independently extract and learn image features and extract more complex and abstract advanced features layer by layer through a multilayer system, which allows them to be truly mature and be applied in clinical practice[7]. Convolutional neural networks (CNNs) are a kind of DL method commonly used in AI-assisted image recognition. These networks contain multilayer perceptrons and imitate human brain neural circuits to carry out high generalization, abstraction, and synthesis to process information. The DL method is an end-to-end learning method without the need to design specific image features[8,9]. With the rapid development of information technology, DL has received increasing attention in the medical field. The computer-aided diagnosis (CAD) of gastrointestinal (GI) diseases has become a hot research topic. CAD is an advanced technology used to preprocess endoscopic images, extract image features, process data, and obtain diagnostic results with the help of computer algorithms and graphics processing technology[10] (Figure 1).

Figure 1 Diagram representation of artificial intelligence domains. ML: Machine learning; DL: Deep learning; CAD: Computer-aided diagnosis; CNN: Convolutional neural network.

APPLICATION OF ARTIFICIAL INTELLIGENCE IN BARRETT’S ESOPHAGUS

BE is a result of chronic inflammation of the esophagus and is a risk factor for the development of EAC[11]. GI societies recommend regular endoscopy for BE patients to detect dysplasia or carcinoma early[12,13]. Endoscopic surveillance currently follows the Seattle protocol: Patients with BE are required to undergo a systematic four-quadrant biopsy, in which the entire BE area is sampled at intervals of 1-2 cm using a "turn and suction technique"[14]. However, this method can be invasive, costly, time-consuming, and difficult for patients to follow[15]. Due to poor patient compliance with the Seattle protocol, the American Society of Gastrointestinal Endoscopy established a performance threshold for optical diagnosis. Random biopsies can be replaced if targeted biopsies assisted by any imaging technique have a per-patient sensitivity of 90%, negative predictive value (NPV) of 98%, and specificity of 80%[16]. However, these requirements can only be achieved by experts.

In addition, early neoplastic lesions and dysplasia are subtle, showing focal distribution, and are difficult to detect endoscopically[17]. Cases of BE progression to early tumors are rare, and endoscopic surveillance is generally carried out in community hospitals; therefore, general endoscopists may not be familiar with these lesions, and this lack of familiarity is an important reason for missed diagnosis[18,19]. In recent years, to improve the diagnosis of BE, many new endoscopic techniques have been developed, such as magnification endoscopy (ME), chromoendoscopy, confocal laser endomicroscopy, and volumetric laser endomicroscopy, most of which are expensive and take a long time for endoscopists to learn[20,21]. Differences in endoscopists' interpretations of the images can also lead to differences in diagnosis[22]. Therefore, there is an urgent need for a practical tool to improve the accuracy of endoscopists in the clinic. Moreover, the endoscopist's diagnosis may be influenced by the time of the endoscopy, psychological state, time pressure, and cumbersome procedures. However, AI has a short learning time and, unlike endoscopists, does not suffer from fatigue easily; therefore, it has good application prospects (Table 1).

Table 1 Application of artificial intelligence in endoscopic detection of Barrett’s esophagus.

Ref.	Target disease	Endoscopic modality	AI technology	Database	Outcomes
van der Sommen et al[23], 2016	Early neoplasia in BE	WLI	SVM	100 images	Per-image sensitivity 83%/specificity 83%; Per-patient sensitivity 86%/specificity: 87%
Struyvenberg et al[24], 2021	BE	WLI/NBI	CNN	Train 494364 images/1247 images; test 183 images/157 videos	Images: Accuracy 84%/sensitivity 88%/specificity 78%; Videos: Accuracy 83%/sensitivity 85%/specificity 83%
de Groof et al[25], 2020	Early neoplasia in BE	WLI	ResNet-UNet	Train 1544 images; test 160 images	Dataset 4: Accuracy 89%/sensitivity 90%/specificity 88%; Dataset 5: Accuracy 88%/sensitivity 93%/specificity 83%
de Groof et al[26], 2020	Barrett’s neoplasia	WLI	ResNet-UNet	Train 1544 images; test 20 patients	Accuracy 90%/sensitivity 91%/specificity 89%
Hong et al[27], 2017	BE	Endomicroscopy	CNN	Train 236 images; test 26 images	Accuracy 80.77%
Hashimoto et al[28], 2020	Early neoplasia in BE	WLI/NBI	CNN	Train 1832 images; test 458 images	Accuracy 95.4%/sensitivity 96.4%/ specificity 94.2%
de Groof et al[29], 2019	Barrett’s neoplasia	WLI	SVM	60 images	Accuracy 92%/sensitivity 95%/specificity 85%

BE: Barrett’s esophagus; WLI: White light imaging; SVM: Support vector machine; NBI: Narrow band imaging; CNN: Convolutional neural network.

Computer-aided diagnosis using white light imaging/narrow band imaging

van der Sommen et al[23] collected 100 images from 44 BE patients and created an ML algorithm called support vector machine (SVM), which employed specific texture and color filters to detect early neoplasia in BE. The sensitivity and specificity of the system were both 83% for the per-image analysis and 86% and 87% for the per-patient analysis, respectively.

Struyvenberg et al[24] developed a CAD system based on a CNN model that was first trained with 494364 images and then further trained with 690 BE neoplasia and 557 nondysplastic BE (NDBE) white light imaging (WLI) images. Next, 112 BE neoplasia and 71 NDBE narrow band imaging (NBI) zoom images were used for training and validation. Finally, 59 BE neoplasia and 98 NDBE NBI zoom videos were used for training and validation. Fourfold cross-validation was used to evaluate the detection performance of the CAD system. The results showed that the accuracy, sensitivity, and specificity of the NBI zoom image based CAD system were 84%, 88%, and 78%, respectively. Accuracy, sensitivity, and specificity of the NBI zoom videos were 83%, 85%, and 83%, respectively.

de Groof et al[25,26] developed a CAD system based on ResNet/U-Net model to help endoscopists detect early BE neoplasia. The system was trained with 1544 endoscopic images of BE neoplasia and NDBE and then validated on 160 images. In an in vitro study, the accuracy, sensitivity, and specificity of the CAD system for detecting early BE neoplasia were 89%, 90%, and 88% in dataset 4, and 88%, 93% and 83% in dataset 5, respectively. Compared with 53 nonspecialist endoscopists, the CAD system outperformed them in terms of accuracy and sensitivity. In an in vivo evaluation of the CAD system, endoscopic examinations were performed on ten patients with NDBE and ten patients with BE neoplasia. The images obtained by WLI were analyzed immediately by the CAD system and used to provide feedback to the endoscopist. The accuracy, sensitivity, and specificity of the CAD system were 90%, 91% and 89%, respectively. Therefore, the CAD system has a high accuracy for tumor detection and low false positive rate; thus, the CAD system can be tested in larger and multicenter trials.

Hong et al[27] constructed a CNN-based CAD system to distinguish intestinal metaplasia (IM), gastric metaplasia (GM), and BE neoplasia. The researchers obtained 236 endoscopic images of BE from the 2016 International Symposium on Biomedical Imaging using 155 IM, 26 GM, and 55 BE neoplasia samples as a training set. Because the number of images in the training set was insufficient, the researchers implemented image distortion to achieve data enhancement and increase the sample size of the data. Then, 26 images, including 17 IM, 4 GM, and 5 BE neoplasia images, were used as the verification set. The results showed that the accuracy of the CAD system for the classification of IM, GM, and BE neoplasia was 80.77%. Although the number of images was small, this study suggested that the CNN-structured CAD system can be applied to the classification of esophageal lesions.

Real-time recognition by computer-aided diagnosis

Hashimoto et al[28] collected 916 images from 70 patients with early neoplastic BE and 916 control images from 30 normal BE patients and then trained a CNN algorithm on ImageNet. The researchers analyzed 458 images using the CNN algorithm. The accuracy, sensitivity, and specificity of the system for detecting early neoplastic BE were 95.4%, 96.4%, and 94.2%, respectively.

de Groof et al[29] designed an ML algorithm called SVM based on WLI images from 40 BE neoplasias patients and 20 NDBEs patients. All of the images were delineated by endoscopic experts, with overlapping areas of at least four delineations marked as "sweet spots" and areas with at least one delineation marked as "soft spots". The CAD system was trained (color and texture features) and then evaluated for its performance using leave-one-out cross-validation. The accuracy, sensitivity, and specificity of the CAD system were 92%, 95%, and 85%, and the localization and labeling of soft spots were 100% and 90%, respectively. Therefore, this CAD system can detect and locate early BE neoplasia with a high accuracy on WLI images, which lays a foundation for the real-time automatic recognition of BE neoplasia in the future.

APPLICATION OF ARTIFICIAL INTELLIGENCE IN ESOPHAGEAL CANCER

EC is usually diagnosed at an advanced stage, and the main treatment is esophagectomy. Surgical treatment is a highly invasive treatment with relatively high mortality and recurrence rates and poor patient prognoses. However, if EC is detected at an early stage, the prognosis can be improved by endoscopic resection[30,31]. Therefore, the early diagnosis of EC is essential for favorable treatment. Some studies have applied certain dyes to the esophageal mucosa that can more clearly reveal the surface vasculature and neoplasia. The most commonly used dyes are acetic acid, iodine, indigo carmine, and methylene blue. However, there are limitations in terms of the cost and complexity of their application[32,33]. NBI provides a better view of intrapapillary capillary loops (IPCLs) and is used to detect superficial ESCC. However, inexperienced endoscopists are still prone to missed diagnoses[34-36]. Therefore, AI, which can outperform humans in image recognition, is expected to be used in the field of EC diagnosis (Table 2).

Table 2 Application of artificial intelligence in endoscopic detection of early esophageal cancer.

Ref.	Target disease	Endoscopic modality	AI technology	Database	Outcomes
Ebigbo et al[37], 2019	EAC	WLI/NBI	CNN	248 images	Augsburg database: Sensitivity 97%/specificity 88% (WLI); Sensitivity 94%/specificity 80% (NBI); MICCAI database: Sensitivity 92%/specificity 100%
Ebigbo et al[38], 2020	EAC	WLI	CNN	Train 129 images; test 62 images	Accuracy 89.9%/sensitivity 83.7%/specificity 100%
Horie et al[39], 2019	EC	WLI/NBI	CNN	Train 8428 images; test 1118 images	Accuracy 98%/sensitivity 98%
Cai et al[40], 2019	ESCC	WLI	DNN	Train 2428 images; test 187 images	Accuracy 91.4%/sensitivity 97.8%/specificity 85.4%
Ohmori et al[41], 2020	ESCC	WLI/NBI/BLI	CNN	Train 22562 images; test 727 images	Non-ME: Accuracy 81.0%/sensitivity 90%/specificity 76% (WLI); Accuracy 77%/sensitivity 100%/specificity 63% (NBI/BLI); ME: Accuracy 77%/sensitivity 98%/specificity 56%
Liu et al[42], 2020	EC	WLI	CNN	Train 1017 images; test 255 images	Accuracy 85.83%/sensitivity 94.23%/specificity 94.67%
Kumagai et al[43], 2019	ESCC	ECS	CNN	Train 4715 images; test 1520 images	Accuracy 90.9%/sensitivity 92.6%/specificity 89.3%
Guo et al[44], 2020	ESCC	NBI	CNN	Train 6473 images; test 6671 images and 80 videos	Images: Sensitivity 98.04%/specificity 95.03%; videos: Non-ME sensitivity 60.8% (per frame)/100% (per lesion); ME sensitivity 96.1% (per frame)/100% (per lesion)
Tokai et al[46], 2020	ESCC	WLI/NBI	CNN	Train 1751 images; test 291 images	Accuracy 80.9%/sensitivity 84.1%/specificity 73.3%
Nakagawa et al[47], 2019	ESCC	WLI/NBI	CNN	Train 14338 images; test 914 images	Accuracy 91%/sensitivity 90.1%/specificity 95.8%
Zhao et al[48], 2019	ESCC	NBI	Double-labeling FCN	1350 images	Lesion level: Accuracy 89.2%; pixel level: Accuracy 93%
Everson et al[49]	ESCC	NBI	CNN	7046 images	Accuracy 93.7%/sensitivity 89.3%/specificity 98%
Uema et al[50], 2021	ESCC	NBI	CNN	Train 1777 images; test 747 images	Accuracy 84.2%
Fukuda et al[51], 2020	ESCC	NBI/BLI	CNN	Train 28333 images; test 144 patients	Accuracy 63%/sensitivity 91%/specificities 51% (detection); accuracy 88%/sensitivity 86%/specificities 89% (characterization)
Shimamoto et al[52], 2020	ESCC	WLI/NBI/BLI	CNN	Train 23977 images; test 102 videos	Non-ME: Accuracy 87%/sensitivity 50%/specificity 99%; ME: Accuracy 89%/sensitivity 71%/specificity 95%
Waki et al[53], 2021	ESCC	WLI/NBI/BLI	CNN	Train 18797 images; test 100 videos	Sensitivity 85.7%/specificity 40%

EAC: Esophageal adenocarcinoma; WLI: White light imaging; NBI: Narrow band imaging; CNN: Convolutional neural network; EC: Esophageal cancer; ESCC: Esophageal squamous cell carcinoma; DNN: Deep neural network; BLI: Blue laser imaging; ME: Magnification endoscopy; ECS: Endocytoscopic system; FCN: Fully convolutional network.

Detection of lesions

Ebigbo et al[37] created a CAD system based on CNN. In the Augsburg database, the sensitivity and specificity of the CAD system for the diagnosis of EAC in WLI images were 97% and 88%, respectively, and the sensitivity and specificity in NBI images were 94% and 80%, respectively. In the MICCAI database, the sensitivity and specificity of the CAD system for the diagnosis of EAC in WLI images were 92% and 100%, respectively. Then, Ebigbo et al[38] developed an artificial neural network of encoder-decoders with 101 layers of ResNet and trained the CAD system using 129 endoscopic images from the Augsburg database. The researchers evaluated 62 images using the CAD system, including 36 images of early EAC and 26 images of BE. Although the number of patients evaluated was low, real-time monitoring of EAC demonstrated good results. The sensitivity and specificity of the system were 83.7% and 100%, respectively, and the overall accuracy was 89.9%.

Horie et al[39] developed a CNN system using DL to correctly detect EC based on 8428 images from 384 patients with EC. The researchers used the CNN system to analyze 1118 images (47 patients with EC and 50 patients without EC). The system takes 27 s and has a sensitivity of 98%; it can detect EC lesions less than 10 mm in size. The NPV was 95%, but the positive predictive value was only 40%. This may be due to the small number of DL training sets and few images from patients with esophageal inflammation. In addition, the system can distinguish between superficial EC and advanced EC with a 98% accuracy. These results indicate that the CNN system constructed by researchers can accurately analyze a large number of endoscopic images in a short period of time, which is conducive to the early diagnosis of EC.

Cai et al[40] developed a CAD system using a deep neural network based on 2428 endoscopic images (746 patients) with the aim of identifying early ESCC from WLI images. Among these images, there were 1332 ESCC images and 1096 normal tissue images. The researchers evaluated the CAD system using 187 images (52 patients), and 16 endoscopic physicians reviewed the images. The results showed that the accuracy, sensitivity, and specificity of the CAD system for the early diagnosis of ESCC were 91.4%, 97.8% and 85.4%, respectively. With the help of the CAD system, the diagnostic accuracy and sensitivity of endoscopists with different seniority levels were improved, especially for those with less seniority. This result indicates that AI-assisted digestive endoscopy can reduce the rate of missed diagnosis and improve the diagnostic level of endoscopists with different experiences in early EC.

Ohmori et al[41] developed a CAD system based on CNN to evaluate the diagnosis of ESCC under ME and non-ME. The researchers used 7844 ME and 9591 non-ME images from ESCC and 3435 ME and 1692 non-ME images from noncancerous or normal esophagi as a training set. Then, 255 non-ME WLI images, 268 non-ME-NBI/blue laser imaging (BLI) images, and 204 ME-NBI/BLI images of ESCC were used as a validation set. The accuracy, sensitivity, and specificity of the CAD system were 81%, 90%, and 76%, respectively, in non-ME WLI images. In the non-ME diagnosis of NBI/BLI images, the accuracy, sensitivity, and specificity of the CAD system were 77%, 100%, and 63%, respectively. In the diagnosis of ME, the CAD system had an accuracy of 77%, sensitivity of 98%, and specificity of 56%. In conclusion, the diagnosis of ESCC with the CAD system was not significantly different from that of experienced endoscopists.

Liu et al[42] developed a CNN model using the DL approach to distinguish among normal esophagi, precancerous lesions, and EC. The model consists of two subnetworks: The O-stream and the P-stream. In the application process, the O-stream is used to input the original images to extract color changes and overall features, and the P-stream is used to input the preprocessing images to lift texture changes and detail features. In total, 1017 images (normal esophagi, precancerous lesions, and EC) were used as the training set, and 255 images (normal esophagi, precancerous lesions, and EC) were used as the validation set. The results showed that the accuracy, sensitivity, and specificity of the CNN model were 85.83%, 94.23% and 94.67%, respectively, which shows good prospects in the diagnosis of esophageal lesions.

Kumagai et al[43] constructed an AI model based on CNN with GoogLeNet to judge benign and malignant endocytoscopic system (ECS) images with different degrees of magnification. The AI system was trained using 4715 esophageal ECS images (1141 malignant and 3574 nonmalignant) and validated using 1520 images (27 ESCCs and 28 nonmalignant lesions). The results showed that the sensitivity of the AI system was 92.6% for the diagnosis of ESCC, the specificity was 89.3% for the diagnosis of nonmalignant lesions, and the accuracy was 90.9% for the overall diagnosis. Early EC under endoscopy usually presents as slight swelling, depression, or a color change in the mucosa, which is difficult to diagnose, especially for less experienced endoscopists. The above research results indicate that AI has good auxiliary value for the endoscopic diagnosis of early EC and its precancerous lesions and plays an important role in guiding learning for the applications of some new standards and technologies.

Scope of lesions

Guo et al[44] developed a CAD system based on CNN for the real-time detection of precancerous lesions and ESCC. A total of 6473 NBI images were used to train the CAD system, and endoscopic static images and dynamic videos were used to validate the CAD system. Each input endoscopic image generates an AI probabilistic heat map, where yellow indicates highly suspected cancerous lesions and blue indicates noncancerous lesions. When the CAD system is used to detect canceration, the identified tumor area is covered with color. The CAD system was used to diagnose 1480 malignant NBI images and 5191 nonmalignant NBI images with a sensitivity and specificity of 98.04% and 95.03%, respectively. In 27 non-ME and 20 ME videos of precancerous lesions and early ESCC, the sensitivities per frame were 60.8% and 96.1%, respectively, and the sensitivities per lesion were 100% and 100%. In 33 normal esophageal videos, the specificities were 99.9% per frame and 90.9% per case. The AI model can mark the location and range of lesions according to the input images, and the range is roughly the same as that marked by endoscopists. This finding indicates the feasibility and great potential of AI in the identification of a range of precancerous or early EC lesions.

Depth of lesions

The depth of EC invasion is a key factor affecting treatment decisions. In principle, endoscopic resection can be performed for intraepithelial esophageal lesions confined to the lamina propria or muscularis mucosa and/or lesions with a submucosal infiltration depth less than 200 μm. Surgical resection and chemoradiotherapy are required for lesions larger than 200 μm. Therefore, accurate determination of the depth of infiltration can avoid the impact of overtreatment on patient quality of life[45].

Tokai et al[46] collected 1751 ESCC images to design an AI diagnostic system using CNN techniques. The system used DI technology to evaluate the infiltration depth of ESCC. The researchers used the AI system to evaluate 55 patients (291 images) and compared them with the evaluations of 13 endoscopists. It was found that the detection rate of the AI system for ESCC was 95.5%, taking 10 s. In the images with ESCC detected, the accuracy, sensitivity, and specificity of the assessment of infiltration depth were 80.9%, 84.1%, and 73.3%, respectively, taking 6 s. Moreover, the AI system was more accurate than 12 of the 13 endoscopists. This result indicates that the AI system has great potential in detecting the infiltration depth of ESCC.

Nakagawa et al[47] developed a CNN-based AI system to assess the infiltration depth of ESCC. The researchers trained the AI system with images from 804 EC patients (8660 non-ME images and 5678 ME images) and then validated the system with images from 155 patients (405 non-ME images and 509 ME images). The accuracy, sensitivity, and specificity of the system were 91%, 90.1%, and 95.8%, respectively. When 16 endoscopists evaluated the same images, the accuracy, sensitivity, and specificity were 89.6%, 89.8%, and 88.3%, respectively. These results suggest that the AI system performs well in assessing the depth of ESCC infiltration, even better than endoscopists.

IPCLs are the hallmark of ESCC, and their morphologic changes correlate with the depth of tumor invasion. Zhao et al[48] used the ME-NBI technique to evaluate patients' esophageal conditions and established a CAD system for the automatic classification of IPCLs based on endoscopic diagnosis and histological analysis. This system uses a double-labeling fully convolutional network to evaluate 1350 images with 1383 lesions and compare them with the evaluations of endoscopists. The results showed that the diagnostic accuracy of the system was 89.2% at the lesion level and 93% at the pixel level, which were higher than those of endoscopists.

Everson et al[49] developed an AI system to detect the presence and stage of early ESCC lesions. A total of 7046 ME-NBI images from 17 patients were used to train the CNN. Among these patients, ten had early ESCC, and seven had a normal esophagus. All of the imaging areas were supported by histological results. Studies have shown that the accuracy of this CNN system for distinguishing normal and abnormal IPCL patterns is 93.7%, and the sensitivity and specificity for distinguishing abnormal IPCL patterns are 89.3% and 98%, respectively. Therefore, the CNN system can relatively accurately distinguish normal and abnormal IPCL patterns and may provide guidance for decision-making regarding the clinical treatment of ESCC.

Uema et al[50] constructed a CNN (ResNeXt-101) model to classify ESCC microvessels. The study used 1777 ESCC images under ME-NBI as a training set and 747 ESCC images under ME-NBI as a validation set (validated by the CAD system and 8 endoscopists). The results showed that the accuracy of the CAD system for microvascular classification was 84.2%, which was higher than the average accuracy achieved by endoscopists. Therefore, this CAD system has good application potential for ESCC microvascular classification.

Dynamic images

Fukuda et al[51] developed a CNN-based CAD system to diagnose ESCC. The researchers used 23746 ESCC images (1544 patients) and 4587 noncancerous images (458 patients) as a training set. Video image clips from 144 patients were used as a validation set, and then 13 endoscopic specialists used the same videos for diagnoses. The accuracy, sensitivity, and specificity of the CAD system in identifying suspicious lesions were 63%, 91%, and 51%, respectively. The accuracy, sensitivity, and specificity in differentiating cancerous from noncancerous lesions were 88%, 86% and 89%, respectively. In previous studies, the diagnosis of ESCC by CAD systems was mainly based on static images, with few video images. Because video images are affected by many factors, such as distance, angle, breathing movement, and esophageal motility, using a CAD system to analyze video images is more challenging. Fukuda et al[51] demonstrated that compared with endoscopic experts, CAD systems are more sensitive to ESCC detection and have a significantly higher accuracy and specificity in differentiating cancer from noncancer, which will provide valuable clinical support for endoscopists in their diagnoses.

Using 23977 ESCC images (6857 WLI images and 17120 NBI/BL images) as a training set, Shimamoto et al[52] developed a CNN-based AI system to assess the infiltration depth of ESCC. The AI system was then validated on 102 video images, while some endoscopic specialists were invited to view the same video images for diagnoses. The study showed that the accuracy, sensitivity, and specificity of AI for ME diagnosis were 89%, 71%, and 95%, respectively, and those for non-ME diagnosis were 87%, 50%, and 99%. Compared with the diagnostic parameters of endoscopic experts, those of the AI system were mostly higher. This suggests that AI system can provide useful support during endoscopy.

Waki et al[53] constructed an AI system based on CNN with 17336 images of ESCC (1376 patients) and 1461 images of noncancerous/normal esophagi (196 patients). While recording the verification video, the endoscopic operator passed through the esophagus at a constant speed to simulate a situation when a lesion was missed. A total of 100 videos (50 ESCCs, 22 noncancerous esophagi, and 28 normal esophagi) were then evaluated by the AI system and 21 endoscopists. The study showed that the sensitivity and specificity of the AI system for ESCC diagnosis were 85.7% and 40%, respectively, and those of the endoscopists were 75% and 91.4%, respectively. With the help of the AI system, the diagnostic specificity of the endoscopists was almost the same, but the sensitivity was improved. Therefore, the AI system, as an auxiliary tool, plays an important role in the diagnosis of ESCC by endoscopists.

PROBLEMS AND PROSPECTS OF AI-ASSISTED ENDOSCOPIC DIAGNOSES

With continuous improvements in endoscopic technology and the diagnostic levels of AI, the combination of AI and endoscopy has become popular. Although AI has made some achievements in the diagnosis of esophageal precancerous lesions and early EC, there are still some problems.

False positive and false negative results

First, almost all AI diagnostic systems yield some false negative results. Small lesions are easily missed in clinical practice, so it is crucial to improve the detection accuracy of these easily neglected lesions. In addition, the AI diagnostic system yields false positive results, which can lead to overtreatment. Shadowed portions, color changes in the gastric antrum and pylorus, and changes in the normal tissue structure and benign lesions (scarring, local atrophy, inflammation, ectopic esophagus, and gastric mucosa) are all reasons for false positive results[54]. To solve this problem, on the one hand, a large number of high-quality endoscopic images should be accumulated for computer algorithm training and verification to produce more accurate results. On the other hand, endoscopic videos often contain more low-resolution real images, which are difficult to capture in still pictures. The use of a large number of images taken from videos as learning materials can reduce the rates of false positives and false negatives to a certain extent.

Retrospective experimental studies have a single source of learning materials, and prospective experimental studies are lacking

At present, most of the training data sets and validation data sets of AI systems have been derived from the same batch of data from the same center. Although the accuracy of AI systems has been internally verified, there is still a lack of external verification[55]. The resolution of examination images obtained by different types of endoscopes varies greatly among different devices. Therefore, future studies should try to include endoscopic image data from multiple institutions, multiple models, and multiple devices to ensure the repeatability of the research results.

In addition, most of the current studies are retrospective, and researchers tend to select clear and high-quality endoscopic images after excluding low-quality images caused by interference factors (such as bleeding, mucus secretion, and food interference), thus resulting in selection bias. This bias often causes the results of retrospective trials to outperform the actual results in clinical applications[56]. In the future, a large number of prospective studies should be carried out to continuously improve AI systems and improve their accuracy, sensitivity, and specificity for clinical trials to lay a solid foundation for the real-time clinical application of AI.

Lack of endoscopic video-assisted diagnoses

Currently, most AI systems are based on the processing of static data rather than the modeling of dynamic videos. Static images are mostly taken after the mucosal environment is well prepared and the lesion location is determined. Due to the lack of environmental impact caused by poor preparation of the mucosal environment and endoscopic movement in dynamic videos, information is missing[57]. There is a large gap between AI training sets and the actual endoscopic working environment, which affects the clinical applicability of AI to some extent. The application of video sets can better solve the above problems. Moreover, endoscopic video analysis can be used for secondary review after real-time endoscopy to quickly identify and screen esophageal diseases and reduce the number of missed diagnoses, as this type of analysis has considerable development potential in DL-assisted endoscopy in the future.

Prospects for development

CAD system based on DL technology has gained increasing attention and is closely related to the good development prospects of DL technology applied in real-time endoscopy. CAD can indicate the lesion site in real-time endoscopic examinations, provide an accurate classification, and serve as a second observer to assist in disease diagnosis. In low-resource or densely populated areas, CAD is used for population-based endoscopic screening, which can avoid missed diagnosis or misdiagnosis of diseases caused by endoscopists' lack of experience and professional knowledge or heavy work fatigue. CAD can be used to train new endoscopists who lack experience, provide them with professional knowledge training, and improve their professional skills. CAD can also be performed online to provide more professional endoscopic diagnoses in areas where experienced endoscopists are lacking, making it easier for patients to visit local hospitals.

CONCLUSION

Most of the current research is still focused on early system development and feasibility studies, but subsequent product development has not followed. The CAD system based on DL is still in the experimental research stage. Therefore, in the future, a large number of high-quality prospective experimental studies should be carried out in combination with high-quality algorithms and frameworks with more powerful functions, higher efficiency, and better stability. With the establishment of a standardized and large sample data center, the CAD system can provide endoscopic physicians with more accurate diagnosis and treatment options, auxiliary teaching, auxiliary assessments, and telemedicine for early EC. An increasing number of patients and physicians will benefit from the progress of the CAD system.