Techniques to integrate artificial intelligence systems with medical information in gastroenterology

Abstract

Gastrointestinal (GI) endoscopy is the central element in contemporary gastroenterology as it provides direct evidence to guide targeted therapy. To increase the accuracy of GI endoscopy and to reduce human-related errors, artificial intelligence (AI) has been applied in GI endoscopy, which has been proved to be effective in diagnosing and treating numerous diseases. Therefore, we review current research on the efficacy of AI-assisted GI endoscopy in order to assess its functions, advantages and how the design can be improved.

Key Words: Gastrointestinal endoscopy, Artificial intelligence, Diagnosis, Advantages

Core tip: Artificial intelligence (AI) has been the center of medical information in the 21^st century and we have witnessed the tremendous change it has triggered in the diagnosis and treatment of many diseases. Gastrointestinal endoscopy is the core element of clinical procedures in modern gastroenterology as it provides direct evidence and guides precise diagnosis and treatment. Therefore, in this article, we review the latest findings on AI-assisted gastrointestinal endoscopy concerning its applications in the diagnosis and treatment of gastrointestinal diseases.

INTRODUCTION

The 21^st century has witnessed a tremendous revolution in life sciences. Targets within cells are increasingly being found so that targeted therapiesc, which will provide the maximum benefits while causing minimum or even no damage, are available to treat difficult miscellaneous diseases; hereditary information is continuously being deciphered in order that much more in-depth information on the mechanism of disease occurrence and progression can be established and interpreted. In addition, the first 20 years of the 21^st century has also experienced the combination of computer science and clinical medicine, or what we call the application of artificial intelligence (AI) in the diagnosis and treatment of diseases. With the help of machine learning and deep learning algorithms, the sensitivity and specificity of diagnosis involving morphological judgement has rapidly increased, such as the diagnosis of diabetic retinopathy and breast cancer screening using mammography[1-3]. Moreover, incorporated with convoluted neural network (CNN) technology, automated classification of the condition of skin lesions is even possible by experts from a distance[4]. Thus, with the development of network technology to change from 4G network to 5G or an even more advanced network, the addition of AI in medicine will play a more important role in helping clinicians to more accurately combat diseases[5].

The diagnosis and treatment in gastrointestinal (GI) diseases has become more accurate and evidence-based since the popularization of GI endoscopy, which helps detect early-stage lesions and malignancies and thus guide the subsequent intervention[6]. In addition, GI endoscopy also contributes to the removal of early-stage lesions, which results in minuscule operative wounds and prevents further malignant change[7]. However, despite the fact that an increasing number of physicians are trained to operate a GI endoscope, a number of mis-diagnoses are reported annually due to physicians’ incompetence, carelessness and visual fatigue[8]. AI-assisted GI endoscopy has been proved to have considerable potential in reducing the number of errors in order to optimize clinical performance by establishing a more suitable treatment strategy and improving long-term prognosis. As many clinical studies have been carried out in recent years, some of the basic disciplines and information concerning the area are known; however, global research is still in a very early phase[9]. Gastroenterology is regarded as a field where AI could have a significant impact and shape the future diagnosis and treatment pattern as both rely greatly on image- or video-based investigations[10]. Some of the research carried out so far has demonstrated that AI-guided endoscopy provides more solid evidence of suspicious neoplasia during examinations and assists optical biopsy to determine the features of lesions and subsequently integrate genomic and epigenomic information to provide optimal therapeutic plans[11]. Therefore, this review aims to summarize high-quality studies completed so far in order to assess the efficiency of the latest AI technology incorporated into GI endoscopy and determine how this technology can be improved.

THE ROLE OF AI IN GI ENDOSCOPY

To date, AI has proved efficient in aiding endoscopic examination and the treatment of GI lesions with high sensitivity, specificity and a successful treatment rate. These lesions include polyps, acute bleeding, precursor lesions and early-stage malignant tumors, especially tumors invading the mucosal and submucosal layers[12]. Without AI, observer variation and errors due to limited experience and expertise occur every now and then. AI-assisted GI endoscopy, is believed to largely reduce these errors and prevent visual fatigue. Among its applications, AI-guided identification and characterization of polyps is the earliest established and the best understood[13]. A team of physicians reported that their AI-guided model could not only accurately recognize the presence of polyps, but could also distinguish hyperplastic and adenomatous polyps based on the assessment of video images under GI endoscopy with a high sensitivity of 98% and a relatively satisfactory specificity of 83%. Their study indicated that AI-guided GI endoscopy was unlikely to miss possible malignant lesions[14]. Misawa et al[15] reported that AI-guided endoscopic optical biopsy based on the EndoBRAIN system could identify and characterize the pathological features of polyps with the aid of indigo carmine dye. If this technology was further improved, it could increase the detection rate of small polyps as well as judge their pathological features, which could lead to correct decision-making regarding resection of the polyps[15]. Similarly, another team used a different algorithm based on CNN to train an AI system using archived images from endoscopic videos. Their test results indicated that the accuracy was as high as 96.4% with an area under the curve (AUC) of the receiver operating characteristics (ROC) of 0.991. They even found that AI-guided GI endoscopy was capable of identifying small adenomas of 1-3 and 4-6 mm in size, and that the number of polyps identified by AI-guided GI endoscopy was much higher than that identified by human-operated GI endoscopy[16]. In 2019, a research group also demonstrated that AI-guided GI endoscopy showed higher efficiency in detecting small adenomas. This research group conducted an open and non-blinded trial with over 1000 patients, who were later randomly divided into 2 groups who underwent GI endoscopy with or without the aid of AI. It was found that AI-guided GI endoscopy increased the identification rate from 20.3% to 29.1% and increased the number of identified adenomas from 0.31 to 0.53 per patient. However, in this study, GI endoscopy with and without AI showed no difference when examining patients with diminutive polyps, as human eyes were also unlikely to miss such apparent lesions[17]. Interestingly, AI-guided GI endoscopy was found to be even more efficient when used by less competent endoscopists and it was reported to be able to increase the skills of these physicians, which might be of significant help in continuous education and promote the popularization of GI endoscopy[18]. Besides the detection of polyps, AI experts along with physicians are now able to detect pre-malignant or early-stage malignant lesions in the GI tract using the latest AI technology, which was a huge challenge as senior endoscopists would sometimes mistakenly ignore such tiny mucosal or submucosal changes[19]. According to a recent study, when used to detect gastric precursor and early-stage malignancy, AI-guided GI endoscopy had the capability of less diagnostic time but resulted in greater sensitivity (65.6% vs 31.9%) and a higher positive predictive value (PPV) (41.9% vs 36.7%) compared with the naked eye[20]. With the increased prevalence of gastroesophageal junctional diseases, such as gastroesophageal reflux disease and others, gastroesophageal junctional adenocarcinoma has been the focus of attention in many gastroenterologists. AI-guided GI endoscopy was demonstrated to be effective in aiding physicians to detect underlying problems in the gastroesophageal junction and judge their pathological features. Moreover, some technologies have even made it possible for an AI-guided endoscopic resection for early-stage lesions in the gastroesophageal area[21]. In addition to the identification of neoplasms and their pathological features, some recent AI-assisted programs have made it possible to evaluate the depth of cancer invasion, which is of great help to clinicians as the invasion depth is difficult to evaluate with the naked eye. A team in Japan demonstrated that by using white light imaging (WLI) and narrow-band imaging (NBI), an AI system could be trained to differentiate superficial and deep invasion of esophageal squamous cell carcinoma (ESCC) within several seconds and with an accuracy of more than 80%[22]. Besides the determination of invasion depth, another team found that AI could actually define the benign and malignant borderline and subsequently help guide endoscopic dissection[23]. Moreover, the ability to judge whether the dissection completely removed the suspected malignancy has contributed greatly to planning subsequent therapy. Therefore, if these technologies could be further validated and developed, AI-guided GI endoscopy could have greater application potential.

URGENT NEED FOR AI-GUIDED ESOPHAGOGASTRODUODENOSCOPY

With the popularization of esophagogastroduodenoscopy (EGD), it is now possible to detect stomach lesions at an early stage. However, as early-stage lesions are much more insidious in terms of size, morphology and biological activity, the efficiency varied with the competence of endoscopists as long-term specialized training is mandatory to gain the expertise and experience needed to detect insidious precursor lesions[24]. This was confirmed by a series of statistics reporting that the rate of mis-diagnosis of upper GI lesions was around 15% over the last 3 years mainly due to human factors[25,26]. To resolve this problem, AI-guided GI endoscopy was invented to reduce the possibility of human-related errors. However, since GI endoscopy carried more uncertainty and anatomical variations, the application of AI in GI endoscopy has been difficult[27].

AI-GUIDED EGD IN DEFINING GI MALIGNANCIES

One of the milestones of EGD is that it has made it possible to detect and resect precursor cancerous tissue and so prevent traditional surgical resection which would produce massive tissue damage. Thus, there was always an urgent need to increase the sensitivity, specificity and accuracy for the detection of precursor cancerous lesions under EGD. The first attempt to combine AI and EGD was by a Japanese scholar who trained his system with WLIs, NBIs and chromoendoscopy based on indigo carmine. Validation with 2296 images provided a sensitivity of 92.2% and a PPV of 30.6%[28,29]. Therefore, this indicated that despite a satisfactory detectable rate, it might also produce a large number of false positive results, thus aggravate the social medical burden. Another Japanese team evaluated a CNN-based model trained using an endoscopic video and reported a sensitivity of 94.1%[20,30]. A Japanese team attempted to diagnose Helicobacter pylori (H. pylori)-related gastritis based on WLIs, NBIs and chromoendoscopy images and videos, and demonstrated a sensitivity and specificity of 81.9% and 83.4%, respectively[31]. A study validated the performance of their AI-guided model using 100 defined gastric cancer examination videos and 100 non-gastric cancer examination videos and found a sensitivity of 94.0%, a specificity of 91.0% and an accuracy of 92.5%[32]. A multicenter study validated the capability of their AI-guided diagnosis system using 7 validation sets collected from over 10 different hospitals to detect upper and lower GI tract tumors. The reported accuracy was between 91.5% and 97.7% with regard to different validation subsets[33]. They also compared the performance of their AI-guided GI endoscopy to the results of senior experienced physicians and junior physicians working in minor hospitals, which indicated that the AI-guided system could achieve comparative sensitivity to that of the experts (94.2% vs 94.5%) and could exceed that of junior physicians (94.2% vs 72.2%). Considering that most patients would consult outside of advanced or national hospitals, the help provided by AI-guided systems is necessary in minor hospitals to ensure diagnostic accuracy. Kanesaka et al[34] trained an AI system with the help of NBIs and successfully achieved an accuracy of 96%. Besides the aforementioned studies, other studies have also reported high accuracy and sensitivity for the detection of early-stage lesions using AI systems trained using magnified NBIs, which seem to be the future direction[35]. According to some other reports, AI-guided GI endoscopy was not only able to detect early-stage lesions, but was also capable of characterizing their features, such as invasion depth or biological activities. For example, an AI-guided system was used to estimate the invasion depth and the accuracy was 89.16%, which was much higher than that by humans[36,37]. Our team also attempted to build an AI-assisted automated system for the diagnosis of precancerous lesions and ESCC by training the system using 6473 NBIs images and 47 video datasets. Our findings demonstrated that the AI system involving deep learning could achieve a sensitivity of 98.04% and a specificity of 95.03% when distinguishing between ESCC and non-cancerous lesions[38].

AI-GUIDED EDG IN DEFINING OTHER GI DISORDERS

Besides defining early GI tumors, AI is also able to determine other benign gastric disorders, such as chronic non-atrophic gastritis, gastric and duodenal ulcers, etc. Among these, the most well-known is the ability to recognize H. pylori gastritis, which has been widely discussed. In 2020, Lui et al[39] carried out a meta-analysis involving 23 studies including 969318 images. They pointed out that the AUC for AI detection of Barrett’s esophagus, neoplastic lesions in the stomach, squamous esophagus and H. pylori infection state were 0.96 (95%CI: 0.93-0.99), 0.96 (95%CI: 0.93-0.99), 0.88 (95%CI: 0.82-0.96) and 0.92 (95%CI: 0.88-0.97), respectively[39,40]. They also pointed out that by using NBIs, the AI system was superior to white light with regard to the detection of neoplastic lesions of the esophagus (0.92 vs 0.83, P < 0.001). Moreover, they reported a superior performance of the AI system over the human eye in detecting neoplastic lesions in the stomach (AUC 0.98 vs 0.87, P < 0.001), Barrett’s esophagus (AUC 0.96 vs 0.82, P < 0.001) and H. pylori state (AUC 0.90 vs 0.82, P < 0.001)[41,42]. Earlier this year, Xia et al[43] developed a new automatic lesion detection system using CNN and faster region-based CNN (Faster-RCNN) and a total of 1023955 MCE images were used to train the AI system and help validate it, including erosion, polyps, ulcers, submucosal tumors, xanthomas, normal mucosa, and invalid images. They found that their AI system could detect gastric lesions with a sensitivity of 96.2% (95%CI: 95.7%-96.5%), a specificity of 76.3% (95%CI: 75.97%-76.3%), a PPV of 16.0% (95%CI: 15.7%-16.3%), a negative predictive value (NPV) of 99.7% (95%CI: 99.74%-99.79%). They also demonstrated the accuracy for each type of lesion, the accuracy for erosion was 77.1% (95%CI: 76.9%-77.3%), the accuracy for polyps was 96.5%, the accuracy for ulcers was 89.3%, the accuracy for submucosal tumors was 87.2%, the accuracy for xanthomas was 90.6%, the accuracy for normal tissues was 67.8% and the accuracy for invalid images was 96.1%[43,44]. Their study also showed that the AI system was likely to indicate problems during an endoscopy examination rather than determine that it was normal. Another team also performed a validation test using an AI model based on WLIs and reported a sensitivity of 86.7%[45,46]. In addition, they pointed out that AI-guided GI endoscopy met difficult problems when trying to define benign lesions compared with malignant lesions as the stomach is often inflamed and even eroded which could add to the difficulty in making a definite diagnosis. Another study also reported the diagnostic value of AI-guided GI endoscopy based on CNN technology with an accuracy of 92.9%detected[47]. Some scientists have started to optimize the AI system by introducing blue light imaging and linked color imaging techniques, and have compared their efficiencies with single WLI. The results showed that the AUCs of ROC analysis of blue light imaging, linked light imaging and WLI were 0.96, 0.95 and 0.66, respectively, which indicated that the newly introduced technologies could enhance the examination findings[45]. In addition to defining H. pylori-related gastritis, deep learning technology has also helped physicians to detect and evaluate gastric and duodenal ulcers and predict their prognosis[40,48]. With regard to polyps, contemporary AI technology is able to precisely detect polyps, make an accurate classification based on histology, predict the possibility of disease progression and guide subsequent treatment. In the past, older models of computer-aided diagnosis could not analyze polyps in real-time, which resulted in the diagnosis of polyps being challenging. A scientific team designed an AI model with the capability of analyzing nearly 100 images a second which greatly increased the speed of machine reading as the previous model was only able to process fewer than 10 images a second[49]. In addition, the technology they applied allowed their model to achieve an accuracy of up to 96.4% when detecting polyps among 8641 images of 2000 patients. Later, similar models were designed and used to compare the detection efficiency between experts only and experts with the help of AI systems. The results demonstrated that the AI system was able to detect all polyps, which were also identified by the experts with a 7% false positive rate. Moreover, the AI system extracted 9 other insidious polyps which were not detected by the naked eye[50]. In addition, scientists developed a more advanced model based on deep learning which could determine the histological features of polyps. This team found that with the help of NBIs, the AI diagnostic model could achieve an accuracy of 95% while restricting the NPV value within the limit set by the Preservation and Incorporation of Valuable Endoscopic Innovations for Adenoma Assessment of Diminutive Adenomas[51]. One of the major purposes of AI-guided GI endoscopy was to reduce human-related factors as much as possible, and to maintain a stable sensitivity, specificity and accuracy regardless of the expertise of the operator. One AI model presented by Mori et al[18] demonstrated that the application of AI systems for real-time histological classification based on NBI or staining and magnification with an integrated endoscopy lens provided NPV rates of > 92 for distal diminutive lesions, which was not related to the operators’ expertise. In addition, full evaluation of the polyps could be done within a minute. The detailed information of some studies concerning the diagnosis of polyps and neoplasms in the GI tract published after 2018 is shown in Table 1.

Table 1 Detailed information on the studies concerning the diagnosis of polyps and neoplasms in the gastrointestinal tract published after 2018.

Ref.	Training	Validation	AUC	Sensitivity	Accuracy
Chen et al[51], 2018	1476 images of neoplasms; 681 images of H. pylori	188 images of neoplasms; 96 images of H. pylori	NA	96.3%	90.1%
Urban et al[16], 2018	8641 images; 9 videos	1330 images; 9 videos	0.974	NA	96.4%
Misawa et al[15], 2018	73 videos	Cross validation	NA	90%	76.5%
Yamada et al[56], 2019	4087 images of polyps; videos	705 images with polyps; 4135 images without polyps	0.975	97.3%	NA
Klare et al[57], 2019	NA	55 colonoscopy examination videos	NA	75.3%	NA
Wang et al[17], 2019	3634 images with polyps; 1911 images without polyps	5541 images with polyps and 21572 images without polyps	0.984	94.4%	NA
Song et al[58], 2020	12480 images	545 images	0.93	82.1%	81.3%
Zachariah et al[59], 2020	8246 images	634 images	NA	96%	94%

H. pylori: Helicobacter pylori; AUC: Area under curve; NA: Not applicable.

DISCUSSION

From the studies we have researched and analyzed in depth so far, we have found that by incorporating several AI technologies, GI endoscopy has achieved higher accuracy, faster diagnostic speed, and fewer human-related errors, etc. Firstly, AI technology has made it possible to eliminate the errors caused by doctors’ incompetence and lack of experience and has guided junior doctors and doctors working in less prestigious hospitals to gain the necessary expertise. Secondly, this technology improves the relevance rate and recall factor of less obvious and less typical lesions due to their size or atypical shape and helps to achieve “early discovery and early treatment”. Thirdly, the present AI technology is able to assist judgement in a number of lesion types including polyps, precursor changes in tumors, all types of mucosal and submucosal abnormalities, and inflammation, etc., which almost covers the disease spectrum of the GI tract. Thus, it can be concluded that as a diagnostic tool, AI greatly contributes to the work of clinical physicians.

However, studies concerning the guidance of AI during treatment under GI endoscopy have rarely been published and trials on training AI systems to gain the ability to direct the resection of malformations have seldom been discussed. One of the major advantages of GI endoscopy is that it allows the resection of abnormalities to be performed in a minimally invasive way, which results in less damage than traditional surgery or laparoscopic surgery, AI guided-treatment under GI endoscopy should be further developed and discussed. Moreover, an AI-guided robot physician may even be possible when AI is trained to guide such a process.

CONCLUSION

The last decade has witnessed a number of studies concerning the application of AI in modern medical procedures. However, due to specific reasons, there is an obvious lack of high-quality prospective clinical trials. In fact, despite the large number of clinical studies published so far, only 6 were prospective randomized controlled trials (RCTs) that were focused on the efficiency and effects of AI-guided models[17]. Far fewer RCTs have emphasized the comparison between machines and the human eye. Gastroenterology has always led RCTs concerning AI, and of the abovementioned 6 RCTs concerning AI in medical fields, 5 of them are related to gastroenterology. Therefore, more RCTs should be planned and carried out to gain more reliable data[52]. To perform effective RCTs, a series of protocols and rules should be strictly followed. For instance, the optimal study design approaches for clinical trials of AI have been put forward and these recommendations have significant implications for GI endoscopy. Clinically-related outcome measures should be prespecified according to the way the AI model is being investigated. Moreover, AI-assisted polyp detection studies should apply validated outcome parameters such as adenoma detection rate, adenomas per colonoscopy, or adenoma miss rate, etc[53-55].

The next couple of years will witness a tremendous change in the medical field with the ever-accelerating development of AI technologies, in which the field of gastroenterology will be the center of such unprecedented change. With the advancement of AI technology, more high-quality RCTs should be designed and carried out to assess the technologies being developed and to correct any errors. In addition, standardized methods that contribute to the storage, organization and labeling of clinical images should also be the focus of attention.