Artificial intelligence in gastroenterology: A state-of-the-art review

Abstract

The development of artificial intelligence (AI) has increased dramatically in the last 20 years, with clinical applications progressively being explored for most of the medical specialties. The field of gastroenterology and hepatology, substantially reliant on vast amounts of imaging studies, is not an exception. The clinical applications of AI systems in this field include the identification of premalignant or malignant lesions (e.g., identification of dysplasia or esophageal adenocarcinoma in Barrett’s esophagus, pancreatic malignancies), detection of lesions (e.g., polyp identification and classification, small-bowel bleeding lesion on capsule endoscopy, pancreatic cystic lesions), development of objective scoring systems for risk stratification, predicting disease prognosis or treatment response [e.g., determining survival in patients post-resection of hepatocellular carcinoma), determining which patients with inflammatory bowel disease (IBD) will benefit from biologic therapy], or evaluation of metrics such as bowel preparation score or quality of endoscopic examination. The objective of this comprehensive review is to analyze the available AI-related studies pertaining to the entirety of the gastrointestinal tract, including the upper, middle and lower tracts; IBD; the hepatobiliary system; and the pancreas, discussing the findings and clinical applications, as well as outlining the current limitations and future directions in this field.

Key Words: Artificial intelligence; Machine learning; Deep learning; Clinical applications; Gastroenterology

Core Tip: Artificial intelligence (AI) clinical applications in gastroenterology and hepatology, which heavily relies on imaging, have dramatically expanded in the last 20 years. These applications include the detection of lesions, identification of premalignant or malignant lesions, development of objective scoring systems for risk stratification, predicting disease prognosis or treatment response, or evaluation of metrics such as bowel preparation score or quality of endoscopic examination. The objective of this review is to pool the available AI-related studies pertaining to the entire gastrointestinal tract, discussing findings and clinical applications, as well as outlining the current limitations and future directions in this field.

INTRODUCTION

As artificial intelligence (AI) continues to rapidly evolve in medicine, the clinical applications of this technology are becoming increasingly evident[1]. Relying heavily on endoscopic and radiologic imaging, gastroenterology has become an attractive field in which to apply AI. Special interest has already been devoted to several areas, including the detection of gastrointestinal neoplastic lesions to assist with rapid diagnosis, reduction of misdiagnosis, improvement in quality of imaging, reduction of interobserver variability in visual classifications, and radiologic and histopathologic interpretation[2-4].

AI is a broad term that encompasses disciplines such as machine learning (ML) and subdisciplines or specific techniques such as deep learning (DL) (Figure 1). The central motivation of ML to use large datasets to recognize patterns of interactions between variables, often ultimately in a way that allows the learned function to be applied to new data[5]. ML is composed of both “supervised” and “unsupervised” learning methods. The goal of supervised learning is to predict a labelled output focusing primarily on classifying data input into specific subgroups, or alternately for prediction of quantitative outcomes[6]. An example of supervised learning is training a system to assist in identifying gastric intestinal metaplasia (GIM) using a large database of lesions that have previously been identified by an operator as corresponding to GIM. In comparison, unsupervised learning does not have an output to predict. It relies on attempting to identify naturally occurring patterns from within the input, often to then group them accordingly (e.g., tissue sample clustering based on similar gene expression values)[6]. DL is a subset of ML, based on artificial neural networks (ANN), which are loosely inspired by the neuronal interplay in the human brain. DL autonomously utilizes the data input to learn, identify, and leverage predictive factors of an outcome, which can use multi-layered systems [i.e., convolutional neural networks (CNN)] to process complex information[3,7]. The realization of the concept of DL has recently become possible with rapid advancements in specialized computer hardware, such as increased graphics processing unit power, and accompanying software and algorithmic achievements.

Figure 1 Primary concepts of artificial intelligence.

With increased recognition of the importance of AI in gastroenterology, the first global AI in gastroenterology and endoscopy summit was held in Washington D.C. in late 2019, which included multiple experts in the domains of academia, industry and regulatory institutions. The consortium anticipated that in the next 10 years the clinical applications of AI in gastroenterology will positively influence patient care and clinical workflow. The consortium recognized the importance of a close multidisciplinary collaboration between gastroenterologists, industry, and regulatory institutions in the development and application of new technologies in the clinical setting[8]. Therefore, the main objective of this review is to introduce the topic of AI and its clinical applications, outline current limitations and knowledge gaps, and trace the future directions in each field by summarizing the global rapidly expanding pool of ever-changing literature to date (Figures 2 and 3).

Figure 2 Geographic distribution of country from which the included studies originated.

Figure 3 Scatterplot depicting distribution of studies across the organ systems in gastroenterology and hepatology. GI: Gastrointestinal; GIB: Gastrointestinal bleeding; IBD: Inflammatory bowel disease; HB: Hepatobiliary.

UPPER GASTROINTESTINAL TRACT

Detection of premalignant and malignant lesions

The upper gastrointestinal tract has several areas of interest for detection of premalignant and malignant lesions, such as identification of dysplasia and early neoplasia in Barrett’s esophagus (BE), esophageal squamous cell carcinoma (SCC), and gastric cancer (GC)[9] (Supplementary Table 1).

Despite the fact that histopathologic analysis is the gold standard to establish the diagnosis of BE and determine the presence of dysplasia, it is of paramount importance for endoscopists to obtain targeted biopsies from specific locations that harbor the actual lesion. By identifying areas that may harbor BE with or without dysplasia, AI can orient the clinician in performing directed biopsies instead of relying on random sampling. Importantly, detection of early esophageal neoplasia with conventional white light imaging (WLI) and digital chromoendoscopy [i.e., narrow band imaging (NBI)] represents a challenge, an issue for which AI has been proposed as a possible solution[10,11].

In total, we included 8 studies that examined BE, all of which addressed the detection of dysplasia or early esophageal adenocarcinoma (EAC) based on endoscopic imaging or laser endomicroscopy[10-17]. The most popular analytic models were CNNs and support vector machines (SVMs), while cross-validation techniques were the primary basis of validation methods. In general, the models were able to discern between normal and dysplastic/neoplastic images with an accuracy of at least 89.9%, performing better than nonexpert endoscopists. De Groof et al[12] developed a DL algorithm based on a hybrid ResNet-UNet model using 4-fold per-patient cross-validation to detect early neoplasia in BE. The model was trained in a stepwise fashion, using a database with over 490000 endoscopic still images of the gastrointestinal tract for training, out of which 1247 corresponded to early BE neoplasia. This model achieved higher level of accuracy than non-expert endoscopists and could also detect the optimal biopsy site in up to 97% of patients. Ebigbo et al[14] developed a computer-aided diagnosis model based on ResNet that differentiated between normal BE and early EAC, with 89.9% accuracy, 83.7 sensitivity and 100% specificity.

Similarly, identification of SCC poses a similar clinical challenge, as traditional diagnostic techniques (e.g., chromoendoscopy with lugol or NBI) have relatively low specificity[18]. As complex novel methods that rely heavily on endoscopic imaging such as endocytoscopy or volumetric laser endomicroscopy have been progressively implemented in clinical practice, the interpretation of large volumes of images has been noted to be a challenging and time-consuming issue[19]. Therefore, AI-assisted image interpretation has also found a use in identifying abnormalities in the image inputs[17].

Thirteen studies examining esophageal cancer were examined, of which 11 specifically studied SCC. Nine of the studies targeted to develop DL models for malignancy detection, while two studies aimed to develop models that predict malignancy depth of invasion with DL models. Most studies (8) were based on CNN models, while others used joint diagonalization principal component analysis (JDPCA), VGG16 Net, or GoogLeNet as classifiers. Although the values of accuracy, sensitivity and specificity in esophageal SCC detection varied between the studies, all models performed at least as good as endoscopists in lesion detection and characterization or substantially improved the endoscopists’ lesion detection[20-30]. Fukuda et al[21] developed a DL model aimed at detecting suspicious lesions and characterize SCC using more than 28000 NBI-enhanced images in the construction of the model. The model achieved a higher sensitivity for SCC detection and higher accuracy for SCC characterization from normal tissue than endoscopic experts. Two studies by Nakagawa et al[31] and Shimamoto et al[32] aimed at developing models that predicted esophageal malignancy depth utilizing a DL model based on a CNN with a belief-propagation decoder using independent validation datasets. These models achieved an accuracy of 89.2% and 91% in predicting invasion depth, with sensitivities of 70.8% and 90.1%, and specificities of 94.4% and 95.8%, respectively.

GC is the fourth most common cause of cancer-related death in the world. Identification of premalignant lesions or early gastric neoplasia is of paramount importance. Unfortunately, as with esophageal diseases, several studies have shown that conventional endoscopic imaging (e.g., WLI or NBI) or other advanced endoscopic modalities (e.g., magnifying endoscopy, blue laser imaging) have relatively low sensitivity and specificity in identifying premalignant or early neoplastic gastric lesions[33-36].

Twenty-four studies examined gastric malignancy or premalignant conditions. Eleven (46%) studies directly addressed early GC (EGC) detection, of which eight models used DL, two used SVM and one used JDPCA. The sample size in the training and validation datasets varied from less than 100 million to 1.03 million. The accuracy in EGC detection of all models ranged from 86.5%-98.7%, with sensitivity of 80.0%-96.7% and specificity of 89.2%-100%[30-37-51]. Specifically, Wu et al[52] developed a CNN-based model using over 9000 images to train the algorithm, which not only detected EGC lesions with a 92.5% accuracy, 94.0% sensitivity and 91.0% specificity but also performed significantly better than expert endoscopists at this task. Two studies, one based on faster region-based CNNs and another one based on quantum neural networks, used biomarkers, histology and computed tomography (CT) images to predict metastasis to the liver or lymph nodes. These models predicted metastasis to liver and lymph nodes with sensitivity of 66.7%, specificity 97.1% and an area under the curve (AUC) of 0.95[53,54]. Four studies developed models predicting survival in GC or stratification of risk of developing GC[55,56]. While 1 model based on SVM achieved higher accuracy than the tumor, nodes, metastasis cancer staging (TNM) system to predict overall survival and disease-free survival[57], another model based on a preoperative ANN was not superior in predicting survival compared to TNM[58]. Zhu et al[59] developed a ResNet50-based computer-aided detection and diagnosis (CAD) model to predict GC invasion depth, based on endoscopic images. The model achieved an accuracy of 89.16% in identifying GC invasion depth, compared to 71.49% in endoscopists.

Non-malignant conditions

Currently, diagnosis of Helicobacter pylori (H. pylori) infection (a known risk factor for the development of peptic ulcer disease and GC) relies on stool or breath testing and histopathology (invasive and expensive). Endoscopic identification of H. pylori has been a target for AI-assisted systems[60,61]. For H. pylori detection and classification, 4 of 4 studies used a CNN model[62-65]. A study by Martin et al[62] using gastric biopsy histopathology images as an input with small number of samples (n = 210 in training dataset and 90-106 in 2 test datasets) had diagnostic accuracy of 98.9%-99.1% for detecting current H. pylori infection. This result was higher than the accuracy achieved in 2 studies using esophagogastroduodenoscopy images as an input with higher number of samples (98.9%-99.1% vs 77.5%-87.7%). The mean accuracy of endoscopists for detecting currently infected H. pylori in 2 studies was around 79.0%-79.4%[62]. A study by Shichijo et al[64] found that the average H. pylori diagnostic time for the AI model was 194 s, while it was 230 ± 65 min for endoscopists. The AI model also had a significantly higher accuracy than the endoscopists by 5.3%, demonstrating that AI algorithms had significantly better accuracy and faster diagnostic time for H. pylori than the endoscopists. More recently, Nakashima et al[63] developed a CAD system based on linked color imaging combined with DL, which achieved 82.5% accuracy, demonstrating comparable diagnostic accuracy of H. pylori to that of experienced endoscopists.

In the area of gastrointestinal bleeding (GIB), risk stratification is of paramount importance not only to identify high-risk patients and guide clinical decision-making but to also identify areas where relatively scarce resources require allocation. In general, risk-stratification tools examine risk factors that are associated with a condition to predict one or several outcomes (e.g., survival, length of hospitalization, rebleeding rates, need for endoscopic therapy, response to treatment)[66-68]. Regarding upper GIB, studies focused on developing prognostic models that predict rebleeding or the need for endoscopic/surgical therapy[69].

Seven studies examining outcome measures in GIB were included, all of which were developed by reviewing medical record parameters to construct ML algorithms, which were primarily based on ANN[67-73]. Half of the studies had both internal and external validation cohorts, and patient sample sizes varied from 147 to over 22800. Shung et al[69] developed a gradient-boosting ML model that identified patients with upper GIB who met a composite endpoint with superior AUC, sensitivity and specificity to that of the Glasgow-Blatchford (GBS), Rockall and AIMS-65 scores. Seo et al[68] developed 4 different ML algorithms in patients with nonvariceal upper GIB that achieved superior AUC for mortality, rebleeding and hypotension than that of the GBS, particularly with the use of the random forest (RF) analytic model. However, for rebleeding prediction, a gradient-boosting model developed by Ayaru et al[73] showed a higher accuracy than the VC model in the aforementioned study (88% vs 78.5%). Das et al[70] developed a predictive ML model in lower GIB and compared it in internal and external validation cohorts to the validated BLEED score. Both internal and external validation cohorts reached accuracies, sensitivities, and specificities that were superior to the BLEED score.

LOWER GASTROINTESTINAL TRACT

Direct endoscopic visualization of the colon continues to be the gold standard for detection and resection of colonic premalignant lesions. In addition, patient- and polyp-specific characteristics have also been identified as predictors of missed colorectal cancer (CRC)[74]. Therefore, quality measures such as cecal intubation time, withdrawal time, bowel preparation quality and adenoma detection rates (ADR) have been instituted to attempt and standardize practice and mitigate the rates of missed CRC[74,75]. Since colonoscopy is an operator-dependent procedure, this may lead to variability in polyp detection rates, polyp characterization and estimation of depth of invasion of malignant lesions between endoscopists. CAD has been studied to address these current shortcomings, as well as to improve ADR and reduce CRC risk and colon cancer-related deaths[76] (Supplementary Table 2).

Detection of premalignant and malignant lesions

Polyp detection during colonoscopy is the cornerstone of CRC prevention. It is estimated that CRC develops in 2%-6% of cases after colonoscopy but before the next scheduled surveillance and, therefore, could represent missed or new postcolonoscopy CRC[74,77]. A large number of research studies have been devoted to identifying factors that play a role in the detection of premalignant lesions during colonoscopy, as well as factors that are associated with missed CRC[77]. An observation that has gained considerable attention is that the presence of experienced endoscopy nurses, fellows or any trained second observer during the procedure itself improves ADR[78,79]. Therefore, and answering the call for improved polyp detection measures during colonoscopy, AI has become a promising technology serving the role of a more standardized “second observer,” appropriately termed “computer-aided detection” (CADe).

Eighteen studies that evaluated polyp detection were included in this review, thirteen of which employed DL models, while five used other ML models. Nearly all the studies were limited to having internal validation only. The training datasets in these studies utilized still images from colonoscopy videos ranging from 176 to more than 8600 images. Seven studies calculated the ADR of the developed model and compared it to that of endoscopists, most of which found that the AI had significantly higher ADR than endoscopists, with a decreased reaction time, while the remaining studies showed equivalent ADR[80-87]. Repici et al[82] conducted a multicenter study showing that the ADR was 40.4% in the study’s participating endoscopists alone, while it was 54.8% in the group using the CADe-based GI-Genius (^®Medtronic, Minneapolis, MN) module. Similarly, Wang et al[85] conducted a study that evaluated over 520 colonoscopies, also examining the use of a CADe model, and found that the ADR increased from 20.34% in endoscopists alone to 29.12% in the CADe-assisted group. Other studies explored the accuracy, sensitivity and specificity of polyp-detection AI models, displaying accuracies of up to 96.4%, sensitivity of up to 99.7% and specificity of up to 93.7%[88-98]. Kominami et al[93] designed an SVM-based model using real-time images with NBI and magnification enhancement to detect colon polyps, achieving an accuracy, sensitivity, and specificity of 93.2%, 93.0%, and 93.3%, respectively. Three studies addressed subjects in the realm of CRC, specifically detecting malignancy on colonoscopy with chromoendoscopy and NBI enhancement, hematoxylin-eosin histopathology slides, or estimating invasion depth on regular WLI colonoscopy images[99-101]. Ito et al[100] developed a DL model based on CNN to detect deeply invasive CRC based on WLI colonoscopy, which utilized over 9900 images from 41 patients to train the model. Furthermore, over 5000 images were used in the testing cohort, achieving an accuracy, sensitivity and specificity in differentiating invasion depth of 81.2%, 67.5%, and 89.0%, respectively. Kudo et al[101] developed a model based on EndoBRAIN (^®Cybernet Systems Co., Tokyo, Japan) using colonoscopy with chromoendoscopy and NBI enhancement to detect malignant lesions in the colon. The training sample used over 69000 images and achieved accuracies of 98.0% and 96.0% for chromoendoscopy and NBI enhancement, respectively. These diagnostic parameters were higher than those of expert and nonexpert endoscopists.

In an attempt to morphologically classify polyps according to their malignant potential, numerous classification systems have been developed. As an example, the Paris classification classifies polyps according to whether they are pedunculated, sessile, slightly raised, or excavated[102]. These visual characteristics are then used to predict whether the polyp has invasive potential or lymph node involvement and potentially contribute to clinical decision-making, such as determining whether the polyp is resectable endoscopically[103]. Unfortunately, research has shown that interobserver variability with this classification is moderately high, suggesting that this visual classification should not be routinely used in research or practice[104]. In addition, considering that most resected polyps during colonoscopy are diminutive and that histopathologic analysis of resected polyps is costly, optical diagnosis has been proposed in this subset of polyps and termed “optical biopsy”[105]. The concept of optical biopsy has been proposed to support cost-effective strategies in CRC screening such as resecting-and-discarding, as well as diagnosing-and-leaving this subset of polyps[106]. However, the concept of optical biopsy has the same limitations as endoscopic polyp characterization does: relatively high interobserver variability. Hence, computer-aided diagnosis has been proposed as a potential solution to standardization of interpretation of endoscopic images.

Nine studies pertaining to colonic polyp classification or differentiation were included in this review[107-115]. While some studies did not report on validation methods, others had cross-validation, internal, and external methods. Most studies relied on ML-based algorithms to assess colonoscopy with enhancement measures (e.g., magnification, NBI, endocytoscopy), while deep CNN models were used in studies evaluating real-time polyp differentiation. Sánchez-Montes et al[114] developed an SVM-based model to predict polyp histologic classification using high-definition WLI from 225 colonoscopies, achieving accuracy, sensitivity, and specificity of 91.1%, 92.3%, and 89.2%, respectively. Additionally, Misawa et al[111] evaluated the performance of EndoBRAIN (^®Cybernet Systems Co., Tokyo) analyzing endocytoscopic images obtained during colonoscopy to characterize polyps as neoplastic vs non-neoplastic with an accuracy, sensitivity, and specificity of 96.9%, 97.6%, and 95.8%, respectively. These findings hold promise in determining screening and surveillance schedules, and in instituting cost-effective strategies such as “resect-and-leave”.

Knowledge on the depth of invasion of early CRC is critical for determining resection modality. Intramucosal and submucosal cancerous lesions can be resected with endoscopic techniques such as endoscopic mucosal resection or endoscopic submucosal dissection[116]. However, lesions that invade the deeper layers have a higher association with lymph node metastasis and, hence, may require a combined surgical and oncologic approach[116,117]. Several features on direct endoscopic visualization have been associated with deep invasion of a lesion, including lesion depression, fold convergence, and significantly irregular, and heterogeneous surface capillary pattern[118]. As with any image-based field, a significant degree of interobserver variability exists in characterizing lesions and determining their depth or invasion, for which CAD represents an attractive option to standardize the approach of estimating lesion invasion depth.

Finally, 6 studies assessed other subjects within the realm of the lower gastrointestinal tract, including developing models that predict outcomes in patients with CRC, models examining quality of colonoscopy performance, and studies estimating cost reductions of the use of AI[119-122]. Of 4 studies examining prediction of outcomes, 2 were based on DL (i.e., CNN-based) models, using a wide numerical range of histopathologic images: from 300 million to 12 million. Skrede et al[123] developed a DL model using over 12 million histopathological images to develop a biomarker for automatic prediction of cancer-specific survival, which outperformed existing markers. The authors conclude this strategy can potentially be utilized in the treatment selection process by identifying high-risk groups. Thakkar et al[120] recently developed a DL model aimed at automatically quantifying quality metrics during colonoscopy, providing intra-procedural feedback to the performing endoscopist.

Non-malignant conditions

Video capsule endoscopy: Owing to its ease of use and noninvasiveness, video capsule endoscopy (VCE) is the current diagnostic method of choice to assess the small bowel for conditions such as occult GIB and Crohn’s disease (CD). However, since it lacks self-locomotion, it relies on the motility of the bowel to advance within the gastrointestinal tract. Therefore, it requires a very large number of images (up to 60000 per examination) to be automatically obtained, rendering the process of interpretation lengthy and tedious[124]. VCE has been an area of focus for AI since its early stages, seeing the initial AI classifying methods such as SVM and multilayer perceptron network[125-127]. With faster processors, increased computational power by improved graphics processing units and evolving CNNs, DL algorithms have become the modality of choice for image analysis in VCE[128,129]. As in the lower gastrointestinal tract, the same principles of CADe and computer-aided diagnosis could be applied to deep balloon-assisted or motorized enteroscopy systems (Supplementary Table 3).

A total of 31 studies pertaining to VCE were identified. Fifteen studies (48%) developed models to assist in detection of active GIB or angioectasia. Four studies used CNN, five used SVM, and the remaining ones used ML-based analytic models. Five studies reported cross-validation methods. All studies reported accuracy ranging from 94% to 98% in identifying GIB and angioectasias, with sensitivity of 92.0%-100% and specificity of 82.9%-99.9%[130-142]. Specifically, Tsuboi et al[140] developed a DL model based on CNN for angioectasia detection, which achieved an AUC of 0.998, with a sensitivity and specificity of 98.8% and 98.4%, respectively.

Six studies assessed for the presence of small-intestinal ulcers, of which five used DL models based on CNN and one used SVM[128,134,143-146]. Aoki et al[128,143] conducted 2 studies in testing DL models to detect ulcerations, yielding a sensitivity of 88.2%, a specificity of 90.9%, and an AUC of 0.958, respectively. Two more studies evaluated the detection of small-bowel CD by identifying ulcers on VCE[147,148]. Klang et al[147] developed a DL model based on CNN that used over 17000 VCE images to identify CD ulcers, achieving an AUC of 0.99. Two studies examined small intestinal polyp detection using NN-based models, both achieving an accuracy of at least 98%, with sensitivity and specificity of up to 95.5% and 98.5%, respectively[149,150].

Regarding celiac disease (CeD) detection, 4 studies were evaluated, 3 of which used CNN and 1 that used a clinical decision support system[151]. A study by Zhou et al[152] reported 100% sensitivity and specificity in detecting CeD using GoogLeNet as classifier. A study by Wimmer et al[153] using AlexNet, VGGf net, and VGG-16 net as classifiers obtained an optimal accuracy of 92.5% using VGG-16, although the report did not include sample size. More recently, Wang et al[154] developed a DL model using the CNN-based InceptionV3 as well as the SVM-based ResNet50 that was 95.94% accurate, 97.20% sensitive and 95.63% specific in identifying CeD in VCE images.

Hookworm infection represents a significant healthcare issue in the developing world, with an estimated 600 million people harboring this infection[155]. As this helminth typically dwells in the small bowel, it may occasionally be a finding in VCE. Three studies evaluated hookworm detection, of which two used CNN and one used SVM with all using cross-validation[156-158]. Specifically, He et al[157] developed a DL model based on deep CNN, which was trained on a large VCE dataset consisting of more than 440000 images. The model outperformed other handcrafted, feature-based methods, reaching an accuracy of 88.5% and a sensitivity of 84.6% in detecting hookworm in the small bowel.

To assess for quality of bowel preparation during VCE Leenhardt et al[159] and Noorda et al[160] developed a DL models based on CNN. Both models achieved accuracy of over 95%, with sensitivity and specificity of 94.7%-96.18% and 94.0%-94.33%, respectively. Overall, Ding et al[131] developed a DL model based on CNN to identify and categorize all small-bowel ulcers, polyps, bleeding, lymphangiectasia, follicular hyperplasia, protruding lesions, diverticula or inflammation using over 100 million VCE images. The model achieved an impressive 99.90% overall sensitivity and 99.88% specificity, whereas gastroenterologists identified lesions with a 74.57% sensitivity in the per-patient analysis and 76.89% sensitivity in the per-lesion analysis. Furthermore, the reading time per patient was 5.9 min by the CNN model, compared to 96.6 min by conventional reading.

Inflammatory bowel disease: The complex interplay of pathophysiological factors in inflammatory bowel disease (IBD) has led to the realization that differences in patients’ biology may confer differences in disease activity and response to therapy- an example of “precision medicine”[161]. Integration of AI algorithms into the IBD realm brings promise not only in diagnosis or reducing interobserver variability in severity grading but also opens up the possibility of analyzing large databases to identify complex or occult patterns of disease[162-167].

A total of 25 studies pertaining to IBD were included in this review, of which 9 dealt with CD, 6 studied ulcerative colitis (UC,) and 10 investigated both. Five studies aimed to detect CD, UC, or IBD in general from VCE images, endoscopic images, histology, magnetic resonance imaging (MRI) images, and genetics. Most of the studies used cross-validated ML methods, of which SVM was the method of choice; while another study used DL-based CNN. These studies displayed high levels of accuracy in detecting IBD, from 83.3%-90.8%[143,168-171]. Aoki et al[143] developed a DL model based on CNN that used more than 10000 VCE images in the validation dataset to detect CD ulcers in the small bowel, achieving an accuracy, sensitivity and specificity of 90.8%, 88.2% and 90.9%, respectively.

Nine studies that addressed predicting disease severity were primarily based on endoscopic imaging; although other studies also used laboratory studies, demographics, histopathology and CT enterography[172-179]. Five studies utilized ML-based analytic models, while the remaining four utilized DL-based models, of which CNNs were the preferred ones. Five studies reported internal validation, and two studies had external validation cohorts; while two studies did not report validation. The size of patient cohorts varied greatly, from 87 to over 3000, with studies reporting up to 40000 images in the validation datasets. Yao et al[179] designed a DL model based on CNN to grade the severity of UC using colonoscopy images, which was constructed with over 16000 images from 3000 patients. The reported accuracy, sensitivity, and specificity was 87.6%, 90.2%, and 87.0%, respectively. The authors concluded that these results support the use of AI in UC severity grading , which approximates the scoring of experienced human reviewers. Maeda et al[172] designed a ML model with CAD to detect histologic inflammation based on colonoscopy images enhanced with endocytoscopy. The model achieved an accuracy, sensitivity and specificity of 90.0%, 74.0%, and 91.0%, respectively, using over 9900 images from 100 patients in the validation cohort.

To predict therapeutic response in patients with IBD treated with thiopurines or biologics seven studies constructed internally validated ML algorithms, based on RF analytic models. All models achieved an accuracy of 80.0%-89.8% or an AUC of 0.73-0.846 in identifying patients who will respond at 6-8 wk to therapeutic regimens[180-186]. Waljee et al[186] designed a model using only demographic and laboratory data from 401 patients to predict response at 8 wk in patients receiving ustekinumab with an AUC of 0.78, potentially laying the foundation to avoid costly therapeutic drug monitoring. Waljee et al[184] also constructed a model based on laboratory values and demographics with a validation dataset of over 6100 patients to predict IBD-related hospitalization and outpatient steroid use, achieving an AUC of 0.87, suggesting such AI models could be used to identify patients at risk of an IBD flare and enable precision medicine-based therapeutic approaches (Supplementary Table 4).

A further 2 internally validated ML models using SVM and RF were developed to predict the risk of IBD based on a genomic datasets[187,188]. Isakov et al[187] developed a gene prioritization model using 4 combined analytic models (RF, SVM GB, and an elastic net regularized generalized linear model) to identify genes related to IBD, achieving an accuracy of 80.8%. An internally validated natural language processing (NLP) model by Hou et al[189] used histopathology reports to identify surveillance vs nonsurveillance colonoscopy in patients with IBD, with an accuracy of 80.0%. Lastly, Firouzi et al[190] developed an internally validated model based on Waikato Environment for Knowledge Analysis that identified, with an accuracy of up to 89.8%, patients with IBD who required a bone mineral density scan, using electronic health record data.

HEPATOBILIARY SYSTEM

Liver diseases are broad and complex, ranging from asymptomatic liver chemistry elevation to life-threatening conditions such as acute liver failure or orthotopic liver transplantation (OLT). Hepatology is can be a fertile ground for applying AI in survival models (e.g., model for end-stage liver disease), disease detection models [e.g., early detection of non-alcoholic fatty liver disease (NAFLD)], disease severity models (e.g., alcoholic hepatitis discriminant function), or disease estimation models (e.g., aspartate aminotransferase-to-platelet ratio index), but also for pattern recognition in radiological images, histopathology and even selection of LT candidates[191,192]. A total of eighty-five studies pertaining to hepatology were examined, out of which twenty assessed prediction of outcome measures, forty-one examined prediction or detection of steatosis, fibrosis or cirrhosis, nine studies examined differentiation of malignant liver neoplasms, six studies explored predictive models for portal hypertension, and eight studies investigated AI models for other purposes.

Nine studies developed models that assisted with detection or classification of hepatobiliary neoplastic lesion, six of which involved DL-based CNN[193-202]. Schmauch et al[197] constructed an internally validated CNN-based DL model using ultrasonographic images of the liver to detect and classify focal liver lesions, achieving an overall AUC of 0.891. Six studies constructed models assisting in predicting the presence of portal hypertension complications in patients with all-cause cirrhosis, based on clinical data or radiological images obtained from modalities such as CT scans[203-208]. Dong et al[204] constructed an ML-based model on an RF analytic model that used clinical data and predicted the presence of esophageal varices in patients with cirrhosis with an AUC of 0.82, potentially being useful in performing a better triage of patients who actually require an upper endoscopy for variceal screening. Liu et al[206] developed an ML model that had a higher diagnostic performance than conventional noninvasive tools (either conventional image-based or serum-based tools) in identifying clinically significant portal hypertension from contrast-enhanced CT or MRI, with an accuracy of 91.1% and 88.9%, respectively.

Forty-two studies developed models addressing detection of steatosis, fibrosis, or cirrhosis based on clinical data, shear-wave elastography, CT or MRI scans, histopathology, or genetics. Of these, 27 studies developed models to detect, quantify, or predict steatosis, fibrosis, or cirrhosis[202,209-234]. Forlano et al[215] developed a ML-based model for quantification of steatosis, inflammation, ballooning, and fibrosis using biopsies from patients with NAFLD. The model identified characteristics of NAFLD with intra- and inter-observer agreement from 0.95-0.99, and has a potential use in objective assessment of treatment response in patients with NAFLD. Yasaka et al[202] constructed a DCCN model based on over 144000 MRI images from 534 patients to stage hepatic fibrosis, achieving AUCs of 0.84, 0.84, and 0.85 for F4, F3, and F2 fibrosis, respectively. In assessing fibrosis in patients with hepatitis B virus, Wang et al[232] created a DL-based CNN model that used ultrasonographic and elastography data from 132 patients to predict fibrosis, with AUCs of 0.97, 0.98 and 0.82 for F4, F3 and F2 fibrosis, respectively. Eight studies developed AI models to establish the diagnosis of fatty liver disease or distinguish between the causes of liver disease[235-242]. In distinguishing NAFLD from non-alcoholic steatohepatitis (NASH), Fialoke et al[237] constructed an ML model testing decision tree, linear regression,, RF, and extreme gradient boosting (XGB) analytic models, of which the XGB achieved the highest accuracy, AUC, sensitivity, and specificity at 79.7%. 0.876, 77.4%, and 80.8%, respectively. Taylor-Weiner et al[239] constructed an ML model that enabled quantitative measurement of liver histology and disease monitoring in NASH, characterizing disease severity, heterogeneity and treatment response in NASH.

Of 20 studies examining outcome predictors, 8 evaluated predictor models in OLT involving donor-recipient matching, recipient survival at determined time frame, graft survival at different time frames, survival predictors, and morbidity predictors[243-250]. Bertsimas et al[245] constructed an internally-validated decision tree-based ML model using clinical data that predicted 3-mo waitlist mortality or removal with an AUC of 0.895. Five studies designed models for outcome prediction in patients with hepatocellular carcinoma (HCC), including response to trans-arterial chemoembolization (TACE), recurrence or survival after resection, most of which were based on CNN[251-255]. Saillard et al[254] designed an externally-validated model based on a pre-trained CNN based on histology slides from 328 patients that independently predicted survival after HCC resection with a c-index of 0.75. Other studies evaluated outcomes in patients with acetaminophen-related ALF, primary sclerosing cholangitis, predicted the presence of choledocholithiasis, predicted hepatotoxicity of stereotactic body radiation, or mortality in patients with cirrhosis[256-263]. Eaton et al[256] constructed a ML model with XGB that accurately predicted hepatic decompensation in patients with primary sclerosing cholangitis with a C-statistic of 0.90.

Other studies focused on developing models for miscellaneous topics such as estimating liver stiffness from MRI, assessing pretransplant cognitive impairment, detecting spectral differences between normal and hepatitis B virus serum samples, classifying seroconversion to HBeAg, predicting hepatotoxicity in early stages of drug development, predicting fibrosis in hepatitis C virus, or AI-assisted liver tumor segmentation[264-272]. Williams et al[271] developed a ML model based on a Bayesian network using hepatic safety assays to predict drug-induced liver injury in compounds during drug development, achieving an accuracy, sensitivity, and specificity of 86.0%, 87.0%, and 85.0%, respectively. An externally-validated ML model based on SVM and radiomics by He et al[267] used MRI and clinical data to estimate liver stiffness, achieving an AUC of 0.80, with an accuracy, sensitivity, and specificity of 75.0%, 63.6%, and 82.4%, respectively. Lastly, models addressing prediction of NAFLD or NASH based on clinical or genetic data, as well as models analyzing donor liver texture for steatosis, have also been developed[272-278] (Supplementary Table 5).

PANCREATIC DISEASES

Pancreatic diseases contain areas where AI can be effectively applied. Of primary interest is the use of AI in improving existing disease severity scoring systems or prognostic models in complicated acute pancreatitis (AP) or chronic pancreatitis (CP), based on clinical and radiological data, detection and differentiation of pancreas cystic neoplasms (PCN) with prediction of malignant potential, radiologic early detection of pancreatic ductal adenocarcinoma (PDAC), radiologic differentiation between PDAC and benign pancreatic conditions [e.g., autoimmune pancreatitis (AIP)], and histopathologic interpretation of tissue samples[279-282]. A total of 59 studies pertaining to the pancreas were reviewed. Of these, 20 (34%) addressed prediction of outcomes in patients with pancreatic diseases ranging from AP to neoplasia.

Eleven of these studies examined outcome prediction in AP. All of the studies’ ANN models outperformed logistic regression models, Glasgow, and APACHE-II scoring systems in predicting AP severity; while requiring less number of parameters[283-293]. Qiu et al[292] compared the performance of SVM, logistic regression analysis, and ANN models to predict multiorgan failure in AP. All 3 models predicted multiorgan failure, with AUC of 0.840, 0.832, and 0.834, respectively, with ANN requiring a lesser number of parameters.

Seven (12%) studies used AI algorithms to construct clinical registries, segment the pancreas based on imaging, or differentiate between certain pancreatic diseases based on cross-sectional imaging or endoscopic ultrasound (EUS)[294-300]. Zhang et al[298] constructed a DL station classification model and a segmentation model to reduce the difficulty in EUS interpretation for trainees. The trainee station recognition accuracy improved from 67.2% to 78.4% in the crossover study. Interobserver agreement between endoscopists and deep CNN with Cohen’s kappa coefficient was substantial, ranging from 0.826-0.879. The authors conclude that this technology may play a key role in shortening the learning curve of EUS among trainees.

Fifteen studies (25%) addressed prediction of malignancy based on imaging findings, or differentiation of benign from malignant pancreatic conditions[280,301-315]. Marya et al[280] developed an EUS-based CNN model that distinguished AIP from normal pancreas with 99% sensitivity and 98% specificity, AIP from CP with 94% sensitivity and 71% specificity, and AIP from PDAC with 90% sensitivity and 93% specificity. Chu et al[301] conducted a study utilizing CT radiomics features to differentiate PDAC from normal pancreas tissue. The accuracy of the RF binary classification was 99.2%, with an AUC of 99.9%. All cases of PDAC were correctly identified, with a sensitivity of 100% and specificity of 98.5%.

Eleven studies (19%) evaluated differentiation of PCNs by classifying them into their respective subtypes based on their characteristics on imaging[316-326]. Springer et al[324] developed a multimodality ML model that integrated clinical, radiological and genetic/biochemical markers data to determine whether patients with pancreas cyst should undergo surgery, monitoring, or no further surveillance. The model correctly identified serous cystic neoplasms in 65% of the cases with 99% specificity, clearly outperforming the current standard of care of clinical identification in only 18% of cases. The authors conclude that these systems may serve an adjunct role in clinical practice, enabling the clinician to take better-informed clinical decisions[324].

Eight studies addressed PDAC, from developing risk scores for development of PDAC based on urinary biomarkers, predicting clinical performance and response to celiac plexus neurolysis, to prediction of survival time. AI models performed at least as well as the logistic regression models in predicting the selected outcome[327-334].

Six studies directly examined early pancreatic cancer detection in PDAC or PCN by examining the imaging characteristics or identifying high-risk patients on electronic health records based on factors such as family history of pancreatic cancer[335-340]. Roch et al[339] developed NLP-based algorithms based on common terminology used by physicians in describing pancreatic cysts and applied them to automatically conduct searches in electronic health records. The algorithm tracked patients with cysts with a 99.9% sensitivity and 98.8% specificity, demonstrating its utility in capturing patients swiftly and with more ease than manual review. Ozkan et al[338] developed a CAD image-processing system using EUS images to diagnose PDAC, taking patient age into consideration. The accuracy of the model was 87.5, with sensitivity and specificity of 83.3% and 93.3%, respectively (Supplementary Table 6).

CURRENT LIMITATIONS AND KNOWLEDGE GAPS

Despite the numerous positive advances in AI, there remain several limitations to current studies and obstacles to overcome for future studies. Most current models are based on labeled data and, hence, interpretation is only as good as the observer who labeled the “gold-standard” data. Current algorithms are specifically fitted for a determined dataset. A sizeable proportion of the AI models applied to the clinical setting are only internally validated. Ideally, models should be externally validated on diverse cohorts to ensure that overfitting does not become an issue. Therefore, this issue could be potentially addressed by the creation of a universal, well-annotated, high-quality dataset, and by creating algorithms with more plasticity. However, creating “universal datasets” creates additional challenges, particularly related to data integrity and privacy. A potential solution to this is the decentralized “federated datasets”, which involves combining multiple datasets stored on their respective servers, addressing these challenges[341]. Specific protocols are required for choosing an analytic model and selecting or developing validation techniques (e.g., external-, internal-, cross-validation) for data fine-tuning or augmentation. Algorithms that yield the best accuracy should be promoted. Calibration has translated into improvements in probability prediction, for which they should be instituted in all models. Most current studies were cohort studies, whereas well-designed randomized controlled trials would be needed to better support conclusions. Some studies utilized custom-built models, which are not explained in detail. Therefore, custom-built algorithms should have their background and processes thoroughly declared. The studies presented numerous different newly developed models, which would require validation to determine whether these can be applied to other datasets. Several studies examine different techniques during an equivalent endoscopic procedure (e.g., endoscopic image processing in NBI, WLI, or chromoendoscopy), which renders comparisons between techniques cumbersome or not possible. Data matrixes should be completely reported[342]. Significant efforts have been devoted to developing guidelines, such as the CONSORT-AI extension, to standardize reporting in trials evaluating performance of the AI. Adherence to current guidelines and flexibility to revise them as technology continues to advance is of paramount importance. The majority of studies use still images and high-definition images, which is not in line with the “real-world experience” of real-time settings, imaging affected by motion artifact or poor image processing from outdated technical equipment.

APPLICATION IN CLINICAL CARE (ARTICLE HIGHLIGHTS)

ML and DL could assist clinicians in the diagnosis of gastrointestinal and liver neoplasms, bleeding, infection, and inflammatory process, and also predict outcome measures in these conditions.

The initial use of ML or DL models might be used in backing up clinicians in establishing diagnoses or determining a treatment plan.

Given its high predictive value, if AI suggestions match the clinician’s reasoning, clinicians could make a decision more confidently. If the answers are discrepant, careful investigation should be undertaken.

In the future, if ML or DL models find a place to be integrated in standard clinical care, to guide in establishing diagnoses, selecting treatment interventions, predicting outcomes, and influencing clinical decision-making. However, future studies are also necessary to explore avenues of how these measures can be better instituted in clinical practice as a whole.

FUTURE DIRECTIONS

As demonstrated by this review, AI applications in clinical gastroenterology and hepatology continue to rapidly expand and evolve at many different levels. For general clinical care, the recent proliferation in AI applications is likely to enable “precision medicine” on a broader scale. Clinically, it is predicted that invasive diagnostic interventions will generally fall out of favor for some conditions, as better noninvasive ML-based algorithms pave the way for improved clinical prediction models. Some diagnostic interventions, such as VCE interpretation, may see a considerable decrease in human interpretation, minimizing the human role to that of supervision and attestation of findings of the model. AI-assisted technology will prove important in real-time clinical settings (e.g., polyp detection during colonoscopy). Integration of monitoring devices (e.g., smartphones, smart watches) with ML in the management of selected diseases is also predicted to significantly receive more attention the coming years. The creation of a universal, large, high-quality, well-labelled dataset is a necessity, from which algorithms could be developed to better define the epidemiology and risk factors of diseases. Well-harnessed AI assistance should decrease physician workload or at least maximize their productivity by allowing them to shift from menial tasks to faster, more accurate clinical decision-making. ML algorithms based on these datasets can also be used for other quality measures, such as improvement of process efficiency or identifying cost-effective interventions. In terms of data analysis, traditional analytic models (e.g., logistic regression and clinical scoring systems) may be substituted or augmented by ML algorithms to achieve greater capability and accuracy. Developing and maintaining multidisciplinary teams of data scientists, physicians, content subject experts and industry is of paramount importance in the advancement of AI in gastroenterology and hepatology. Finally, educating clinicians and patients in the future paths of AI applications is critical to increase understanding of future value and decrease reluctance in engagement.

CONCLUSION

The latest advances in AI in gastroenterology and hepatology are promising for aspect many fields of clinical care, from detection of neoplastic lesions on endoscopic assessment and improving current survival models to predicting treatment response. The application of AI to large and complex datasets may assist in the identification of new associations between variables, potentially leading to changes in clinical practice. Furthermore, the use of AI-assisted technologies has the potential to dramatically improve the quality of care. Finally, the time for assisted precision medicine is at hand, with the AI being able to tailor a treatment regimen or potentially predict the response to treatment in a specific patient based on extensive amounts of clinical data from large patient datasets. It is important to realize that, while AI currently does not substitute human clinical reasoning, it has a bright future in the betterment of patient care.

ACKNOWLEDGEMENTS

The authors are grateful to for Wang Z, PhD, Zhang HJ, PhD, Sun T, MD, PhD, Hassan Virk H, MD, and Aiumtrakul N, MD assistance with the additional literature search.