Long-term impact of artificial intelligence on colorectal adenoma detection in high-risk colonoscopy

Abstract

BACKGROUND

Improved adenoma detection rate (ADR) has been demonstrated with artificial intelligence (AI)-assisted colonoscopy. However, data on the real-world application of AI and its effect on colorectal cancer (CRC) screening outcomes is limited.

AIM

To analyze the long-term impact of AI on a diverse at-risk patient population undergoing diagnostic colonoscopy for positive CRC screening tests or symptoms.

METHODS

AI software (GI Genius, Medtronic) was implemented into the standard procedure protocol in November 2022. Data was collected on patient demographics, procedure indication, polyp size, location, and pathology. CRC screening outcomes were evaluated before and at different intervals after AI introduction with one year of follow-up.

RESULTS

We evaluated 1008 colonoscopies (278 pre-AI, 255 early post-AI, 285 established post-AI, and 190 late post-AI). The ADR was 38.1% pre-AI, 42.0% early post-AI (P = 0.77), 40.0% established post-AI (P = 0.44), and 39.5% late post-AI (P = 0.77). There were no significant differences in polyp detection rate (PDR, baseline 59.7%), advanced ADR (baseline 16.2%), and non-neoplastic PDR (baseline 30.0%) before and after AI introduction.

CONCLUSION

In patients with an increased pre-test probability of having an abnormal colonoscopy, the current generation of AI did not yield enhanced CRC screening metrics over high-quality colonoscopy. Although the potential of AI in colonoscopy is undisputed, current AI technology may not universally elevate screening metrics across all situations and patient populations. Future studies that analyze different AI systems across various patient populations are needed to determine the most effective role of AI in optimizing CRC screening in clinical practice.

Key Words: Artificial intelligence, Colonoscopy, Adenoma detection rate, Screening, Colorectal adenoma

Core Tip: This study analyzed the long-term impact of artificial intelligence (AI)-assisted colonoscopy in a diverse at-risk patient population undergoing diagnostic colonoscopy for positive colorectal cancer (CRC) screening tests or symptoms. It was found that in patients with an increased pre-test probability of having an abnormal colonoscopy, the current generation of AI did not yield enhanced screening metrics over high-quality colonoscopy. Future studies that analyze different AI systems across various patient populations are needed to determine the most effective role of AI in optimizing CRC screening in clinical practice.

INTRODUCTION

Endoscopic procedures are indispensable to the effective diagnosis and treatment of gastrointestinal (GI) diseases. Colonoscopy is the gold-standard for colorectal cancer (CRC) screening and is associated with approximately 52% reduction in CRC incidence and 62% reduction in CRC-related mortality[1]. Despite its effectiveness, colonoscopy remains imperfect with a reported polyp miss rate of up to 27%[2] which is the primary cause of post-colonoscopy CRC.

Adenoma detection rate (ADR) is the most validated quality indicator in colonoscopy with higher ADR’s associated with lower risk of post-colonoscopy CRC[3]. Improvement in the ADR of individual endoscopists is associated with reduced risk of interval CRC and death[4]. Recent advances in artificial intelligence (AI) have led to significant progress in real-time computer-assisted endoscopic image analysis and have been associated with increased colonoscopy ADR[5,6]. Computer-assisted detection (CAD) utilizes convolutional neural network, a deep learning AI system that can independently extract relevant features from large datasets and achieve autonomous learning[7]. The GI Genius module (Medtronic) is one such system that helps detect colorectal polyps real-time during colonoscopy. It received de novo clearance from the Food and Drug Administration (FDA) in April of 2021.

The potential of AI to facilitate increased ADR and lower rates of post-colonoscopy CRC has garnered significant interest in recent years. However, data on the real-world application of AI and its effect on long-term CRC screening outcomes are limited. In many resource-restricted safety net healthcare systems, fecal immunochemical testing (FIT) is the preferred initial screening strategy, with colonoscopies reserved for diagnostic indications such as history of colorectal adenomas, family history of CRC, anemia, GI bleeding, or positive FIT. As such, this patient population has an increased pre-test probability of an abnormal colonoscopy. We evaluated the real-world, long-term impact of AI on a diverse minority-predominant patient population undergoing diagnostic colonoscopy for positive CRC screening tests or symptoms.

MATERIALS AND METHODS

Study population and data collection

Participants in this study consisted of adult patients who underwent colonoscopy at our institution between August 1, 2022, and September 30, 2023. Exclusion criteria included history of CRC, poor bowel preparation, incomplete colonoscopy, repeat procedure to remove a previously detected lesion, and history of inflammatory bowel disease.

AI software (GI Genius, Medtronic) was incorporated into the standard procedure protocol in November of 2022. CRC screening outcomes were evaluated at distinct intervals after the implementation of AI to assess the initial learning curve and the potential over-reliance effect of AI. Procedures performed during November of 2022 were not included in our analysis to allow time for the incorporation of the AI system into the routine workflow. The study periods were defined as pre-AI (August 1, 2022 to October 31, 2022), early post-AI (December 1, 2022 to February 28, 2023), established post-AI (March 1, 2023 to April 30, 2023), and late post-AI (August 1, 2023 to September 30, 2023). All colonoscopies were performed by or under the supervision of endoscopists with significant (> 3000 colonoscopies) experience. Second look examination or retroflexion was routinely performed in the right colon.

Data was collected on patient demographics, procedure indication, polyp size, location, and pathology. Polyp detection rate (PDR), ADR, advanced ADR (AADR), and non-neoplastic PDR (NNPDR) were calculated and compared between the post-AI groups and the pre-AI historical control group. Advanced adenomas were defined as at least one of the following: size greater or equal to 10 mm, presence of villous or tubulovillous features, presence of high-grade dysplasia, or the presence of three or greater adenomas. Non-neoplastic polyps included hyperplastic polyps, lymphoid aggregates, and fragments of colonic mucosa.

Colonoscopies were stratified into risk tiers defined by the 2020 United States Multisociety Task Force (USMTF) polypectomy guidelines[8]. Tier 1 was defined as a colonoscopy without adenomas. Tier 2 was defined as 1-2 adenomas whose size were < 10 mm. Tier 3 was defined as 3-4 adenomas whose size were < 10 mm. Tier 4 was defined as 5-10 adenomas, adenoma size > 10 mm, the presence of villous or tubulovillous features, or the presence of high-grade dysplasia. Tier 5 was defined as > 10 adenomas.

Statistical analysis

Descriptive statistics were performed to summarize patient demographics and procedural indications. Continuous variables were reported as mean ± standard deviation and compared using two-tailed unpaired student t-tests. Categorical variables were reported as frequencies and percentages and compared using two-tailed chi-squared tests.

We compared the baseline PDR, ADR, AADR, and NNDPR to each of the post-AI study periods (early, established, and late post-AI) using two-tailed chi-squared tests. The distribution of colonoscopies into risk tiers as defined by the USMTF polypectomy guidelines was compared using a 5 x 5 chi-squared test.

Ethics

Patient confidentiality and data protection was ensured throughout the study. Informed consent was waived by the Institutional Review Board (IRB number: 18CR-31902-01).

RESULTS

We evaluated 1008 colonoscopies during the study period. There were 278 patients in the pre-AI group, 255 in the early post-AI group, 285 in the established post-AI group, and 190 in the late post-AI group. The average age was 56.9 years with 45.8% males. The most common ethnicity was Hispanic (66.0%), followed by African American (10.7%), Asian (9.3%), and non-Hispanic White (6.7%). The remaining 7.2% of patients had an unspecified or other ethnicity. There were no statistically significant differences in patient demographics between groups. FIT positivity was the most common procedural indication (32.6%), followed by GI bleeding (19.8%), other indication (17.6%), prior history of polyps (13.7%), anemia (12.4%), and family history of CRC (5.4%). GI bleeding was a more common procedure indication in the early post-AI group compared to the pre-AI group (27.8% vs 19.1%, P = 0.02). There were no statistically significant differences in the degree of fellow involvement during colonoscopy (Table 1).

Table 1 Patient demographics and procedure indication, n (%).

	Pre-AI (n = 278) (A)	Post-AI early (n = 255) (B)	Post-AI established (n = 285) (C)	Post-AI late (n = 190) (D)	P value (A vs B)	P value (A vs C)	P value (A vs D)
Age (mean ± SD)	56.7 ± 10.4	56.7 ± 10.8	57.2 ± 11.2	57.1 ± 11.2	0.99	0.60	0.68
Sex
Male	122 (43.9)	113 (44.3)	134 (47.0)	93 (48.9)	0.92	0.51	0.28
Female	156 (56.1)	142 (55.7)	151 (53.0)	97 (51.1)	0.92	0.51	0.28
Ethnicity
Asian	30 (10.7)	24 (9.4)	27 (9.5)	13 (6.8)	0.70	0.60	0.15
Black	27 (9.7)	26 (10.2)	36 (12.6)	19 (10.0)	0.85	0.27	0.92
Hispanic	181 (65.1)	172 (67.5)	178 (62.5)	134 (70.5)	0.57	0.51	0.22
White	21 (7.6)	16 (6.3)	19 (6.7)	12 (6.3)	0.56	0.68	0.61
Other	19 (6.8)	17 (6.7)	25 (8.8)	12 (6.3)	0.94	0.39	0.82
Insurance status
No insurance	29 (10.4)	41 (16.1)	24 (8.4)	25 (13.2)	0.05	0.41	0.36
Medicaid	186 (66.9)	169 (66.3)	188 (66.0)	135 (71.1)	0.87	0.81	0.34
Medicare	6 (2.2)	2 (0.8)	5 (1.8)	7 (3.7)	0.19	0.73	0.32
Medicaid + medicare	30 (10.8)	31 (12.2)	41 (14.4)	16 (8.4)	0.62	0.20	0.40
Private or other	27 (9.7)	10 (3.9)	27 (9.5)	7 (3.7)	0.01	0.92	0.01
BMI (mean ± SD)	30.2 ± 7.0	30.4 ± 6.7	30.5 ± 7.0	30.8 ± 7.6	0.78	0.65	0.42
Fellow involvement	181 (65.1)	167 (65.5)	170 (59.6)	110 (57.9)	0.93	0.18	0.11
Procedure indication1
Fecal immunochemical testing positive	95 (34.2)	97 (38.0)	79 (27.8)	58 (30.5)	0.35	0.10	0.41
Anemia	33 (11.9)	29 (11.4)	36 (12.6)	27 (14.2)	0.86	0.78	0.46
Gastrointestinal bleeding	53 (19.1)	71 (27.8)	45 (15.8)	31 (16.3)	0.02	0.31	0.45
History of polyps	46 (16.5)	31 (12.2)	37 (13.0)	25 (13.2)	0.15	0.23	0.32
Family history of colorectal cancer	15 (5.4)	21 (8.2)	10 (3.5)	8 (4.2)	0.19	0.28	0.56
Other	42 (15.1)	42 (16.5)	64 (22.5)	29 (15.3)	0.67	0.03	0.96

¹Some patients had multiple procedure indications.

Continuous comparisons used two-tailed unpaired student t-tests and categorical comparisons used two-tailed chi-squared tests. AI: Artificial intelligence.

In the established post-AI group, 277/373 (74.3%) of polyps were diminutive in size (less than 5 mm), compared to 236/364 (64.8%) in the pre-AI group (P = 0.005). The percentage of polyps greater than 10 mm in size was lower in the established post-AI group compared to the pre-AI group (11.0% vs 16.2%, P = 0.04). There were no significant differences in polyp size between the pre-AI group and the early and late post-AI groups (Table 2).

Table 2 Colorectal cancer screening metrics before and after the implementation of artificial intelligence, n (%).

	Pre-AI (n = 278) (A)	Post-AI early (n = 255) (B)	Post-AI established (n = 285) (C)	Post-AI late (n = 190) (D)	P value(A vs B)	P value(A vs C)	P value(A vs D)
Bowel preparation quality
Excellent	111 (39.9)	106 (41.6)	95 (33.3)	63 (33.2)	0.70	0.10	0.14
Good	124 (44.6)	108 (42.4)	121 (42.5)	76 (40.0)	0.60	0.61	0.32
Fair	40 (14.4)	36 (14.1)	61 (21.4)	43 (22.6)	0.93	0.03	0.02
Unspecified	3 (1.1)	5 (2.0)	8 (2.8)	8 (4.2)	0.40	0.32	0.03
Total number of polyps detected (polyps per colonoscopy)	364 (1.3)	419 (1.6)	373 (1.3)	235 (1.2)
Total number of adenomas detected (adenomas per colonoscopy)	211 (0.8)	253 (1.0)	227 (0.8)	138 (0.7)
Polyp size
< 5 mm	236 (64.8)	277 (66.1)	277 (74.3)	161 (68.5)	0.71	0.005	0.35
5 mm-10 mm	66 (18.1)	91 (21.7)	50 (13.4)	43 (18.3)	0.21	0.08	0.96
> 10 mm	59 (16.2)	51 (12.2)	41 (11.0)	30 (12.8)	0.10	0.04	0.25
Unspecified	3 (0.8)	0 (0.0)	5 (1.3)	1 (0.4)	0.06	0.50	0.56
Polyp detection rate	166/278 (59.7)	152/255 (59.6)	161/285 (56.5)	116/190 (61.1)	0.98	0.44	0.77
Adenoma detection rate	106/278 (38.1)	107/255 (42.0)	114/285 (40.0)	75/190 (39.5)	0.37	0.65	0.77
Advanced adenoma1 detection rate	45/278 (16.2)	44/255 (17.3)	43/285 (15.1)	33/190 (17.4)	0.74	0.72	0.74
Non-neoplastic polyp2 detection rate	83/278 (30.0)	89/255 (34.9)	77/285 (27.0)	58/190 (30.5)	0.21	0.46	0.88

¹Advanced adenomas were defined as size ≥ 10 mm, presence of villous or tubulovillous features, presence of high-grade dysplasia, or ≥ adenomas.

²Non-neoplastic polyps included hyperplastic polyps, lymphoid aggregates, and colonic mucosa.

AI: Artificial intelligence.

The overall PDR was 59.7% in the pre-AI group, compared to 59.6% (P = 0.98), 56.5% (P = 0.44), and 61.1% (P = 0.77) in the early, established, and late post-AI groups respectively. The pre-AI ADR was 38.1%, compared to the early (42.0%, P = 0.37), established (40.0%, P = 0.65), and late post-AI ADR (39.5%, P = 0.77). The pre-AI AADR was 16.2%, compared to the early (17.3%, P = 0.74), established (15.1%, P = 0.72), and late post-AI AADR (17.4%, P = 0.74). The pre-AI NNPDR was 30.0%, compared to the early (34.9%, P = 0.21), established (27.0%, P = 0.46), and late NNPDR (30.5%, P = 0.88). The average number of polyps detected per colonoscopy (PPC) and adenomas detected per colonoscopy (APC) were highest in the early post-AI group, with a PPC of 1.6 and an APC of 1.0 (Table 2 and Figure 1).

Figure 1 Polyp detection rates, adenoma detection rates, advanced adenoma detection rates, and non-neoplastic polyp detection rates before and after the implementation of artificial intelligence. AI: Artificial intelligence.

The distribution of colonoscopies into risk tiers as defined by the USMTF polypectomy guidelines was similar across time periods (P = 0.95) (Figure 2).

Figure 2 Percentage of colonoscopies stratified into risk groups as defined by the 2020 United States Multisociety Task Force polypectomy guidelines. Tier 1 was defined as no adenomas. Tier 2 was defined as 1-2 adenomas whose size were < 10 mm. Tier 3 was defined as 3-4 adenomas whose size were < 10 mm. Tier 4 was defined as 5-10 adenomas, adenoma size > 10 mm, the presence of villous or tubulovillous features, or the presence of high-grade dysplasia. Tier 5 was defined as > 10 adenomas. AI: Artificial intelligence.

DISCUSSION

The implementation of AI did not result in improvements in PDR, ADR, AADR, or NNPDR in our study population. Diminutive polyps (size < 5 mm) were more common in the established post-AI group, although this difference was not observed in the early and late post-AI groups. The PPC (1.6) and APC (1.0) were highest in the early post-AI group, though the subsequent established and late post-AI groups had a regression of PPC and APC to approximately 1.3 and 0.8 respectively. These findings suggest that in our safety-net patient population, the current generation of AI did not yield enhanced CRC screening metrics over high-quality diagnostic colonoscopy performed by experienced endoscopists. Although there were mild increases in PPC and APC in the early post-AI period, this change was not sustained.

When GI Genius was approved by the FDA in April of 2021, it was the first CAD system to be adopted in endoscopy centers nationwide. AI has the potential to revolutionize the field of colonoscopy via enhanced diagnostic accuracy and improved patient outcomes. The initial clinical trials studying and validating GI Genius were promising and demonstrated higher ADRs and detection rates of diminutive adenomas[9] particularly in endoscopists with less experience (< 2000 colonoscopies)[10]. However, not all subsequent studies have yielded positive results. In a 3-month pragmatic implementation trial within the Stanford University healthcare system, Ladabaum et al[11] found no significant effect of AI (GI Genius, Medtronic) on ADR, AADR, APC, and sessile lesion detection rates when compared to historical controls. Similarly, in a 6-month retrospective study at a large academic medical center in Israel, Levy et al^[12] reported lower PDR (36.5% vs 40.9%, P = 0.004), ADR (30.3% vs 35.2%, P < 0.001), and AADR (5.7% vs 8.6%, P = 0.01) after implementation of GI Genius. Of note, this study excluded patients who underwent colonoscopy for suspected malignancy, yielding a study population with a comparatively lower pre-test probability of having an abnormal colonoscopy compared to our study cohort.

It is essential to note that our study specifically focused on a minority-predominant patient population in a safety-net healthcare setting, which could have unique demographic and socioeconomic implications. Minority patients often face multiple barriers to care and are less likely to access primary care resources and receive specialist referrals[13]. In a retrospective study of the 2009 California Health Interview Survey, authors found that latino patients were 31% less likely to undergo colonoscopies compared to non-hispanic whites[14]. As such, patients from marginalized populations such as ours frequently present at advanced stages of the disease[15]. In addition, colonoscopies in our healthcare system are reserved for diagnostic indications for positive CRC screening tests or symptoms. Therefore, our patient population had an inherently increased pre-test probability of having an abnormal colonoscopy. Our endoscopists were not blinded to either procedural indications or patient medical history, and this bias may have led them to be more alert compared to the endoscopists performing routine screening colonoscopies in prior clinical trials. This may have contributed to baseline lower polyp miss rates and thus less potential benefit of AI assistance in our cohort.

Another explanation for the lack of improvement in CRC screening metrics after AI implementation is the potential ceiling effect in colonoscopy. Our endoscopists exhibited a relatively high baseline ADR of 38.1% prior to AI implementation which is significantly higher than the recommended benchmark ADR of 25% set forth by the American Society of Gastrointestinal Endoscopy and American College of Gastroenterology[16]. Experienced endoscopists with baseline high ADRs may not derive as noticeable benefits from AI compared to less experienced endoscopists with lower baseline ADRs. The incremental advantages that AI offers may be overshadowed by the already proficient detection rates achieved by skilled practitioners. Early studies that demonstrated higher ADRs in AI-assisted colonoscopy also reported comparatively lower ADRs in the control group ranging from 9%-28%[17-20]. Conversely, a randomized clinical trial of FIT positive patients undergoing diagnostic colonoscopy demonstrated no significant differences in ADR between the AI-assisted colonoscopy group compared to the control group [64.2% vs 62.0%, adjusted risk ratio, 1.06 (99.9%CI, 0.91-1.23)][21]. Authors attributed this lack of improvement in part to the high lesion prevalence and the high-performing endoscopists with high ADRs in the control group. Similar studies that showed no performance benefit with AI also reported relatively higher baseline ADRs of approximately 35%-40%[11,12]. Perhaps AI could confer a more noticeable improvement in CRC screening metrics in endoscopists with low baseline ADRs.

One concern that AI-assisted colonoscopy poses is the potential for increased false positives during which the AI system flags normal artifact within the colonic mucosa as a potential neoplastic lesion. Prior studies examining various CAD systems in colonoscopy have reported false positive rates of up to 60%[22]. Higher false positives can lead to unnecessary biopsies and polypectomies which increase the risk of procedural complications, lead to lengthier procedure times, and increase healthcare expenditure and use of pathology resources[23]. Frequent flagging of non-neoplastic artifact may also lead to alert fatigue and serve as a distraction to the endoscopist. Reassuringly, our study found no differences in NNPDR between groups, suggesting that AI did not lead to an increased number of unnecessary biopsies or polypectomies. In addition, there were no differences in risk tier stratification of the colonoscopies per the USMTF polypectomy guidelines, suggesting that the recommended post-colonoscopy surveillance intervals did not change after AI initiation.

Our study had several limitations. This was a single-center retrospective study in a safety net healthcare setting, and thus our study population may not be generalizable to others throughout the country. Reporting of bowel preparation quality was subjective and did not use validated reporting instruments such as the Boston Bowel Preparation Scale. Data on colonoscope withdrawal time was not obtained. Although endoscopists were instructed to perform their procedures as they normally would have, they were aware of the study and knew that their performance metrics would eventually be analyzed, which could have led to subconscious changes in behavior.

CONCLUSION

Our experience suggests that in patients with an increased pre-test probability of having an abnormal colonoscopy, the current generation of AI did not yield enhanced CRC screening metrics over high-quality colonoscopy performed by experienced endoscopists. Although previous studies have shown AI-driven improvements in performance metrics such as ADR, our findings underscore the nuanced interaction between AI technology, baseline endoscopist performance, and patient population. Current AI technology may not universally elevate screening metrics across all situations and patient populations. The potential of AI in colonoscopy is undisputed, although future studies that analyze and validate different AI systems in different patient cohorts are needed to determine the most effective role of AI in optimizing CRC screening in clinical practice.