Correlation of endoscopic disease severity with pediatric ulcerative colitis activity index score in children and young adults with ulcerative colitis

Abstract

AIM

To investigate of pediatric ulcerative colitis activity index (PUCAI) in ulcerative colitis correlate with mucosal inflammation and endoscopic assessment of disease activity (Mayo endoscopic score).

METHODS

We reviewed charts from ulcerative colitis patients who had undergone both colonoscopy over 3 years. Clinical assessment of disease severity within 35 d (either before or after) the colonoscopy were included. Patients were excluded if they had significant therapeutic interventions (such as the start of corticosteroids or immunosuppressive agents) between the colonoscopy and the clinical assessment. Mayo endoscopic score of the rectum and sigmoid were done by two gastroenterologists. Inter-observer variability in Mayo score was assessed.

RESULTS

We identified 99 patients (53% female, 74% pancolitis) that met inclusion criteria. The indications for colonoscopy included ongoing disease activity (62%), consideration of medication change (10%), assessment of medication efficacy (14%), and cancer screening (14%). Based on PUCAI scores, 33% of patients were in remission, 39% had mild disease, 23% had moderate disease, and 4% had severe disease. There was “moderate-substantial” agreement between the two reviewers in assessing rectal Mayo scores (kappa = 0.54, 95%CI: 0.41-0.68).

CONCLUSION

Endoscopic disease severity (Mayo score) assessed by reviewing photographs of pediatric colonoscopy has moderate inter-rater reliability, and agreement was less robust in assessing patients with mild disease activity. Endoscopic disease severity generally correlates with clinical disease severity as measured by PUCAI score. However, children with inflamed colons can have significant variation in their reported clinical symptoms. Thus, assessment of both clinical symptoms and endoscopic disease severity may be required in future clinical studies.

Key Words: Ulcerative colitis; Pediatric ulcerative colitis activity index; Mayo score

Core tip: There is controversy regarding what the best method of assessing disease activity in pediatric ulcerative colitis. Currently, the best accepted tool is the pediatric ulcerative colitis activity index (PUCAI), developed by Turner and colleagues, which is a physician reported measure. Because of its formal validation and ease of use, the PUCAI has been widely accepted both as a clinical tool by physicians. Other experts have suggested that biomarkers, patient reported outcomes, or endoscopic disease activity may be better measures. In this study, we show physicians looking at endoscopic photos may grade the Mayo endoscopic scores differently, and that the PUCAI generally correlates well with endoscopic disease activity.

INTRODUCTION

The development of proper measures to assess the response to drug therapy in pediatric inflammatory bowel disease is an important priority for both clinicians and regulatory agencies. Currently, there are many different disease activity indices utilized in published studies of adult inflammatory bowel disease (IBD)[1,2]. In children, Turner and colleagues developed the pediatric ulcerative colitis activity index (PUCAI) in 2007, based on input from a Delphi group of thought leaders, and development of individual criteria utilizing state of the art statistical methodology[3]. The PUCAI score is based on 6 quantifiable items (abdominal pain, stool frequency, stool consistency, amount of blood in the stool, nocturnal diarrhea, and well-being). The PUCAI has been widely used by clinicians in studies of drug efficacy, and has also been utilized in assessing the efficacy of surrogate markers[4,5]. In a recent trial of infliximab, PUCAI scores were useful in predicting the important outcome of sustained steroid-free clinical remission at 1 year[6]. Physician reported PUCAI scores also correlate well with patient self-reported PUCAI scores[7].

Recently, physicians at regulatory agencies have suggested that the PUCAI may not be an optimal measure of disease activity in clinical trials[8]. The PUCAI is a clinician reported outcome rather than patient reported outcome. In addition, the PUCAI does not contain information about mucosal inflammation or endoscopic remission (mucosal healing). There is an increasing amount of literature on the importance of mucosal healing (aka reduction in endoscopic inflammation) as an aim of treatment in ulcerative colitis[9]. Mucosal healing may change the natural history of UC and improve clinical outcomes[10,11]. Direct examination of the colonic mucosa has become the preferred method of measuring disease activity in adult UC trials[12,13]. The physician’s clinical assessment of active disease in ulcerative colitis (UC) may not correlate well with mucosal inflammation and endoscopic assessment of disease activity[14]. Often endoscopic and histologic disease activity in UC lags behind improvement in clinical symptoms[15].

The Mayo endoscopic score is a commonly utilized measure of disease severity in adult clinical trials, and has been utilized in multiple studies, including the ACT 1 and 2 trials of infliximab, and the recent vedolizumab UC trial[16,17]. However, the inter-rater reliability of the Mayo endoscopic score has not been adequately assessed in pediatrics. In addition, limited data exists comparing clinical disease activity (PUCAI) to endoscopic disease activity (Mayo score)[18].

To address these questions, we conducted a retrospective study comparing clinical disease activity to endoscopic photos in our pediatric and young adult ulcerative colitis population that has undergone endoscopic assessment unrelated to clinical trial participation. The primary aim of our study was to assess the correlation between endoscopic disease activity (Mayo score) and clinical disease activity (PUCAI) in UC. We also sought to assess if expert clinicians looking at endoscopic photos would give similar Mayo scoring of disease activity (Inter observer variability). Our primary outcome was agreement between ratings of disease activity as measured by both scores, while we used ratings by two independent physician reviewers to determine interrater agreement of Mayo scores.

MATERIALS AND METHODS

Chart review

This was a retrospective study performed at Boston Children’s Hospital. We utilized our IBD center database to identify ulcerative colitis patients who underwent colonoscopy over last 3 years. We have utilized the Probation endoscopy software for our colonoscopy reporting, and colonoscopy photographs are taken at the discretion of the endoscopist. For inclusion in this study, we allowed only one procedure per patient (i.e., patients with more than one procedure only had their first procedure selected). Reports were reviewed by research coordinators (SW, JBS) for the presence of at least one photograph of the rectum and/or sigmoid. In addition, charts were reviewed for clinical disease severity (PUCAI score) by the study physician (BK). In a subset of charts, PUCAI had been entered by the treating physician at the time of clinic visit. In other cases, the PUCAI score was calculated based on data abstracted from the chart. Because of the retrospective nature of this study, endoscopic and clinical assessments were not always performed on the same day. We included data to allow comparisons between endoscopic and clinical disease activity as long as there was no change in the clinical status of the patient, no addition of new medications, and no change in the dose of aminosalicylates, immunomodulators, and biologics between assessments.

Inter-rater reliability of mucosal inflammation

Two physicians with expertise in IBD endoscopy and grading of mucosal inflammation (AB and PR) independently reviewed photographs taken of the rectum and/or sigmoid. A standard photographic reference guide to Mayo scoring developed for an industry clinical trial was utilized by both physicians as a reference[19]. Physicians were blinded as to the clinical disease activity score and to the endoscopic score from the other physician. A Mayo score of “0” was assigned when mucosal findings were consistent with normal or inactive disease; “1” indicated mild disease, as characterized by erythema, decreased vascular pattern, mild friability; “2” was assigned when mucosal findings suggested moderate disease, as characterized by marked erythema, absent vascular pattern, friability and erosions; and a score of “3” was assigned when mucosal images showed severe disease, characterized by mucosal evidence of spontaneous bleeding and ulceration.

The rectum and sigmoid were scored independently by each rater. If one region was scored but the other was not, the region not scored was treated as missing data.

Correlation between PUCAI and Mayo score

For this analysis, we utilized the PUCAI abstracted or calculated from the chart, as well as individual components of the PUCAI (e.g., amount of rectal bleeding), and correlated it to the rectal and sigmoid Mayo scores. Since in some patients, PUCAI was not obtained on the same day as the endoscopic score, we allowed up to 35 d between the endoscopic score and the clinical score, as long as there had been no significant change in medical therapy (see above). If there was disagreement between the two physicians (AB and PR) on Mayo score, a third physician (JL) gave an independent blinded assessment of Mayo score, and the score chosen by two of the three reviewers was utilized.

Statistical analysis

To assess agreement of Mayo score between the two reviewers, Cohen’s kappa was calculated. If the kappa value was greater than 0.5, the two reviewers were considered to concur with reasonable agreement (0.01-0.20 slight, 0.21-0.40 fair, 0.41-0.60 moderate, 0.61-0.80 substantial, 0.81-0.99 almost perfect)[20]. Observations categorized as unknown were excluded from the kappa calculation. The percent agreement between the two reviewers was also calculated.

To assess the primary outcome of the correlation between PUCAI and Mayo scores, assuming a 5% two-sided significance level and the 99 available subjects, we have 80% power to detect whether a correlation coefficient of 0.27 or higher (R-squared value of at least 0.07) differs from 0 (nQuery Advisor Version 7.0, 1995-2007, Janet D. Elashoff). One-way ANOVA modeling was used to assess the relationship between continuous PUCAI and Mayo scores. Considering the Mayo scores as 0/1 and 2/3 with categorical PUCAI scores, a Cochran-Armitage test of trend was used to investigate whether a relationship held. Pearson’s correlation coefficients were used to assess correlations between subcomponents of the PUCAI and Mayo scores. Data was abstracted and entered into an SPSS database. Statistical analysis was performed by a biostatistician utilizing SAS Version 9.3 (SAS Institute Inc, Cary, NC). A 5% two-sided significance level was used for all statistical testing.

Ethical review

This study was conducted under a protocol for chart review approved by the Children’s Hospital Committee on clinical investigation.

RESULTS

Demographics

We identified 163 colonoscopies performed in children and young adults with ulcerative colitis performed during the study period. Of these, we excluded 64 procedures because they were either duplicate procedures on the same patient, there were no adequate colonoscopy photos to assess mucosal inflammation, there was no clinic visit in proximity to the endoscopy, or medical interventions had been performed between the colonoscopy and the clinical assessment that would affect disease activity. The final population for analysis was 99 patients, with demographics of the study population provided in Table 1. There were 47 males and 52 females, with a median age of 16 years. Most patients (82%) had had colitis for 1 year or more; 68% were receiving treatment with aminosalicylates, and 45% were receiving either thiopurines, methotrexate, or anti-TNF therapy.

Table 1 Descriptive information regarding the dataset (n = 99).

Characteristic	n (%)
Gender
Male	47 (47)
Female	52 (53)
Ulcerative colitis (UC) duration - scope
New onset	1 (1)
< 1 yr	16 (16)
1-3 yr	30 (30)
3-5 yr	16 (16)
> 5 yr	36 (36)
Indication for procedure
Suspicion of UC	1 (1)
Active symptoms	61 (62)
Consideration of Medication change	10 (10)
Assessment of Medication efficacy	13 (14)
Cancer screening	14 (14)
Paris classification
E1 Proctitis	1 (1)
E2 Left-sided	20 (20)
E3 Extensive	5 (5)
E4 Pancolitis	73 (74)
Medications at time of colonoscopy
5-aminosalicylic acid (5 ASA)	67 (68)
Steroids	30 (30)
Mercaptopurine/methotrexate	33 (33)
Infliximab/adalimumab	12 (12)
Antibiotics	10 (10)
Five ASA Enema	5 (5)
Hydrocortisone Enema	10 (10)
Methotrexate	1 (1)
Tacrolimus	3 (3)
VSL 3 probiotic	10 (10)

Inter-rater reliability of colonoscopy photos

For this analysis, the inter-rater reliability of two different regions (rectum and sigmoid) was compared. All patients had at least one interpretable photo of the rectum or the sigmoid colon. Of the 99 procedures, 18 did not have adequate rectal photography, while 5 did not have photos of the sigmoid. Not including the 18 in the “unknown” category, the percent agreement between the two reviewers for the rectum measure was 68% and the Cohen’s kappa value (95%CI) was calculated to be 0.54 (0.41, 0.68), indicating “moderate-substantial” agreement between the two graders (Table 2) In contrast, there was less agreement (60%) in the sigmoid scoring. For the sigmoid, Cohen’s kappa value (95%CI) was calculated to be 0.44 (0.32, 0.56), consistent with “fair-moderate agreement” (Table 3). Details of the data collection is in shown in Table 4.

Table 2 Inter-observer agreement in grading rectal disease severity (Mayo scores) between reviewers.

Reviewer AB	Reviewer PR
	0	1	2	3	Unknown	Total
0	22	4	1	0	0	27
1	1	22	12	1	0	36
2	0	2	8	4	0	14
3	0	0	1	3	0	4
Unknown	0	0	0	0	18	18
Total	23	28	22	8	18	99

Table 3 Inter-observer agreement in grading sigmoid disease severity (Mayo score) between reviewers.

Reviewer AB	Reviewer PR
	0	1	2	3	Unknown	Total
0	22	5	0	0	0	27
1	2	22	12	4	0	40
2	0	3	9	7	0	19
3	0	0	5	3	0	8
Unknown	0	0	0	0	5	5
Total	24	30	26	14	5	99

Table 4 Pediatric ulcerative colitis activity index data by component.

Characteristic	mean ± SD or n (%)
Age at time of colonoscopy (yr)	15.7 (4.1)
Days between colonoscopy and PUCAI score	14.8 (10.6)
Method of PUCAI collection
Prospectively entered at time of clinic visit	27 (27)
Calculated from medical record	72 (73)
Abdominal pain
No pain	52 (53)
Present but ignored	36 (36)
Cannot be ignored	11 (11)
Rectal bleeding
None	49 (49)
Small amount, < 50%	20 (20)
Small amount, most	27 (27)
Large amount	3 (3)
Consistency
Formed	57 (58)
Partially formed	35 (35)
Completely unformed	7 (7)
Number of stools
0-2	61 (62)
3-5	21 (21)
6-8	11 (11)
> 8	6 (6)
Nocturnal stools	18 (18)
Activity limitation
No limitation	62 (63)
Occasional limitation	29 (29)
Severe restriction	8 (8)
Overall activity
Remission, 0-9	33 (33)
Mild, 10-34	39 (39)
Moderate, 35-64	23 (23)
Severe, > 64	4 (4)

PUCAI: Pediatric ulcerative colitis activity index.

Correlation between Mayo and PUCAI scores

For this analysis, we compared the endoscopic severity (Mayo score) at time of colonoscopy with clinical disease severity (PUCAI) from a proximal clinic visit or inpatient assessment. For the comparison, we utilized the common Mayo score if both endoscopy raters agreed, or the three rater consensus score if the two raters disagreed. In our population, PUCAI scores were obtained before colonoscopy in 30% of patients, and after colonoscopy in 70% of patients. The median time between endoscopic assessment and calculation of PUCAI score was 14 d, interquartile range 5-21 d, and 91% of scores were obtained within 30 d.

There was strong correlation between Mayo endoscopic score of disease severity in the rectum and PUCAI (Figure 1; R-squared = 0.43, P < 0.001). Similarly, there was strong correlation between sigmoid inflammation and PUCAI (R-squared = 0.25, P < 0.001, data not shown). However, within each stratum of PUCAI score, there was variability within the Mayo rectal endoscopic scores. This was most notable in the mild to moderate range of PUCAI (Figure 2). Severe clinical disease correlated well both clinically and endoscopically, with no patients with PUCAIs of over 65 having Mayo scores of less than 2. However, 20% of patients with rectal Mayo scores of 1 or 2 had PUCAIs < 10 (clinical remission), suggesting that a subset of patients with mild to moderate endoscopic disease have minimal symptoms.

Figure 1 Correlation between disease severity in rectum (Mayo score) and Pediatric ulcerative colitis activity index. PUCAI: Pediatric ulcerative colitis activity index.

Figure 2 Distribution of rectal Mayo scores within each pediatric ulcerative colitis activity index stratum. PUCAI: Pediatric ulcerative colitis activity index.

While all six subcomponents of the PUCAI demonstrated statistically significant correlation with rectal Mayo scores, the strongest correlations were seen with rectal bleeding (Pearson’s r = 0.59), stool consistency (r = 0.50), and number of stools (r = 0.49). The weakest correlation was with abdominal pain (r = 0.33).

DISCUSSION

Our study was designed to ascertain the association between standardized scores of clinical and endoscopic disease severity in children and young adults with ulcerative colitis. In particular, we compared the Pediatric Ulcerative Colitis Activity Index (PUCAI) as a measure of clinical activity with the Mayo score of mucosal disease. We demonstrated that the degree of mucosal inflammation may be difficult to ascertain in pediatric patients with ulcerative colitis, even by experienced IBD endoscopists looking at the same endoscopic photos using a standardized guide. While statistically significant inter observer agreement existed, there was significant variability, especially with mild to moderate inflammation (Mayo scores of 1 and 2). Agreement was improved in patients in endoscopic remission (Mayo score 0) or severe colitis (Mayo score 3). We also identified a statistically significant correlation between endoscopic and clinical severity, but there was also variation in clinical disease severity within each stratum of endoscopic severity. We also demonstrate that some aspects of the PUCAI score (e.g., rectal bleeding) correlate better with endoscopic severity than other more subjective components (abdominal pain).

Our determination of inter-rater reliability of endoscopic disease severity is comparable to what has been reported in the literature when rating adults with IBD. In a prospective study of endoscopic disease severity utilizing video endoscopy and Mayo score grading, the endoscopic score had the lowest inter-rater reliability among four clinicians (κ = 0.38). In contrast, inter-rater reliability was more consistent in clinical aspects of the Mayo score like rectal bleeding[21]. Another recent paper utilizing experienced central readers reviewing videos demonstrated improved inter-rater reliability[22]. Our two clinicians assessing endoscopic disease severity have over 40 years combined experience in UC colonoscopy, yet our agreement looking at the same photos was still only modest. The inter-observer variability is a potential pitfall of using endoscopic disease severity as an inclusion criterion or a primary endpoint in pediatric clinical trials. Whether or not central reading is absolutely necessary in such studies, or whether physicians can objectively assess disease severity by simply looking at photos, requires further study.

Disease activity in ulcerative colitis has traditionally been assessed by clinical scoring systems, including the Simple clinical colitis activity index (SCCAI), Seo index, or clinical Mayo index[4]. For recent clinical trials of biologics in UC, the Mayo score (a composite index that includes both clinical parameters and endoscopic disease severity) has been the primary endpoint of choice. In pediatrics, the PUCAI, a well validated clinical scoring system has been utilized in both retrospective and prospective drug studies. Advantages of the PUCAI include its face validity, ease of use, and responsiveness to change. The PUCAI score after day 5 of intravenous corticosteroid therapy can identify a subset of patients that will require either surgery or medical salvage therapy[23]. The PUCAI has also been utilized to demonstrate response in studies of medical rescue therapies (infliximab and tacrolimus)[6,24]. The PUCAI was also the primary endpoint in the infliximab trial that obtained regulatory approval. However, the two limitations of the PUCAI recently raised are that it is not a patient reported outcome, and that there is no mucosal healing component.

Our current study establishes that there is a strong correlation between endoscopic disease severity and PUCAI score. Nevertheless, approximately 20% of patients who felt “well” in our study (i.e., had PUCAIs less than 10 signifying endoscopic remission), had mucosal inflammation. Beattie and colleagues noted a similar discrepancy between clinical and endoscopic severity in patients treated with corticosteroids[25]. Such patients (clinical remission but endoscopic inflammation) may be more likely to flare than those in full endoscopic remission based on adult data[26]. However, it is unclear whether the benefits of intensifying therapy (aka. “Stepping up to immunomodulators or biologics) outweighs the risks of additional immunosuppression. Another question is of whether adding immunosuppression in patients in remission but with endoscopic disease activity may reduce the risk of colorectal cancer, but current guidelines do not recommend adding thiopurines or biologics solely for cancer prevention[27].

Our study has several limitations due to its retrospective nature. Most importantly, the endoscopic assessment of disease severity (Mayo) and the clinical score (PUCAI) were not obtained at the same time. While we excluded patients that had major changes in treatment between the two assessments, we cannot exclude the possibility of some spontaneous deterioration or improvement. The endoscopic photographs were also not performed in a standardized manner, and in a subset of patients, photographs of the rectum and sigmoid were not taken. In addition, we did not evaluate any surrogate markers such as calprotectin or lactoferrin in this study which have recently been shown to be cost- effective in diagnosing and monitoring IBD[28]. In spite of the limitations, we feel our study adds valuable knowledge about the association between clinical and endoscopic disease severity in children. We plan to follow up this with additional prospective studies, which will hopefully further address the relationship between endoscopic and clinical disease activity in UC.

Endoscopic disease severity (Mayo score) assessed by reviewing photographs of pediatric colonoscopy has moderate inter-rater reliability, and agreement was less robust in assessing patients with mild disease activity. Endoscopic disease severity generally correlates with clinical disease severity. However, children with inflamed colons can have significant variation in their reported clinical symptoms. Thus, assessment of both clinical symptoms and endoscopic disease severity may be required in future clinical studies.

COMMENTS

Background

The incidence of inflammatory disease has been increasing across the world, particularly in children. There are no good parameters to assess the progress the of disease. There are also no good endpoints for drug trials in inflammatory bowel disease (IBD). Clinical indices like pediatric ulcerative colitis activity index (PUCAI) have been validated to assess disease burden. However, PUCAI score vary due to symptoms that may not be related to IBD. Regulatory agency like FDA have recommended endoscopic assessment for mucosal healing as end point in drug trials.

Research frontiers

Two commonly utilized indices to assess ulcerative colitis are the pediatric ulcerative colitis activity index (PUCAI, a clinical score) and the Mayo endoscopic score (endoscopic disease activity). Little data exists on inter-rater reliability of the Mayo score for assessing pediatric disease, or on the correlation between the PUCAI and Mayo score.

Innovations and breakthroughs

The authors demonstrated significant variability in degree of mucosal inflammation by endoscopy in patients with mild to moderate inflammation. There is a strong correlation between endoscopic disease severity and PUCAI score. However, approximately 20% of patients with remission as per PUCAI score had mucosal inflammation.

Applications

Clinical indices like PUCAI are not good end points for assessing disease activity or out comes in drug trial of ulcerative colitis. Mayo endoscopic score are associated intra observer variance especially in intermediate disease. A score that is combination of clinical indices and endoscopic assessment may be good to determine the outcomes. Endoscopic disease activity score may be better end points than clinical indices for drug trials in IBD.

Peer-review

The manuscript is well written and interesting.