Use of the Crohn’s disease activity index in clinical trials of biological agents

Abstract

The Crohn’s disease activity index (CDAI) has been commonly used to assess the effects of treatment with different agents in Crohn’s disease (CD). However, these studies may be compromised, if the results compared to a placebo or standard therapy group (in the absence of a placebo) substantially differ from the expected response. In addition, significant concerns have been raised regarding the reliability and validity of the CDAI. Reproducibility of the CDAI may be limited as significant inter-observer error has been recorded, even if measurements are done by experienced clinicians with expertise in the diagnosis and treatment of CD. Finally, many CDAI endpoints are open to subjective interpretation and have the potential for manipulation. This is worrisome as there is the potential for significant financial gain, if the results of a clinical trial appear to provide a positive result. Physicians caring for patients should be concerned about the positive results in clinical trials that are sponsored by industry, even if the trials involve respected centers and the results appear in highly ranked medical journals.

Key Words: Crohn’s disease; Crohn’s disease activity index; Clinical trials; Infliximab; Adalimumab; Corticosteroids; Azathioprine

INTRODUCTION

The Crohn’s disease activity index (CDAI) is a numerical calculation derived from the sum of products from a list of 8 items (Table 1), and multiplied by weighting factors for each item to define the severity of “disease activity” in patients with Crohn’s disease (CD)[1]. Essentially, the CDAI represents a numerical estimation of a physician’s interpretation of patient symptoms. Initially, the CDAI was correlated with a physician’s overall global assessment for a group of 112 patients with CD after a total of 186 patient visits done in 13 different hospitals in the United States[1]. This instrument was first applied in the National Cooperative Crohn’s Disease Study (NCCDS), published in 1979, and used to quantitatively compare the results in a placebo group to groups treated with the following drugs: sulphasalazine, prednisone and azathioprine.

Table 1 CDAI items and weighting factors.

Item (daily sum per week)	Weighting factor
Number of liquid or very soft stools	2
Abdominal pain score in one week (rating, 0-3)	5
General well-being (rating, 1-4)	7
Sum of physical findings per week:	20
Arthritis/arthralgia
Mucocutaneous lesions (e.g. erythema nodosum, aphthous ulcers)
Iritis/uveitis
Anal disease (fissure, fistula, etc)
External fistula (enterocutaneous, vesicle, vaginal, etc)
Fever over 37.8°C
Antidiarrheal use (e.g. diphenoxylate)	30
Abdominal mass (no = 0, equivocal = 2, yes = 5)	10
47 minus hematocrit (males) or 42 minus hematocrit (females)	6
1-x (1-body weight divided by a standard weight)	1

Index values of 150 and below were associated with quiescent or non-active disease (i.e. “remission”). Values over 150 were indicative of active disease, and over 450, extremely severe disease. Of particular note, in patients randomized to placebo in the NCCDS, 32% achieved a spontaneous remission at the end of 17 wk, and 53% of these were still in remission at the end of 24 mo[2]. The results of drug treatment using the CDAI in active disease showed that the response to prednisone or sulfasalazine was significantly better than placebo, while the response to azathioprine was better than placebo, but did not reach statistical significance[3]. Patients with colonic involvement were especially responsive to sulfasalazine, and those with small bowel involvement were especially responsive to prednisone[3]. Since these studies were reported, other publications[4–7] have appeared employing the CDAI as well as other indices[8–12] to assess disease activity in the evaluation of therapeutic effectiveness of different pharmacological and biological agents[13].

PLACEBO RESPONSE

Knowledge of the expected response to a placebo may be useful, but may not always be available. The NCCDS compared each of the treatment groups to a placebo group. Therapy with a new agent may now require comparison to the standard treatment, instead of a placebo. In part, at least, this relates to the ethical need to ensure that all participants in a clinical trial have access to proven therapy. Placebo alone may be difficult to justify, especially if patients are significantly symptomatic. And, if a new treatment shows a statistically positive result, it could reflect a limited effect of standard treatment (or the placebo). As in day-to-day clinical practice, the standard treatment may not always perform as well (or as badly) in each clinical trial. To best appreciate the results of a clinical trial involving a new agent, the expected response to placebo and treatment groups, as in the NCCDS, may be both important and informative.

OBSERVER DIFFERENCES

Although the CDAI appeared to correlate with the global clinical assessment in a limited number of CD patients, it was appreciated that the same observer or different observers might derive different results at different times. Thus, efforts were made by the CDAI investigators and later groups to evaluate the reliability and validity of the CDAI. In the initial report of its development[14], the consistency of the CDAI was examined in 2 successive visits for 32 patients. A positive association with the physician global assessment (with some overlap) was noted with a total of 50 index points being the apparent difference between slight clinical improvement (or worsening) and no actual change in symptoms for 2 successive visits. Subsequent studies also suggested that the reliability of the CDAI is within a defined moderate to good range, but not in a defined very good to excellent range[1516]. Later efforts to re-calculate the CDAI by these investigators showed no significant difference from the original estimation, so no further changes in the CDAI were recommended[16].

The CDAI appeared to be relatively reliable if used by these observers well experienced in its application. However, significant inter-observer differences were later noted in a series of separate studies, published as a single paper[17]. In one study, the CDAI was calculated from 10 “paper cases” evaluated by 5 consultants in surgery or gastroenterology, and 2 research assistants. Major discrepancies in calculated CDAI values were noted for each of the 7 different cases ranging from 166 to 430 points! In a second study, 15 members of the IOIBD prospectively evaluated a single case. The range of estimated values was 320 to 391, or 71 points. Improvement in observer differences, however, could be achieved with terminology discussion prior to calculation. A final study assessed the ability of 6 experienced gastroenterologists to independently elicit patient data, rather than being provided by the information from the “paper case” format. Wide variations in the estimated CDAI were seen ranging (in 1 case) up to 500 points! “Good agreement” was believed to have been achieved even in the best 2 patients, the difference was over 50 points. In spite of these critical concerns, the CDAI is used today in most clinical trials to evaluate different agent effects on symptom activity in CD.

OTHER CDAI ISSUES

There are other recognized difficulties with the CDAI per se. First, as noted elsewhere[18], a significant component of the total CDAI score is derived from highly subjective items, such as “general well being” and “intensity of abdominal pain”. Many symptoms overlap with those that might be ascribed to some other functional causes. To overcome this issue, it has been logically recommended that these patients should be randomly distributed to each of the treatment groups[18]. But, in practice, this may be difficult to accomplish and may not be done. Second, the CDAI is computed using a diary card that must be developed by the patient for 7 d prior to submission. In practice, this may be difficult for some patients to prospectively do well without close monitoring which may not be feasible. Some clinical study coordinators apparently assist patients in retrospective completion of 7-d diaries on the study visit[18] and this could clearly impact results. As noted[18], there are no data on the prevalence of this practice in clinical trial centers and it is not generally mentioned in the Methods section of the published reports. Third, some symptoms may be difficult to interpret or quantitate easily. For example, “liquid” stools may be difficult to precisely define[17]. Finally, even though enterocutaneous fistulae may be very troublesome for the patient, their impact on the CDAI may be limited. “General well being” may be severely impacted by perianal disease, but the patient may be fully functional. As a result, other indices have been developed to directly evaluate this component[19]. Even here, however, the endpoint measured may be criticized. For example, an open fistula may be differentiated from a closed fistula based on expression of purulent material from the tract following application of “gentle pressure”. Data on intra-observer and inter-observer agreement for such endpoints in clinical trials are not available. Moreover, full disappearance of fistula tracts is rarely documented.

IMPACT OF INDUSTRY

Another significant issue is the impact of the pharma-ceutical industry in the conduct of clinical trials. This issue has been addressed in detail elsewhere[20]. In 1979, the NCCDS was reported to be largely supported through peer-review national research grant agencies for all patient study costs while the pharmaceutical industry was simply acknowledged for their donation of the study medications. Of course, it is not known if there were other financial benefits provided by industry then since requirements for reporting of industry support were minimal. Now, virtually all clinical trials are conducted almost entirely through industry support, either from private or shareholder-owned public companies. Many clinical trials are largely authored by a select group of “experts”, often, but not always, affiliated with university centers. Some authors are fully employed by the sponsoring industry and have the responsibility of collecting and monitoring all data collected, evaluating results and even drafting the manuscript. Some, but not all, journals require that all investigators declare income received from industry in the form of honoraria and other forms of financial support, although the precise amounts are never disclosed. Others may actually be owners or shareholders of the company concerned. This declaration is meant to alert the reader that the investigator may have a conflict of interest, not increase investigator credibility. If the journal did not require the declaration, it would not be provided. The problem, however, is even deeper. While hospitals involved in provision of research facilities may not be directly involved in conduct of the clinical trial, a “fee” is usually attached to the clinical trial costs for use of facilities, and so, indirectly, the institution (and its reputation possibly earned over many decades) is also being “purchased” by industry. Many teaching institutions, limited in resources, often consider industry funding as a positive asset for faculty members in the promotion process. In everyday clinical practice, it may be difficult to wade through all of these issues. The CDAI provides a simple numerical means to statistically evaluate treatment response, but this measurement has the potential to be manipulated as a positive spin will surely impact the share price in a positive direction.

TOP-DOWN AND BOTTOM-UP CDAI TRIALS

A recent clinical trial compared infliximab to “con-ventional therapy” for CD[21]. An industry marketing label for these respective approaches has already appeared in the literature: “top-down” and “bottom-up”. In this study, both investigators and patients were aware of the drugs being used for treatment. The study reported a positive result in favor of infliximab and azathioprine. The potential conflict of interest for a positive study result was clearly evident for all concerned in this open label study. Patients can actively influence the results of open label clinical trials, especially if an ordinarily expensive treatment is offered at no cost. Many of the authors noted financial support from a number of companies, including those with a vested commercial interest in the direct global marketing of infliximab. Even the editorialist[22] listed support from the same companies, certainly not an independent view of the data.

This open label study illustrated more about the modern conduct of clinical trials, their potential for conflict of interest and the credibility of the investigators associated with the results of this trial. Clinicians will need to be more suspicious of the results of clinical trials for CD, especially if based partly, or solely, on the CDAI, an index whose reliability and validity may be quite limited. Moreover, as the CDAI was used originally to assess the effectiveness of therapeutic agents having been used by physicians for decades, the present use of the CDAI seems to have evolved from its original intent. Now, potent biologicals (rather than pharmaceuticals) are being explored in clinical trials to determine if CDAI numbers, reflecting clinical symptoms, can be altered sufficiently to produce a statistically relevant result. Interestingly, even in an early report with infliximab in CD, a statistically positive result was observed with a specific dose of the agent, but in the same study, a dose-response could not be defined with higher doses[13]. This lack of dose-response with this biological agent was not explained. However, in retrospect, the study design clearly provided an advantage to infliximab treatment. There were three different infliximab groups that could be compared to only one placebo group. A statistically significant result for any one of these infliximab groups compared to the placebo would have been considered positive in favor of the new agent. Also, it was reported that 33% of the infliximab-treated patients had a CDAI-defined remission compared to only 4% for placebo. While statistically different, the remission rate for the infliximab-treated group was remarkably similar to the earlier reported placebo rate in the NCCDS. It is not known if these results reflected, to some degree, the inherent limitations of the CDAI to estimate patient symptoms, even in a blinded clinical trial, and permit comparisons with earlier studies.

Where will this approach related to the “che-motherapy of the CDAI” take us? No one really knows. However, the impact of an investigator bearing therapeutic “gifts” for the patient in a clinical trial, while, at the same time, personally receiving “gifts” from industry should raise concerns among physicians that are responsible for patient care.

NEED FOR LONG-TERM STUDIES ON NATURAL HISTORY OF CD

A final paragraph might be added to raise a separate issue. It is probably not sufficient to simply raise criticism and concern regarding the CDAI. Others have explored the use of other indices, some perhaps simpler to apply in a clinical setting. For example, the Harvey-Bradshaw Index[23] evaluates symptoms and signs during the preceding 24 h, rather than the previous 7 d, reduces the original 8 items in the CDAI to 5 (i.e. excluding antidiarrheal use, hematocrit and body weight), and eliminates the weighting factor. Its use appears to correlate well with the CDAI[2324]. Another, the Cape Town Index (or South African Index) was developed as another alternative to avoid the “single parameter” bias inherent in the Harvey-Bradshaw Index[25]. Another index, the Van Hees Index[26] was developed in an attempt to avoid subjective clinical criteria by employing a physician global assessment variable. Most important, each of these different indices really attempts to measure only a very limited temporal window in the clinical course of CD. A longer term view of CD is required[2728] so that objective measurement of the effects of different therapeutic efforts can be seen to positively or negatively alter the natural history of this chronic inflammatory process.