Elaboration and validation of Crohn’s disease anoperineal lesions consensual definitions

Abstract

AIM

To establish consensual definitions of anoperineal lesions of Crohn’s (APLOC) disease and assess interobserver agreement on their diagnosis between experts.

METHODS

A database of digitally recorded pictures of APLOC was examined by a coordinating group who selected two series of 20 pictures illustrating the various aspects of APLOC. A reading group comprised: eight experts from the Société Nationale Française de Colo Proctologie group of study and research in proctology and one academic dermatologist. All members of the coordinating and reading groups participated in dedicated meetings. The coordinating group initially conducted a literature review to analyse verbatim descriptions used to evaluate APLOC. The study included two phases: establishment of consensual definitions using a formal consensus method and later assessment of interobserver agreement on the diagnosis of APLOC using photos of APLOC, a standardised questionnaire and Fleiss’s kappa test or descriptive statistics.

RESULTS

Terms used in literature to evaluate visible APLOC did not include precise definitions or reference to definitions. Most of the expert reports on the first set of photos agreed with the main diagnosis but their verbatim reporting contained substantial variation. The definitions of ulceration (entity, depth, extension), anal skin tags (entity, inflammatory activity, ulcerated aspect), fistula (complexity, quality of drainage, inflammatory activity of external openings), perianal skin lesions (abscess, papules, edema, erythema) and anoperineal scars were validated. For fistulae, they decided to follow the American Gastroenterology Association’s guidelines definitions. The diagnosis of ulceration (κ = 0.70), fistulae (κ = 0.75), inflammatory activity of external fistula openings (86.6% agreement), abscesses (84.6% agreement) and erythema (100% agreement) achieved a substantial degree of interobserver reproducibility.

CONCLUSION

This study constructed consensual definitions of APLOC and their characteristics and showed that experts have a fair level of interobserver agreement when using most of the definitions.

Key Words: Crohn’s disease, Anoperineal lesions, Fistula, Interobserver agreement

Core tip: We present the first study that establishes consensual definitions of anoperineal lesions of Crohn’s disease (APLOC) and assesses interobserver agreement on their diagnosis. With the help of APLOC experts, we conducted this work using a formal consensus method validated by the French National Authority for Health. We herein establish the missing consensual definitions and show that experts have substantial interobserver agreement when using them to diagnose and describe fistulae, ulcerations, activity of external openings and erythema from photos. Even if inspection is only one step in the diagnoses of APLOC, we believe this work will help future studies evaluate if and how treatments influence these lesions.

INTRODUCTION

Anoperineal lesions occur frequently during Crohn’s disease (CD) and are associated with a worse prognosis[1]. Some lesions require instrumental and/or medical specific treatments and the most severe may lead to severe discomfort including the loss of continence[2]. The Cardiff classification system proposed to classify them into three groups: ulceration, fistulae (or abscesses) and stenosis[3]. Their diagnosis relies on inspection and palpation and for fistulae, diagnosis is improved by endoscopic ultrasonography (EUS) and/or pelvic magnetic resonance imaging (MRI)[4,5].

The American Gastroenterological Association (AGA) guidelines recommend a physical examination of the perianal area to identify anal skin tags, anal fissures, perianal fistulae, suspected perianal abscesses, anorectal strictures and rectovaginal fistulae including an endoscopic evaluation to determine whether or not there is macroscopically evident inflammation of the rectum[4]. It recommends classifying the fistulae as either simple or complex. A simple fistula is low (superficial, intersphincteric or low transsphincteric origin of the fistula tract) with a single external opening; no pain or fluctuation to suggest perianal abscess; and no evidence of rectovaginal fistula or anorectal stricture. A complex fistula is high (high intersphincteric, high transsphincteric, extrasphincteric or suprasphincteric origin of the fistula tract); may have multiple external openings; may be associated with the presence of pain or fluctuation to suggest a perianal abscess; may be associated with the presence of a rectovaginal fistula; may be associated with the presence of an anorectal stricture; and may be associated with the presence of active rectal disease determined by endoscopy[4]. Hugues[3] anatomical and pathophysiological classification, distinguishes between primary and secondary infected lesions taking into account the type of lesion, its location and depth. The Perianal Disease Activity Index uses a five-point Likert scale: discharge, pain/restriction of activity, restriction of sexual activity, type of perianal disease and degree of local induration[6]. First proposed by Present et al[7]. The Fistula Drainage Assessment defines an active fistula by the existence of a purulent discharge after gentle finger compression of an external opening.

The occurrence, improvement or worsening of endoscopic lesions[8] and APLOC need to be carefully described in daily practice and clinical trials and this requires consensual definitions with good interobserver agreement.

The aims of this study were: (1) to find in literature, or to establish, a list of consensual descriptors of APLOC; and (2) to evaluate inter-individual variation in the use of these descriptors.

MATERIALS AND METHODS

Neither written consent nor institutional review board approval was required as French law considers that it is not mandatory for non-interventional research using blinded documents.

Literature search

Medical databases (e.g., PubMed, MEDLINE, EMBASE) were systematically searched for eligible literature. The eligibility criteria were: studies published in English or in French; studies published between 1955 and 2014 containing the terms: perianal CD, CD, perineal lesions, anal ulceration, anal fissure, anal skin tag, anoperineal fistula, vulva and anal stricture. The literature was searched for definitions of APLOC to be presented to the expert group.

Consensus method

A database of digitally recorded APLOC photos was examined by a coordinating group. They selected two series of 20 pictures illustrating the various aspects of APLOC. The reading group included eight experts from GREP and one academic dermatologist. All members of the coordination and reading groups participated in dedicated meetings. The study included two phases.

Phase 1: Establishment of consensual definitions

A formal consensus method was used to evaluate the level of agreement among experts on the definition wording. This method was both a guideline method and a consensus method (French National Authority for Health. Practice guidelines: “formal consensus method”. Saint-Denis. HAS; 2010). The experts identified and selected, through iterative ratings with feedback, the points on which they agreed, disagreed or were undecided (Figure 1).

Figure 1 Flow chart for the consensus steps.

For the first round of reading, each expert provided a written report for each of the first series of APLOC photos (Figure 2). The only information provided to them were the photos and they did not have access to MRI or digital examinations. The verbatim descriptions of the APLOC (names and adjectives) were extracted from these reports and from the literature. The coordinating group provided, to the whole group, the list of descriptors and corresponding definitions.

Figure 2 Selection of photos used in phase 1.

Agreement on those definitions was assessed using two votes. First, each expert assessor graded their agreement for each proposal between 1 (meaning the proposal was totally inappropriate) and 9 (meaning the proposal was totally appropriate). Agreement for every term was defined on the basis of the distribution of the scores of all experts: there was agreement when the scores were all ≤ 5 or all ≥ 5. Proposals were immediately accepted as appropriate when all scores were between 7 and 9 (median ≥ 7). They were immediately considered as inappropriate and rejected when all scores were between 1 and 3 (median ≤ 3). A meeting was then held to allow the reconsideration of the definitions which had not received strong agreement at the first vote. It consisted of a general discussion leading to the proposition of revised definitions and a second vote, on the new definitions, took place. In the case of missing values, analysis was considered valid if at least 9 scores were obtained for a proposal. If there were no missing values, one of the scores could be excluded, in the analysis of the degree of agreement, according to the following rules: the lowest score was excluded if the median was > 5, the highest score was excluded if the median was ≤ 5. The analysis distinguished the proposals that were deemed appropriate, those that were deemed inappropriate and those on which the group did not reach a consensus.

Phase 2: Study of interobserver agreement on the diagnosis of APLOC

Each expert evaluated a second set of 20 new APLOC photos, selected by the organising group and completed an online questionnaire on a Formstack^© platform. This platform provides the ability to set-up a conditional algorithm and enables the automatic input of the results into a Microsoft Excel^© database. The questionnaire required the evaluation of the presence or absence of each of the descriptors selected during the first phase of the study. All items for each picture had to be completed on the platform to allow access to the next photo.

Statistical analysis

Quantitative data from the two ratings were expressed as averages. Qualitative data from the annex of the first vote were expressed as proportions. Statistical analysis was performed with IBM SPSS Statistic for Windows© Version 20.0. The interobserver agreement on the diagnosis of ulceration, anal skin tag, fistula, anoperineal scar and perianal skin lesion was measured using Fleiss’s kappa coefficient with 95%CI. The interpretation of the values was performed according to the Landis and Koch scale[9]: almost perfect agreement between 0.81-1, substantial agreement between 0.61-0.80, moderate agreement between 0.41-0.60, fair agreement between 0.21-0.40, slight agreement between 0.01- 0.20 and absence of agreement < 0. The interobserver agreement for the other items was estimated using descriptive statistics and a threshold of 80% was arbitrarily chosen to define agreement.

RESULTS

Phase 1: Establishment of consensual definitions of descriptor terms and adjectives

Analysis of the usual descriptions used in literature and reports Terms used in literature to evaluate visible APLOC do not include precise definitions or reference to definitions. Most of the expert reports on the first set of photos agreed with the main diagnosis but their verbatim descriptions contained substantial variation. For example, some experts used the term fissure while others used the term ulceration; and the fistula opening was qualified as inflammatory, productive or active.

Consensus steps

Vote 1: A list and definition of descriptors was discussed (each with its own qualifier). The list of definitions submitted to the first vote and the results are shown in Table 1. During the subsequent consensus meeting that aimed to reconsider definitions without strong agreement, experts considered it important to qualify ulceration by location, inflammatory aspect of edges and depth. They also considered it worth describing the extension of the lesions. For fistulae, they decided to follow the AGA’s guidelines which defines fistula complexity and insisted that while complexity can sometimes be diagnosed from a photo, inspection alone is insufficient to qualify a fistula as simple[3].

Table 1 Initial proposed definitions submitted for the first vote and the results of the agreement for those propositions.

Lesions and descriptors	Definitions submitted to the votes	Agreement scores1
Ulceration	Skin or mucosal defect	8.7 (8-9)2
Depth
Deep or cavitating	Visible muscular and/or granulation tissue and/or undermined and inflammatory edges	8 (7-9)2
Superficial	Which is not deep	7.7 (7-9)2
Location
Anal canal and anal margin	Visible after unfolding the anal radial folds	7 (1-9)
Anal margin	Anal external edges	7.9 (5-9)
Perianal	Outside the anus radial folds	8.6 (6-9)
Extension
Extensive	≥ 5 mm	5.6 (1-9)3
Limited	< 5 mm	5.6 (1-9)3
Skin tag	Tense nodular lesion with granulomatous but not papillomatous aspect	4.8 (2-9)3
Activity
Inflammatory	Edematous aspect	6 (3-9)3
Non inflammatory	Fibrous aspect	6.2 (3-9)3
Perianal skin lesions
Papula	Visible and elevated palpable lesion < 2 cm in diameter	8 (6-9)
Node	Nodular cutaneous elevation, visible and/or palpable, diameter ≥ 2 cm	6.1 (1-9)
Edema	Swollen area	6 (3-7)3
Erythema	Redness
Pustula	Raised, visible and palpable lesion, with cloudy fluid content	8.2 (6-9)
Abscess	Swollen red skin, sometimes surrounded by necrosis area and which can release pus	6.5 (5-9)
Fistulae
Complexity
Complex	Recto-vaginal fistula or multiple external openings or abscess or remote external opening	6.6 (4-9)
Simple	Single external opening, next to anal margin, without abscess	6.3 (3-9)3
Note	Inspection alone can sometimes confirm the diagnosis of a complex fistula	9 (9-9)3
	Inspection alone cannot establish a diagnosis of simple fistula with certainty	8 (6-9)3
Drainage
Well drained	No abscess, no visible discharge and non-inflammatory external openings	7.6 (5-9)
Poorly drained	Inflammatory external opening(s) and/or abscess	7.4 (4-9)
External opening(s)
Inflammatory	Erythematous surrounded skin and budding port(s) with undermined edges	8.5 (7-9)2
Scar appearance	Fibrous and retracted aspect of the anal margin	7.8 (6-9)
	Deformed anus	7.8 (5-9)

¹Median and distribution;

²Definitions with strong agreement;

³Missing data.

Vote 2: The list of definitions submitted for the second vote and the results are shown in Table 2. In accordance with the protocol, extreme data were excluded for six of the proposals. The definition of a lesion node obtained relative agreement [median 7.8 (6-9); two values < 7]. The remaining definitions received strong agreement.

Table 2 Consensual definitions and agreement scores.

Lesions and descriptors	Consensual definitions	Agreement scores1
Ulceration	Skin or mucosal defect	8.7 (8-9]3
Depth
Superficial	Which is not deep	8 (7-9)3
Deep	Visible muscular and/or granulation tissue and/or undermined and inflammatory edges	7.7 (7-9)3
Cavitating	Deep decaying and destructive ulceration	8.6 (8-9)
Localisation
Anal canal	Visible after unfolding the anus radial folds	8.5 (7-9)
Anal margin	Anal external edges	8.5 (7-9)
Perianal	Outside the anus radial folds	9
Extension
Number		8.6 (8-9)
Percentage of ulcerated area	< 25%, 26-50%, > 50% of the anal circumference	8.3 (5-9)
Skin tag	Skin thickening of the anus	7.2 (3-9)
Activity
Inflammatory	Oedematous, swollen, tense	8 (4-9)
Non inflammatory	Fibrous, firm and non-oedematous aspect	8.7 (2-9)
Ulcerated (vs not)	Ulceration on the skin tag (vs no ulceration on the skin tag)	7.8 (2-9)
Perianal skin lesions
Papula	Elevated, circumscribed and solid lesion without liquid content	8.1 (6-9)
Node2	Elevated, nodular and protruding lesion, impression of deep extension	7.8 (6-9)
Oedema	Swollen appearance	8.2 (7-9)
Erythema	Flat redness	8.3 (5-9)
Abscess	Swollen red skin, which may be surrounded by necrosis area and which can release pus	8.3 (7-9)
Fistula
Complexity	AGA’s definitions (3)
	It is possible to recognize that a fistula is complex by inspection alone	7.2 (1-9)
	Inspection alone is not always sufficient to reliably recognize that a fistula is simple	8.5 (7-9)
Drainage
Well drained	Absence of abscess, of purulent discharge an non inflammatory external opening(s)	7.8 (5-9)
Poorly drained	Inflammatory external opening(s) and/or abscess and/or spontaneous purulent discharge	7.5 (5-9)
	Poor drainage of a fistula can be diagnosed by inspection alone	8.6 (7-9)
	Inspection alone is often not sufficient to be certain that a fistula is well drained	7.6 (1-9)
External opening
Inflammatory	Erythematous surrounded skin and budding port(s) with undermined edges	Validated by vote 1
Scar appearance	Deformed anus with fibrous aspect and/or retractile appearance of the anal margin	7.6 (8-9)
	With inflammatory activity or not	7.6 (3-9)

¹Median and distribution;

²Validated by vote 1;

³All definitions obtained a strong agreement except the entity “node”.

Phase 2: Interobserver agreement on the description of APLOC using consensual definitions

There was substantial agreement for the diagnosis of ulceration (κ = 0.70, 95%CI: 0.62-0.78). There was 73.3% agreement for qualifying marginal and perianal locations. There was no consensus for either the qualification of anal canal location (53.3%). The depth feature did not reach an agreement for ulceration, as well as the characteristic extension. There was moderate agreement for the diagnosis of anal skin tags (κ = 0.49, 95%CI: 0.41-0.57) and 75% agreement for the evaluation of activity and ulcerated appearance. There was substantial agreement for the diagnosis of fistulae (κ = 0.75, 95%CI: 0.66-0.83) and the evaluation of external opening inflammation (86.6%). There was some disagreement when appreciating complexity (60%) and quality of drainage (33.3%). Experts considered that fistula complexity and quality of drainage could not be evaluated from the photos in 10% and 21% of the test cases respectively. The diagnosis of perineal skin lesions obtained moderate agreement (κ = 0.50, 95%CI: 0.42-0.59). The items erythema and abscesses obtained good agreement (100% and 84.6% respectively). There was fair agreement for the diagnosis of scar appearance (κ = 0.34, 95%CI: 0.26-0.43) and an agreement of 75% for the inflammatory characteristics. Figure 3 shows the APLOC where diagnosis had substantial interobserver agreement (four photos from the selection used in Phase 2).

Figure 3 Anoperineal lesions of Crohn's which achieved a substantial degree of interobserver reproducibility for their diagnosis (photos used in phase 2).

DISCUSSION

Consensual definition of the words and adjectives used to describe the APLOC are lacking in medical literature except for fistulae. We established the missing consensual definitions and showed that experts have substantial interobserver agreement when using them to diagnose and describe fistulae, ulcerations, activity of external openings and erythema from photos.

This study has several limitations. The first is that inspection is only one step in the diagnoses of APLOC which also requires palpation (digital external and anal examination), endoscopic rectal examination and often MRI and/or EUS when fistulae are suspected[5]. The experts had only access to the photos but not to MRI or digital examination, which are often considered as confirmation elements or gold standard. Another limitation is that only French experts were involved. However, there is a need to establish consensual wording describing the visible lesions and we believe that this step will help future studies evaluate if and how treatments influence lesions. Obtaining expert group consensus was relatively simple and consequently, we expect the definitions will also get consensus in other countries and continents. A similar approach, using consensual definitions, has allowed significant improvement in medical trials and daily practice using endoscopy as an endpoint, although central reading by specialised experts still decreases interobserver variability[10,11]. Consequently, we feel it is important for surgeons and non-operating specialists to use a common language (the majority of the experts in this study perform surgical treatment of APLOC). Even though a fistula cannot be considered as completely closed without palpation, especially under anesthesia, visual inspection of lesions without a surgeon present does provide additional information and we believe the definition of lesions and their characteristics as proposed here is valid.

APLOC are heterogeneous. The most severe are deep ulcers, fistulae and abscesses (penetrating lesions) and stenosis. Established classifications of fistulae help predict the risks and influence of treatment decisions[4,12]. The Fistula Drainage Assessment, established and first used by Present et al[7], characterises fistulae as open (i.e., actively draining) or closed. A fistula is considered to be open if an investigator can express purulent material from the fistula by gentle finger compression. Following the principles of this study, improvement can be recognised in the case of reduction of an open or secreting fistula by greater than 50% compared with baseline levels on at least two consecutive examinations and remission can be defined as a cessation of secretion of all fistulae in relation to the baseline level on at least two consecutive examinations. The gentle finger compression technique can vary between investigators and its reproducibility is unknown. Moreover, several studies using MRI or EUS have demonstrated that a fistulae may have persistent activity even when it looks closed[13]. We think that the inflammatory aspect of the external orifice provides meaningful information for the clinician and show here that it can be assessed with good interobserver reproducibility using our definitions. On the other hand, some chronic fistulae have a “cold appearance” with epithelialization of the external opening allowing them to open. Using a proper description to classify them would probably help establish the best treatment strategy in this specific setting. We consensually concluded that the adjectives: “well drained” or “poorly drained”; and “inflammatory” or “not inflammatory” aspects of the external opening(s) and their definitions were the most appropriate way to qualify fistula during inspection. We also believe, in daily practice these characteristics are often taken into account when deciding to optimise anti-TNF treatment and common wording would assist the assessment for future treatment studies. The duration of fistula persistence, sometimes dependent on its mode of onset, is probably important for the risk of evolution towards a complex form and fibrosis. Using a validated and consensual vocabulary should allow the demonstration and quantification of this.

Superficial APLOC also have a prognostic value as some represent the early stages of penetrating and stenosing lesions. In the literature and the spontaneous verbatim reporting by the experts during the first step of this study, the words fissure, ulceration and ulcer had not been defined neither their differences. We came to the consensus that the term ulceration was appropriate for all losses of substances (be it in the skin or anal area) and the depth of ulceration could be reproducibly and meaningfully described using the adjectives superficial, deep and cavitating. The definitions we have proposed should help future studies describe the natural history of lesions or their evolution during treatment. We think that inflammatory lesions including inflammatory skin tags, inflammatory external openings and frank erythema also benefit from treatment optimisation (especially with anti-TNF agents) and this should be evaluated in future studies.

In conclusion, this study constructed consensual definitions of APLOC and their characteristics and demonstrated that experts have substantial interobserver agreement when using them. While definitions of the most severe lesions (fistulae) had been proposed here, there was a lack of definition for early-lesions which should be treated more aggressively to avoid irreversible lesions. These definitions should ease clinical studies and improve teaching of fellows and clinical investigators.

ACKNOWLEDGMENTS

The authors thank the GREP for their contribution to the study.

COMMENTS

Background

Except for fistulae, there are no consensual definitions of anoperineal lesions of Crohn’s (APLOC) disease. The verbatim reporting is thus variable in clinical reports and literature.

Research frontiers

Studying the evolution of APLOC requires proper and precise description (i.e., consensual definitions of their nature and characteristics) to establish if and how any treatment modifies the lesions.

Innovations and breakthroughs

They arrived at consensual definitions of ulceration (entity, depth, extension), anal skin tags (entity, inflammatory activity, ulcerated aspect), fistula (complexity, quality of drainage, inflammatory activity of external openings), perianal skin lesions (abscess, papules, edema, erythema) and anoperineal scars. For fistulae, they decided to follow the American Gastroenterology Association’s guideline definitions. From a series of APLOC photos, the diagnosis of ulceration, fistulae, inflammatory activity of external fistula openings, abscesses and erythema achieved a substantial degree of interobserver reproducibility.

Applications

Using these consensual definitions in daily practice and clinical studies should allow for improvement in the reproducibility of reports describing APLOC and their fate.

Terminology

The definitions of fistulae which already exist in literature have not been changed. Those of other APLOC are easy to remember and apply.

Peer-review

It is a study which explored the elaboration of consensual definitions of APLOC and inter observer agreement on the diagnosis of these lesions between experts. Authors suggest that herein they established the missing consensual definitions and show that experts have a substantial interobserver agreement when using them to diagnose and describe ulcerations, fistulas, activity of external openings and erythema on photographs.