Prognostic value of risk scoring systems for cirrhotic patients with variceal bleeding

Abstract

BACKGROUND

Acute variceal bleeding is one of the deadliest complications of cirrhosis, with a high risk of in-hospital rebleeding and mortality. Some risk scoring systems to predict clinical outcomes in patients with upper gastrointestinal bleeding have been developed. However, for cirrhotic patients with variceal bleeding, data regarding the predictive value of these prognostic scores in predicting in-hospital outcomes are limited and controversial.

AIM

To validate and compare the overall performance of selected prognostic scoring systems for predicting in-hospital outcomes in cirrhotic patients with variceal bleeding.

METHODS

From March 2017 to June 2019, cirrhotic patients with acute variceal bleeding were retrospectively enrolled at the Second Affiliated Hospital of Xi’an Jiaotong University. The clinical Rockall score (CRS), AIMS65 score (AIMS65), Glasgow-Blatchford score (GBS), modified GBS (mGBS), Canada-United Kingdom-Australia score (CANUKA), Child-Turcotte-Pugh score (CTP), model for end-stage liver disease (MELD) and MELD-Na were calculated. The overall performance of these prognostic scoring systems was evaluated.

RESULTS

A total of 330 cirrhotic patients with variceal bleeding were enrolled; the rates of in-hospital rebleeding and mortality were 20.3% and 10.6%, respectively. For in-hospital rebleeding, the discriminative ability of the CTP and CRS were clinically acceptable, with area under the receiver operating characteristic curves (AUROCs) of 0.717 (0.648-0.787) and 0.716 (0.638-0.793), respectively. The other tested scoring systems had poor discriminative ability (AUROCs < 0.7). For in-hospital mortality, the CRS, CTP, AIMS65, MELD-Na and MELD showed excellent discriminative ability (AUROCs > 0.8). The AUROCs of the mGBS, CANUKA and GBS were relatively small, but clinically acceptable (AUROCs > 0.7). Furthermore, the calibration of all scoring systems was good for either in-hospital rebleeding or death.

CONCLUSION

For cirrhotic patients with variceal bleeding, in-hospital rebleeding and mortality rates remain high. The CTP and CRS can be used clinically to predict in-hospital rebleeding. The performances of the CRS, CTP, AIMS65, MELD-Na and MELD are excellent at predicting in-hospital mortality.

Key Words: Cirrhosis; Variceal bleeding; Rebleeding; Mortality; Risk score

Core tip: Acute variceal bleeding is one of the most serious complications of cirrhotic patients with a high risk of in-hospital rebleeding and mortality. This study validated and compared the overall performance of eight prognostic scores for predicting in-hospital adverse outcomes in cirrhotic patients with variceal bleeding. We screened out some useful prognostic scores for predicting in-hospital adverse outcomes, especially for predicting in-hospital mortality. These prognostic scores can be easily used for early identification of high-risk patients. For high-risk patients, a transfer to a better hospital, close monitoring and aggressive treatments can help to reduce the risk of in-hospital adverse outcomes.

INTRODUCTION

Cirrhosis is an end-stage liver disease with high mortality and manifests as various degrees of portal hypertension and hepatic dysfunction. Based on the presence or absence of decompensation events (ascites, variceal bleeding, encephalopathy, and jaundice), cirrhosis can be categorized into different prognostic stages: compensated or decompensated cirrhosis[1,2]. Acute variceal bleeding is one of the most life-threatening complications. 22%-61% of cirrhotic patients receiving primary prophylaxis will develop first variceal bleeding during the first two years of follow-up[3]. Furthermore, variceal bleeding is associated with a high risk of rebleeding and mortality. A recent study reported that rebleeding and mortality rates within one month were 25.7% and 15.2%, respectively[4]. Although patient prognosis has improved with modern treatments that can control bleeding, the adverse event rate after variceal bleeding remains high. Therefore, high-risk patients with cirrhosis must be identified early, which can help determine appropriate candidates for risk communication, early intervention, close monitoring, or even early transfer to an intensive care unit.

Some clinical scoring systems have been established and used for predicting clinical outcomes in patients with acute upper gastrointestinal bleeding (UGIB). Among these systems, the most widely used are the clinical Rockall score (CRS), AIMS65 score (AIMS65), and Glasgow-Blatchford score (GBS). These three systems were developed in patients with both nonvariceal UGIB and variceal UGIB and have been widely validated in previous studies for patients presenting with UGIB[5]. However, most of these studies excluded patients with variceal bleeding or included only a small number of these patients. For patients with variceal bleeding, very limited data regarding the prognostic value of these scoring systems are available[5,6]. Furthermore, only a few studies have used these scores to predict the in-hospital outcomes of patients with variceal bleeding, and their conclusions were controversial[4,7-9]. Adverse outcomes during hospitalization are typically the focus of patients and doctors after admission. In addition, two newly created scoring systems have not been externally validated[10,11]. The model for end-stage liver disease (MELD), Child-Turcotte-Pugh score (CTP), and MELD-Na are considered useful for predicting short-term prognoses[12]. The CTP, MELD, MELD-Na, CRS, AIMS65, GBS, modified GBS (mGBS) and Canada-United Kingdom-Australia score (CANUKA) are selected as candidates as they are generally recognized and considered to be useful for predicting short-term outcomes. In addition, validation of these prognostic scores in Chinese patients is rare in terms of predicting in-hospital outcomes. On the other hand, these prognostic scores are easy to calculate using clinical and readily available laboratory variables, so they can be widely used by hospitals of different levels. Therefore, this study aimed to validate and compare the overall performance of these eight prognostic scoring systems for predicting in-hospital outcomes in cirrhotic patients with variceal bleeding.

MATERIALS AND METHODS

Study population

This retrospective cohort study was reported following the TRIPOD statement[13] and conducted at the Second Affiliated Hospital of Xi’an Jiaotong University. Consecutive cirrhotic patients with endoscopically confirmed variceal bleeding between March 2017 and June 2019 were identified by reviewing medical records. The inclusion criteria were adult patients with liver cirrhosis who were admitted to our hospital due to variceal bleeding. The exclusion criteria included the following: (1) Patients who were younger than 18 years old; (2) Patients who refused or could not tolerate endoscopy; (3) Patients with endoscopy-confirmed acute UGIB from non-variceal origins; (4) Transferred patients who were treated at external hospitals; and (5) Patients with incomplete medical records. The diagnosis of liver cirrhosis was made either by clinical assessment with a physical examination, laboratory indices, radiological findings or liver biopsy. Variceal bleeding was diagnosed if gastroscopy showed any of the signs of variceal bleeding according to the Chinese guidelines[14]. All patient management was in line with the standard protocol for cirrhotic patients with acute variceal bleeding[14]. Bleeding patients underwent a preliminary clinical assessment and were resuscitated as soon as possible. Resuscitation measures included endotracheal intubation, oxygen inhalation, multiple peripheral lines or deep vein access, fluid resuscitation, blood transfusions, nasogastric tube insertion, and medication administration (antibiotics, octreotide, somatostatin, terlipressin or anti-hepatic encephalopathy regimens). Balloon tamponade or emergency endoscopic treatment was performed as needed, or the patient was transferred to the intensive care unit. Gastroscopy was scheduled as early as possible, and endoscopic therapies were performed as needed. Transjugular intrahepatic portosystemic stent-shunt (TIPSS) or surgery was considered for cirrhotic patients when endoscopic therapies failed or were unsuitable. These special treatments were performed after obtaining informed consent; if the patients did not consent, they received only medication to control bleeding. This study protocol was approved by the ethics committee of our institution (2019042).

Data collection

The prognostic scores for CTP, MELD, MELD-Na, CRS, AIMS65, GBS, mGBS and CANUKA were calculated for each patient. The formulas and components of the eight scoring systems for calculating the prognostic scores are summarized in Supplementary Table 1. The data required by the scoring systems, demographic data, disease history, laboratory and imaging data were collected within 24 h of hospital admission. Medical record review and data extraction were performed by two trained researchers who were blinded to the study purpose.

Table 1 Baseline characteristics of the included patients.

Variables	Value
No. of patients	330
Age, mean ± SD, yr	54.9 ± 12.7
Male sex, n (%)	203 (61.5)
Etiology, n (%)
Viral	229 (69.4)
Alcohol	21 (6.4)
Autoimmune	34 (10.3)
Other	46 (13.9)
Laboratory tests, median (IQR)
White blood cells (109/L)	5.4 (3.6-7.8)
Hemoglobin (g/L)	80 (66-95)
Platelets (109/L)	77 (51-115)
INR	1.2 (1.1-1.4)
Total bilirubin (µmol/L)	22.6 (16.1-34.8)
ALT (IU/L)	23 (15-38)
AST (IU/L)	35 (25-56)
Albumin (g/L)	31.8 (28.4-35.3)
BUN (mmol/L)	8.7 (6.1-12.4)
Creatinine (µmol/L)	62.2 (48.5-77.2)
Sodium (mmol/L)	138.0 (134.9-140.3)
Vital signs, median (IQR)
SBP (mmHg)	110 (98-120)
DBP (mmHg)	65 (57-72)
Heart rate (beats/min)	88 (76-101)
Location of variceal bleeding, n (%)
Esophageal varices	295 (89.4)
Gastric varices	35 (10.6)
Grading of esophageal varices
Mild/ moderate/severe, n (%)	2 (0.7)/44 (14.9)/249 (84.4)
Types of gastric varices
GOV2/IGV1, n (%)	22 (62.9)/13 (37.1)
Ascites, n (%)	233 (70.6)
Hepatic encephalopathy, n (%)	28 (8.5)
Hepatocellular carcinoma, n (%)	73 (22.1)
Bacterial infection, n (%)	63 (19.1)
Portal vein thrombosis, n (%)	97 (29.4)
Hospital intervention, n (%)
Endoscopic therapy	141 (42.7)
TIPSS	21 (6.4)
Surgery	21 (6.4)
Charlson comorbidity index > 6, n (%)	45 (13.6)
Hospital stay (d), median (IQR)	13 (9-20)
Scoring system, median (IQR)
CTP	7 (6-9)
CTP grade A/B/C, n (%)	101 (30.6)/187 (56.7)/42 (12.7)
MELD	10 (9-13)
MELD-Na	12 (10-16)
CRS	2 (1-3)
GBS	12 (9-14)
mGBS	9 (7-11)
AIMS65	1 (0-1)
CANUKA	10 (8-12)

IQR: Interquartile range; INR: International normalized ratio; ALT: Alanine transaminase; AST: Aspartate transaminase; BUN: Blood urea nitrogen; SBP: Systolic blood pressure; DBP: Diastolic blood pressure; GOV2: Type 2 gastroesophageal varices; IGV1: Type 1 isolated gastric varices; TIPSS: Transjugular intrahepatic portosystemic stent shunt; CTP: Child-Turcotte-Pugh score; MELD: Model for end-stage liver disease; CRS: Clinical Rockall score; GBS: Glasgow-Blatchford score; mGBS: Modified Glasgow-Blatchford score; AIMS65: AIMS65 score; CANUKA: Canada-United Kingdom-Australia score.

Study outcomes

Patient follow-up began on the day of admission and ended at patient discharge or death during the same hospitalization period. The primary outcome was in-hospital rebleeding. Secondary outcomes were in-hospital mortality and a composite of in-hospital rebleeding and death. In-hospital rebleeding was defined as recurrence of hematemesis or melena accompanied by hemodynamic instability after the stabilization of vital signs and hemoglobin for at least 24 h. In-hospital mortality was defined as death due to any cause during hospitalization.

Statistical analysis

The sample size estimation was based on the number of positive and negative patients to assess the area under the receiver operating characteristic curve (AUROC). This study hypothesized that the validated scores can effectively predict the risk of in-hospital rebleeding; in other words, the AUROCs of the scores should be greater than 0.5. A previous study reported that the AUROCs of these scores were 0.664-0.756[9]. The minimum value (AUROC = 0.664) was selected as the reference value to obtain the maximum required sample size. The rebleeding rate was reported to be approximately 20% in cirrhotic patients with acute variceal bleeding[4,15]. Using PASS 11.0 software (NCSS, United States), 32 patients with in-hospital rebleeding and 128 nonrebleeding patients were required to achieve 90% power using a one-sided z-test at a significance level of 0.05[16]. Continuous variables with a normal distribution were reported as the mean ± SD and non-normal variables were presented as medians and interquartile ranges (IQRs). Categorical variables were expressed as counts and proportions. The discriminative ability of the prognostic scores was assessed using AUROC with a corresponding 95% confidence interval (CI), and an AUROC greater than 0.7 was considered clinically useful. Comparisons between paired AUROCs were performed using the DeLong test. The optimal threshold in each scoring system was determined by the maximum of the Youden index. The sensitivity, specificity, positive and negative predictive values, and corresponding 95%CIs were calculated for the clinically useful prognostic scores. The calibration of prognostic scores was evaluated by the Hosmer-Lemeshow (H-L) test. A Hosmer-Lemeshow P-value > 0.05 was considered to indicate good calibration. Calibration was also graphically analyzed for prognostic scores with high discriminative ability. Patients were stratified into different risk strata, and then the actual event probability was compared with the predicted event probability within the risk strata. In addition, sensitivity analyses were also performed focusing on patients with esophageal variceal bleeding and patients receiving endoscopic treatments. All data analyses were conducted using SPSS version 22.0 (IBM SPSS Statistics, United States) and MedCalc version 19.0.4 (MedCalc Software bvba, Belgium). A two-tailed P < 0.05 was considered statistically significant.

RESULTS

Patient characteristics

A total of 490 consecutive cirrhotic patients with acute UGIB were screened, and 160 patients were excluded for the following reasons: patients younger than 18 years of age (n = 1), endoscopy was refused or intolerant (n = 42), transferred patients (n = 42), patients with incomplete records (n = 10), and nonvariceal UGIB (n = 65). Finally, 330 independent patients with acute variceal bleeding were included based on the inclusion criteria (Figure 1). The characteristics of the included patients are listed in Table 1. The mean age of these patients was 54.9 ± 12.7 years (range, 25 to 85 years), and 203 patients (61.5%) were male. The vast majority of cirrhosis cases were caused by viral hepatitis; 54.8% patients had HBV, and 13.9% patients had HCV. Alcoholic and autoimmune cirrhosis accounted for 6.4% and 10.3% of the total, respectively. The location of variceal bleeding was esophageal varices in 89.4% of patients and gastric varices in 10.6% of patients. The proportion of severe esophageal varices was 84.4%. Type 1 gastroesophageal varices (GOV1) were classified into esophageal varices. Varices of the stomach fundus included 62.9% type 2 gastroesophageal varices (GOV2) and 37.1% type 1 isolated gastric varices (IGV1). A total of 69.4% of the patients with cirrhosis were categorized as CTP grade B or C. With regard to complications, 70.6% of cirrhotic patients had ascites, 8.5% had hepatic encephalopathy, 22.1% had hepatocellular carcinoma, 19.1% had bacterial infection, and 29.4% had portal vein thrombosis. The Charlson Comorbidity Index (CCI) was used for comorbidity assessment, and the proportion of patients with a CCI greater than 6 points was 13.6%. To control bleeding, 44.8% of patients received only medication, 42.4% received endoscopic treatments, and 12.8% received TIPSS or surgery. The median hospital stay was 13 (9-20) d.

Figure 1 Flowchart of the included patients.

In-hospital rebleeding

Sixty-seven patients underwent in-hospital rebleeding, and the hospital rebleeding rate was 20.3%. The median time interval between admission and rebleeding was 5 d. 40.3% of rebleeding events occurred within 3 d, 70.1% within 7 d, and 29.9% beyond 7 d (Table 2). For predicting in-hospital rebleeding, the AUROCs of the CTP, CRS, MELD-Na, MELD, CANUKA, AIMS65, GBS and mGBS scoring systems were 0.72, 0.72, 0.68, 0.66, 0.66, 0.64, 0.62 and 0.60, respectively (Figure 2A; Table 3). All AUROCs were statistically significant (P < 0.05). Pairwise comparisons of the AUROCs found no significant differences in discriminative ability among the CTP, CRS, MELD-Na, MELD and CANUKA (P > 0.05). Only the AUROCs of the CTP and CRS were clinically acceptable (AUROC > 0.7). Table 4 presents the diagnostic value indices for the clinically useful scoring systems. The cut-off points for the CTP and CRS were 7 and 2, respectively. The sensitivity, specificity, positive predictive value and negative predictive value for the CTP were 74.6%, 63.9%, 34.5% and 90.8%, respectively, and the corresponding values for the CRS were 65.7%, 74.1%, 39.3% and 89.4%, respectively. In addition, the calibration of each scoring system was good, and no significant difference was found between the actual and predicted probabilities (Table 3). A graphical analysis of the scoring system calibration showed a “good” goodness-of-fit for the CTP and CRS (Supplementary Figure 1).

Table 2 Clinical outcomes of the study population, n (%).

Outcomes	Value
In-hospital rebleeding	67 (20.3)
Admission to rebleeding (time interval, days), median (IQR)	5 (3-8)
Rebleeding occurred within 3 d	27 (40.3)
Rebleeding occurred within 7 d	47 (70.1)
Rebleeding occurred beyond 7 d	20 (29.9)
In-hospital mortality	35 (10.6)
In-hospital mortality with rebleeding	17 (25.4)
In-hospital mortality without rebleeding	18 (6.8)
In-hospital adverse outcomes	85 (25.8)

IQR: Interquartile range.

Figure 2 Area under the receiver operating curves for the risk scoring systems with regard to in-hospital rebleeding (A), in-hospital mortality (B), and in-hospital adverse outcomes (C). AUROC: Area under the receiver operating characteristic curve; CTP: Child-Turcotte-Pugh score; MELD: Model for end-stage liver disease; CRS: Clinical Rockall score; GBS: Glasgow-Blatchford score; mGBS: Modified Glasgow-Blatchford score; AIMS65: AIMS65 score; CANUKA: Canada-United Kingdom-Australia score.

Table 3 Discrimination and calibration assessments of all the scoring systems for in-hospital adverse outcomes.

Score	AUROC	95%CI	P value	AUROC difference	95%CI	P value	H-L test, P value
In-hospital rebleeding
CTP	0.717	0.648-0.787	< 0.001	Reference			0.134
CRS	0.716	0.638-0.793	< 0.001	0.0016	-0.0851-0.0883	0.9713	0.062
MELD-Na	0.680	0.609-0.752	< 0.001	0.0369	-0.0266-0.1000	0.2547	0.613
MELD	0.661	0.588-0.734	< 0.001	0.0560	-0.0026-0.1150	0.0610	0.386
CANUKA	0.656	0.579-0.732	< 0.001	0.0614	-0.0237- 0.1470	0.1575	0.186
AIMS65	0.642	0.565-0.719	< 0.001	0.0753	0.0120 - 0.1380	0.0196	0.321
GBS	0.617	0.534-0.699	0.003	0.1010	0.0153 - 0.1860	0.0208	0.041
mGBS	0.600	0.518-0.683	0.011	0.1170	0.0314 - 0.2020	0.0074	0.064
In-hospital mortality
CRS	0.883	0.822-0.943	< 0.001	Reference			0.166
CTP	0.878	0.825-0.931	< 0.001	0.0047	-0.0620-0.0714	0.8901	0.566
AIMS65	0.856	0.787-0.925	< 0.001	0.0269	-0.0390-0.0928	0.4232	0.175
MELD-Na	0.856	0.786-0.925	< 0.001	0.0271	-0.0453-0.0995	0.4630	0.636
MELD	0.840	0.767-0.914	< 0.001	0.0423	-0.0360- 0.1210	0.2900	0.472
mGBS	0.734	0.631-0.838	< 0.001	0.1480	0.0552-0.2420	0.0018	0.013
CANUKA	0.728	0.627-0.830	< 0.001	0.1540	0.0675-0.2410	0.0005	0.046
GBS	0.723	0.621-0.826	< 0.001	0.1590	0.0668-0.2520	0.0007	0.004
In-hospital adverse outcomes
CTP	0.784	0.725-0.843	< 0.001	Reference			0.218
CRS	0.775	0.709-0.841	< 0.001	0.0087	-0.0650-0.0824	0.8171	0.002
MELD-Na	0.739	0.675-0.803	< 0.001	0.0446	-0.0136-0.1030	0.1335	0.723
MELD	0.724	0.659-0.789	< 0.001	0.0598	0.0074-0.1120	0.0254	0.464
AIMS65	0.711	0.642-0.780	< 0.001	0.0728	0.0160-0.1300	0.0120	0.101
CANUKA	0.677	0.606-0.748	< 0.001	0.1070	0.0274-0.1860	0.0084	0.023
GBS	0.656	0.582-0.729	< 0.001	0.1280	0.0516-0.2050	0.0010	0.008
mGBS	0.649	0.574-0.723	< 0.001	0.1350	0.0586-0.2120	0.0005	0.024

AUROC: Area under the receiver operating characteristic curve; CI: Confidence interval; H-L: Hosmer-Lemeshow; CTP: Child-Turcotte-Pugh score; MELD: Model for end-stage liver disease; CRS: Clinical Rockall score; GBS: Glasgow-Blatchford score; mGBS: Modified Glasgow-Blatchford score; AIMS65: AIMS65 score; CANUKA: Canada-United Kingdom-Australia score.

Table 4 The predictive value indices for predicting in-hospital adverse outcomes in clinically useful scoring systems.

Score	Youdenindex	Cut-off	SEN	95%CI	SPE	95%CI	PPV	95%CI	NPV	95%CI
In-hospital rebleeding
CTP	0.3851	> 7	74.6	62.5-84.5	63.9	57.8-69.7	34.5	29.8-39.4	90.8	86.6-93.8
CRS	0.3982	> 2	65.7	53.1-76.8	74.1	68.4-79.3	39.3	33.1-45.8	89.4	85.8-92.2
In-hospital mortality
CRS	0.6751	> 2	94.3	80.8-99.3	73.2	67.8-78.2	29.5	25.4-33.9	99.1	96.6-99.8
CTP	0.5632	> 7	94.3	80.8-99.3	62.0	56.2-67.6	22.8	20.0-25.8	98.9	96.0-99.7
AIMS65	0.6339	> 1	80.0	63.1-91.6	83.4	78.6-87.5	36.4	29.6-43.7	97.2	94.8-98.6
MELD-Na	0.5685	> 17	71.4	53.7-85.4	85.4	80.9-89.2	36.8	29.1-45.1	96.2	93.7-97.7
MELD	0.5157	> 12	74.3	56.7-87.5	77.3	72.1-81.9	28.0	22.6-34.1	96.2	93.5-97.8
mGBS	0.4349	> 10	68.6	50.7-83.1	74.9	69.6-79.8	24.5	19.4-30.4	95.3	92.5-97.1
CANUKA	0.3521	> 12	45.7	28.8-63.4	89.5	85.4-92.7	34.0	24.0-45.8	93.3	91.1-95.0
GBS	0.3700	> 12	74.3	56.7-87.5	62.7	56.9-68.2	19.1	15.6-23.2	95.4	92.1-97.3
In-hospital adverse outcomes
CTP	0.4699	> 7	78.8	68.6-86.9	68.2	61.9-73.9	46.2	41.0-51.5	90.3	85.9-93.4
CRS	0.5095	> 2	71.8	61.0-81.0	79.2	73.6-84.1	54.5	47.5-61.2	89.0	85.1-91.9
MELD-Na	0.3758	> 13	69.4	58.5-79.0	68.2	61.9-73.9	43.1	37.5-48.8	86.5	82.2-89.9
MELD	0.3241	> 13	45.9	35.0-57.0	86.5	81.6-90.5	54.2	44.4-63.6	82.2	79.0-84.9
AIMS65	0.3830	> 1	51.8	40.7-62.7	86.5	81.6-90.5	57.1	47.7-66.1	83.8	80.5-86.6

SEN: Sensitivity; SPE: Specificity; PPV: Positive predictive value; NPV: Negative predictive value; CI: Confidence interval; CTP: Child-Turcotte-Pugh score; MELD: Model for end-stage liver disease; CRS: Clinical Rockall score; GBS: Glasgow-Blatchford score; mGBS: Modified Glasgow-Blatchford score; AIMS65: AIMS65 score; CANUKA: Canada-United Kingdom-Australia score.

In-hospital mortality

Thirty-five patients died during hospitalization. The in-hospital mortality rate was 10.6% in all patients, 25.4% in patients with in-hospital rebleeding and 6.8% in patients without in-hospital rebleeding (Table 2). Only one patient died of extrahepatic disease; the cause of death in the other patients was variceal bleeding or organ failure. The CRS, CTP, AIMS65, MELD-Na and MELD showed excellent discriminative ability; their AUROCs were greater than 0.8 and statistically significant (Figure 2B; Table 3). Furthermore, pairwise comparisons found no significant differences in these scoring systems. The AUROCs of mGBS, CANUKA and GBS were relatively small, but clinically acceptable (AUROCs > 0.7). The diagnostic value indices for predicting in-hospital mortality are presented in Table 4. Moreover, the calibration of all scoring systems was excellent (Table 3). The graphical analysis showed similar results for the CTP and CRS (Supplementary Figure 2).

In-hospital adverse outcomes

In-hospital adverse outcomes included rebleeding and death events. A total of 85 (25.8%) patients suffered from in-hospital adverse events (Table 2). The discriminative abilities of CTP, CRS, MELD-NA, MELD and AIMS65 were found to be clinically useful (AUROCs > 0.7) (Figure 2C; Table 3). Pairwise comparisons showed no significant differences among the CTP, CRS and MELD-Na scoring systems (P > 0.05). However, the CTP was superior to the MELD, AIMS65, CANUKA, GBS and mGBS in predicting in-hospital adverse outcomes (P < 0.05). The diagnostic value indices for predicting in-hospital adverse outcomes are presented in Table 4. The calibration of all scoring systems was good, except for the CRS (P < 0.05) (Table 3). The graphical analysis showed similar results for the CTP and CRS, but the CRS may underestimate the risk of adverse outcomes in high-risk strata (3-6) (Supplementary Figure 3).

Sensitivity analysis

In the sensitivity analysis of patients with esophageal variceal bleeding, the results were almost unchanged. The CTP and CRS remained the two best scoring systems for predicting in-hospital outcomes. For predicting in-hospital rebleeding, the AUROCs of the CTP and CRS were 0.75 (0.68-0.82) and 0.72 (0.64-0.80), respectively. For predicting in-hospital mortality and in-hospital adverse outcomes, the AUROCs of the CTP and CRS were 0.88 (0.83-0.94) and 0.89 (0.83-0.95), 0.81 (0.75-0.87) and 0.78 (0.71-0.85), respectively. In addition, the calibration of the CTP and CRS was good for predicting in-hospital rebleeding or mortality (P > 0.1). When focusing on patients who received endoscopic treatments, only the CTP was statistically significant for predicting in-hospital outcomes. The AUROC of the CTP was 0.70 (0.55-0.84) for predicting in-hospital rebleeding, 0.79 (0.63-0.94) for in-hospital mortality and 0.71 (0.57-0.84) for in-hospital adverse outcomes. The calibration of the CTP was good for predicting any in-hospital outcome (P > 0.1).

DISCUSSION

The results of the present study revealed that these scoring systems could effectively predict the occurrence of in-hospital adverse outcomes in cirrhotic patients with variceal bleeding. For in-hospital rebleeding, all scoring systems were able to distinguish whether in-hospital rebleeding occurred, and the calibration ability of these scores was good. However, only the CTP and CRS were clinically acceptable in terms of their discriminative ability. For in-hospital mortality, the CRS, CTP, AIMS65, MELD, and MELD-Na showed excellent discriminative and calibration abilities. The discriminative ability of the other prognostic scoring systems (GBS, mGBS and CANUKA) was slightly poor, but clinically acceptable.

Acute variceal bleeding is one of the most serious complications in patients with cirrhosis. This study found that the rate of in-hospital rebleeding and mortality could be as high as 20.3% and 10.6%, respectively, and the mortality rate was higher in patients with in-hospital rebleeding than in those without. These findings are similar to the results reported by previous studies[4,15]. Considering the harmfulness of variceal bleeding, appropriate risk stratification is critical for the optimal management of these patients. Close monitoring and aggressive treatment should be considered for high-risk patients. However, some ideal prognostic scores are controversial due to poor external validation. In fact, the CRS, AIMS65, CANUKA and GBS scoring systems were established independently with different study purposes and populations. The GBS was developed and used to predict UGIB patients’ risk of requiring blood transfusion or intervention, decreased hemoglobin, rebleeding and mortality. Similarly, the CANUKA was used to identify high-risk patients with 30-day rebleeding or death, radiologic or surgical intervention for bleeding control, and the need for therapeutic endoscopy or transfusion. Both the GBS and CANUKA are recommended for screening patients for hospital intervention or outpatient treatment[11,17]. The mGBS is similar to the GBS; it was developed by removing the subjective variables of the GBS[10]. In contrast, the CRS and AIMS65 were developed to determine the risk of in-hospital rebleeding or mortality in patients with UGIB[18,19]. All of the above prognostic scores included unselected UGIB as the research subject, and both patients with variceal bleeding and those with nonvariceal bleeding were enrolled for analysis. However, the proportion of patients with variceal bleeding was very low. Therefore, the predictive performance of these scores in predicting the risk of rebleeding or mortality for patients with variceal hemorrhage is unclear. Stanley et al[16] performed an international multicenter prospective study and found that the GBS was the best scoring system in predicting the need for intervention (transfusion, endoscopic treatment, interventional or surgical intervention) or death. According to their study, the latest guideline recommended that a GBS score of ≤ 1 could be used to identify low-risk patients for nonvariceal UGIB[20]. However, their study only included a few patients with variceal bleeding (7%) and did not perform a subgroup analysis for this group of patients. Gaduputi et al[21] reported that the AIMS65 score may be as useful as the Rockall score for predicting the risk of rebleeding and death in noncirrhotic patients. External validation studies confirmed that these scores had poorer predictive ability in patients with variceal bleeding than in those with nonvariceal bleeding[22,23].

For predicting in-hospital adverse outcomes, few studies have explored the usefulness of these scores in patients with variceal hemorrhage, and the conclusions are controversial. Motola-Kuba et al[9] found that the GBS was better at predicting in-hospital rebleeding than the Rockall score (RS), AIMS65, CTP and MELD. Sarwar et al[8] showed that the Rockall score had good discriminative value for predicting in-hospital rebleeding. However, Choe et al[7] and Jairath et al[4] asserted that the GBS, CRS and AIMS65 had limited ability for predicting the risk of in-hospital rebleeding, with AUROCs of approximately 0.6. The present study was performed in Chinese patients, included more comprehensive scoring systems, and found that these scores, apart from the CTP and CRS, had poor predictive ability. In addition, the AUROCs of the CTP and MELD were the same as those reported in a study from South Korea[24]. These differences among different studies are understandable as these studies were conducted in different countries or regions and enrolled patients with different characteristics. For predicting in-hospital mortality, the AUROCs of these scores also varied between different studies. Compared with predicting in-hospital rebleeding, previous studies have generally reported that these scores were better at predicting the risk of in-hospital death[8,9,24,25]. These findings are consistent with our results. In fact, the component variables of these scores indicate that they are more suitable for predicting short-term death rather than rebleeding. The variables included in these scores are age, systolic blood pressure, heart rate, hemoglobin, comorbidity, albumin, international normalized ratio and blood urea nitrogen (Supplementary Table 1). Previous studies have found that these variables were independently associated with short-term mortality in patients with cirrhosis[26,27]. In contrast, most of these variables have not been confirmed to be associated with short-term rebleeding. Some independent factors, such as ascites, portal vein thrombosis and portal hypertension, were identified to be associated with early variceal rebleeding, but they are not included in the validated scores[26,28,29]. In addition, many studies have shown that the CTP, MELD and MELD-Na have good predictive values in predicting in-hospital death, and our study confirms this finding[30].

This study has some strengths. First, a relatively adequate sample size helped to evaluate the predictive value of these scores. Second, most risk scoring systems lack external validation, especially in Chinese patients. Third, some of these prognostic scoring systems were validated for in-hospital adverse outcomes for the first time. Finally, our study provides some evidence in Chinese patients. However, some limitations should also be mentioned. First, the present study was a single-center retrospective study, and the applicability of the results may be limited. Second, not all patients were treated following the current guidelines. However, our data were obtained from clinical records and reflected real clinical practices. Third, some transferred patients were excluded because some of the data could change after external treatments, and the data from other hospitals were not available. Fourth, some endoscopy-based scores were not considered in this study because a risk assessment was delayed or even unachievable in some healthcare settings using an endoscopy score. Furthermore, most endoscopic data are subjective. In addition, some subgroup or sensitivity analyses could not be performed because some the relevant data were not available or the effective sample size was insufficient after patients were split into several groups. The predictive values of these prognostic scores require validation based on different degrees of portal hypertension, grading and types of varices, and types of special treatment. In conclusion, the risk of in-hospital rebleeding and mortality remains high in cirrhotic patients with variceal bleeding. The predictive value of the CTP and CRS are clinically acceptable for predicting in-hospital rebleeding. The performances of these scoring systems are better at predicting in-hospital mortality than in-hospital rebleeding, especially the CRS, CTP, AIMS65, MELD-Na and MELD. Further prospective and multicenter studies are warranted to confirm our findings.

ARTICLE HIGHLIGHTS

Research background

Several risk scoring systems have been developed and are regarded as useful tools for predicting clinical outcomes in patients with upper gastrointestinal bleeding (UGIB). As a common form of acute UGIB, patients with variceal bleeding often have an increased risk of in-hospital adverse outcomes. Data are limited regarding the predictive value of these risk scoring systems for patients with variceal bleeding.

Research motivation

Variceal bleeding is a serious complication of cirrhosis, and discovering valuable prognostic scores will be useful for early identification of high-risk patients. These patients will benefit if necessary measures are taken timely.

Research objectives

The present study aimed to validate the predictive value of eight scoring systems for in-hospital outcomes in cirrhotic patients with variceal bleeding.

Research methods

Consecutive patients with acute variceal bleeding, from March 2017 to June 2019, were included at the Second Affiliated Hospital of Xi’an Jiaotong University. By reviewing medical records, required data were collected and prognostic scores were calculated for the clinical Rockall score (CRS), AIMS65 score (AIMS65), Glasgow-Blatchford score (GBS), modified GBS (mGBS), Canada-United Kingdom-Australia score (CANUKA), Child-Turcotte-Pugh score (CTP), model for end-stage liver disease (MELD) and MELD-Na. The discriminative ability of these prognostic scores was assessed using the area under the receiver operating characteristic curve (AUROC), and the calibration was evaluated by the Hosmer-Lemeshow (H-L) test.

Research results

We retrospectively enrolled 330 cirrhotic patients with variceal bleeding. The rate of in-hospital rebleeding for these patients was 20.3%, and the rate of in-hospital mortality was 10.6%. For predicting in-hospital rebleeding, although all AUROCs of these prognostic scores were statistically significant, only the AUROCs of the CTP and CRS were clinically acceptable (AUROC > 0.7). The calibration of all prognostic scores for in-hospital rebleeding was good. For predicting in-hospital mortality, all AUROCs of these prognostic scores were good with statistical significance, especially the CRS, CTP, AIMS65, MELD-Na and MELD (AUROCs > 0.8). The calibration of all prognostic scores for in-hospital mortality was also good.

Research conclusions

The risk of in-hospital adverse outcomes remains high in cirrhotic patients with variceal bleeding. The CTP and CRS have acceptable abilities for predicting in-hospital rebleeding. All of these prognostic scores are useful for predicting in-hospital mortality, especially the CRS, CTP, AIMS65, MELD-Na and MELD. Clinicians from hospitals of different grades can select suitable models for early identification of high-risk patients.

Research perspectives

The predictive value of these prognostic scores still need to be confirmed in patients with special risk factors, such as gastric variceal bleeding, high portal pressure and those receiving special treatments. Predictive models with high accuracy need to be established for predicting in-hospital rebleeding taking into account the limitations of existing models.