Li YL, Zhang DD, Xiong YY, Wang RF, Gao XM, Gong H, Zheng SC, Wu D. Development and external validation of models to predict acute respiratory distress syndrome related to severe acute pancreatitis. World J Gastroenterol 2022; 28(19): 2123-2136 [PMID: 35664037 DOI: 10.3748/wjg.v28.i19.2123]
Corresponding Author of This Article
Dong Wu, MD, Professor, Department of Gastroenterology, Peking Union Medical College Hospital, No. 1 Shuaifuyuan Wangfujing, Dongcheng District, Beijing 100730, China. wudong061002@aliyun.com
Research Domain of This Article
Gastroenterology & Hepatology
Article-Type of This Article
Retrospective Cohort Study
Open-Access Policy of This Article
This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Yun-Long Li, Yang-Yang Xiong, Dong Wu, Department of Gastroenterology, Peking Union Medical College Hospital, Beijing 100730, China
Ding-Ding Zhang, Medical Research Center, Peking Union Medical College Hospital, Beijing 100730, China
Ding-Ding Zhang, Dong Wu, Clinical Epidemiology Unit, International Clinical Epidemiology Network, Beijing 100730, China
Yang-Yang Xiong, Department of Gastroenterology, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou 310003, Zhejiang Province, China
Rui-Feng Wang, Department of Gastroenterology, The Fourth Affiliated Hospital of Harbin Medical University, Harbin 150001, Heilongjiang Province, China
Xiao-Mao Gao, Department of Gastroenterology, The Sixth Hospital of Beijing, Beijing 100191, China
Hui Gong, Shi-Cheng Zheng, Department of Gastroenterology, West China Longquan Hospital Sichuan University, Chengdu 610100, Sichuan Province, China
Author contributions: Li YL, Zhang DD, and Xiong YY contributed equally to this work; Li YL, Zhang DD, Xiong YY and Wu D designed the research study; Li YL, Xiong YY, Wang RF, Gao XM, Gong H, Zheng SC, and Wu D performed the study and collected the data; Li YL, Zhang DD, and Xiong YY analyzed the data and wrote the manuscript; All authors have read and approved the final manuscript.
Supported bythe Chinese Natural Science Foundation, No. 32170788.
Institutional review board statement: This study was approved by the Ethics Committee of Peking Union Medical College Hospital (Approval No. S-K1772).
Conflict-of-interest statement: All authors have no conflict of interest to disclose.
Data sharing statement: No additional data are available.
STROBE statement: The authors have read the STROBE statement, and the manuscript was prepared and revised according to the STROBE statement.
Open-Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: https://creativecommons.org/Licenses/by-nc/4.0/
Corresponding author: Dong Wu, MD, Professor, Department of Gastroenterology, Peking Union Medical College Hospital, No. 1 Shuaifuyuan Wangfujing, Dongcheng District, Beijing 100730, China. wudong061002@aliyun.com
Received: December 4, 2021 Peer-review started: December 4, 2021 First decision: January 27, 2022 Revised: February 9, 2022 Accepted: April 3, 2022 Article in press: April 3, 2022 Published online: May 21, 2022 Processing time: 164 Days and 5.6 Hours
Abstract
BACKGROUND
Acute respiratory distress syndrome (ARDS) is a major cause of death in patients with severe acute pancreatitis (SAP). Although a series of prediction models have been developed for early identification of such patients, the majority are complicated or lack validation. A simpler and more credible model is required for clinical practice.
AIM
To develop and validate a predictive model for SAP related ARDS.
METHODS
Patients diagnosed with AP from four hospitals located at different regions of China were retrospectively grouped into derivation and validation cohorts. Statistically significant variables were identified using the least absolute shrinkage and selection operator regression method. Predictive models with nomograms were further built using multiple logistic regression analysis with these picked predictors. The discriminatory power of new models was compared with some common models. The performance of calibration ability and clinical utility of the predictive models were evaluated.
RESULTS
Out of 597 patients with AP, 139 were diagnosed with SAP (80 in derivation cohort and 59 in validation cohort) and 99 with ARDS (62 in derivation cohort and 37 in validation cohort). Four identical variables were identified as independent risk factors for both SAP and ARDS: heart rate [odds ratio (OR) = 1.05; 95%CI: 1.04-1.07; P < 0.001; OR = 1.05, 95%CI: 1.03-1.07, P < 0.001], respiratory rate (OR = 1.08, 95%CI: 1.0-1.17, P = 0.047; OR = 1.10, 95%CI: 1.02-1.19, P = 0.014), serum calcium concentration (OR = 0.26, 95%CI: 0.09-0.73, P = 0.011; OR = 0.17, 95%CI: 0.06-0.48, P = 0.001) and blood urea nitrogen (OR = 1.15, 95%CI: 1.09-1.23, P < 0.001; OR = 1.12, 95%CI: 1.05-1.19, P < 0.001). The area under receiver operating characteristic curve was 0.879 (95%CI: 0.830-0.928) and 0.898 (95%CI: 0.848-0.949) for SAP prediction in derivation and validation cohorts, respectively. This value was 0.892 (95%CI: 0.843-0.941) and 0.833 (95%CI: 0.754-0.912) for ARDS prediction, respectively. The discriminatory power of our models was improved compared with that of other widely used models and the calibration ability and clinical utility of the prediction models performed adequately.
CONCLUSION
The present study constructed and validated a simple and accurate predictive model for SAP-related ARDS in patients with AP.
Core Tip: Severe acute pancreatitis (SAP)-related acute respiratory distress syndrome (ARDS) affect the mortality of patients with AP. Early identification of patients at high risk for SAP and ARDS can aid clinicians to adopt interventions to stop disease progression. However, current predictive models are either too complicated due to various parameters or unreliable due to lack of validation. This study developed new models to predict SAP and ARDS using only four routine clinical items within 24 h of admission. New models were externally validated and performed as well as or with a higher efficiency than other models.
Citation: Li YL, Zhang DD, Xiong YY, Wang RF, Gao XM, Gong H, Zheng SC, Wu D. Development and external validation of models to predict acute respiratory distress syndrome related to severe acute pancreatitis. World J Gastroenterol 2022; 28(19): 2123-2136
Acute pancreatitis (AP) is a common event incurring pain, socioeconomic loss, and even death. The majority of the patients who present with mild organ injury and self-limited course are diagnosed with mild acute pancreatitis (MAP) or moderately severe acute pancreatitis (MSAP)[1,2]. However, it is estimated that approximately 20% of patients are critically ill and develop SAP, leading to consistent organ failure and significant mortality[2,3]. Our previous studies indicated that the lung are the most commonly affected organs in SAP[4,5], and acute respiratory distress syndrome (ARDS) is recognized as an important cause of respiratory failure, with a high mortality rate[6-8]. It is reported that 4%-15% of AP patients are complicated with ARDS[9], while this proportion might be as high as one third in SAP[10]. However, to date, the therapeutic options for SAP and ARDS are limited. Therefore, it is necessary to identify patients at risk and adopt interventions to prevent MAP or MSAP from progressing to SAP and ARDS. The protective effect of early intervention for patients with predicted SAP or patients at risk of ARDS has been confirmed by numerous clinical trials and meta-analyses, although the inclusion criteria for patients have varied according to different studies[11-15].
A plethora of models have been published to predict the risk of SAP in AP patients, including acute physiology and chronic health evaluation II (APACHE-II) score, Ranson criteria, computed tomography severity index (CTSI), and bedside index for severity in acute pancreatitis (BISAP)[16,17]. Lung injury prediction score (LIPS) and other models have also been used to evaluate the risk of ARDS in patients with non-AP[18-20]. However, to date the models used to predict ARDS in AP are scarce. Furthermore, the majority of the SAP predictive models are hard to use in practice due to various parameters, complicated calculation and dependence on radiological assessment. The majority of the models also lack internal or external validation, which reduces their reliability in other cohorts[16]. Therefore, a new concise model may be more practical in the emergency department in order to identify SAP and ARDS in the early course of AP. This model should involve limited available clinical data and should not rely on radiological examinations.
The objective of the present study was to develop and validate models to predict SAP and ARDS in patients with AP based on multicenter retrospective cohorts. The comparison of novel models with quick sequential organ failure assessment (qSOFA), systemic inflammatory response syndrome (SIRS) and BISAP is essential to display the power of different models with a low number of variables. These models usually contain three, four and five items, respectively[21,22].
MATERIALS AND METHODS
Study design and participants
This was a multicenter retrospective study. Sample size was calculated with PASS 11.0. The proportion of SAP was set to 20%, and the incidence of ARDS in SAP and non-SAP patients was set to 1/3 and 4%, respectively. Considering that the dropout rate was low in hospitals, we set it to 5%. In the end, with α = 0.01 and β = 0.10, a total of 211 participants were needed. Patients diagnosed with AP between 1 January 2017 and 31 December 2019 were recruited from different regions of China (Peking Union Medical College Hospital and The Sixth Hospital of Beijing at Northern China, The Fourth Affiliated Hospital of Harbin Medical University at Northeastern China, West China Longquan Hospital Sichuan University at Southwestern China). The patients were categorized into the derivation cohort in order to develop a clinical predictive model. The independent external validation cohort consisted of patients diagnosed with AP between 1 January 2020 and 31 May 2021 at the Peking Union Medical College Hospital. AP was diagnosed if at least two of the three following criteria were met: (1) abdominal pain consistent with AP; (2) serum lipase or amylase levels that were more than three times the upper limit of the normal range; and (3) characteristic radiological findings of AP on computed tomography (CT) and magnetic resonance imaging or ultrasonography[23]. SAP was identified by the presence of persistent organ failure for > 48 h[23]. ARDS was diagnosed based on Berlin definition[24]. Patients aged < 18 years who lacked the necessary information provided by the Atlanta Classification or relevant etiology information were excluded.
Clinical variables
The following demographic and laboratory data were collected from the electronic medical record system within 24 h of admission: age, sex, temperature, heart rate (HR), respiratory rate (RR), systolic blood pressure, Glasgow coma score (GCS), white blood counts, hematocrit, platelet, serum electrolyte concentration (K, Na and Ca), creatine, blood urea nitrogen (BUN) and glucose. Other clinical information, such as, admission date, local complication, length of hospital stay, length of intensive care unit stay, mortality and ventilator use, was also collected. Weekend admission corresponded to admission on Saturday or Sunday and local complication included acute peripancreatitc fluid collection, acute necrosis collection, pseudocyst and walled-off necrosis. The ventilator included invasive or noninvasive mechanical ventilation. qSOFA, SIRS and BISAP scores were calculated based on the aforementioned data. All data were collected and checked by two or more authors independently. Missing items were added following review of the clinical records. The data that could not be completed were removed and the complete-case dataset was finally analyzed.
Statistical analysis
Categorical variables were expressed as frequencies and percentages and compared using χ2 or Fisher’s exact tests. Normally distributed continuous variables were described as the mean ± SD and compared using a two-sided Student’s t test. Non-normally distributed continuous variables were expressed as the median with the interquartile range and compared using the Mann-Whitney U test. Continuous variables were analyzed in their original forms to preserve information[25]. The least absolute shrinkage and selection operator (LASSO) regression method was used to select predictors in the derivation cohort. The predictive models were further built using multiple logistic regression analysis. The nomogram was formulated based on multivariate logistic regression analysis. Receiver operating characteristic (ROC) curve and the area under ROC curve (AUC) were used to evaluate the discriminative power of the predictive model, which referred to the ability of the model to differentiate between the subjects that did or did not experience the outcome event[25]. The calibration curves were plotted to measure the predictive accuracy of the model, which reflected the agreement between predictions from the model and observed outcomes. A well-calibrated model indicated that the prediction was lying on or around the 45° line of the calibration plot[25]. Hosmer-Lemeshow (H-L) goodness of fit test was used to quantify the calibration curve. The P value was determined by the H-L test. P > 0.05 suggested an optimal consistency between model prediction and the criteria required for standard diagnosis. Decision curve analysis (DCA) was used to assess the clinical utility of the model, which indicated he relationship between a model-predicted probability threshold and the relative value of net benefit[25].
Statistical analysis was performed using R 4.0.3[26] and MedCalc 15.8 software. A two-side P < 0.05 was considered to indicate a statistically significant difference. The nomogram and calibration curve were plotted using rms package and DCA was plotted using rmda package. ROC was plotted with MedCalc 15.8.
RESULTS
Baseline characteristics
Between January 1 2017 and May 31 2021, 628 patients with AP were recruited from four hospitals and reviewed. The exclusion criteria included the following: age < 18 years (2 patients), lack of Atlanta Classification or etiology (26 patients) and incomplete data (3 patients). Following screening, 407 and 190 patients were involved in the derivation and validation cohorts, respectively. The number of participants in each cohort met the requirement of sample size. The detailed demographic and clinical information were described in Tables 1 and 2.
Table 1 Characteristics of non-severe acute pancreatitis and severe acute pancreatitis patients in derivation and validation cohort.
Characteristic
Derivation cohort
P value
Validation cohort
P value
non-SAP (n = 327)
SAP (n = 80)
non-SAP (n = 131)
SAP (n = 59)
Age (yr)
49.0 (36.0; 63.5)
46.0 (34.8; 62.2)
0.424
40.0 (32.0; 58.5)
44.0 (38.0; 53.0)
0.371
Female (%)
131 (40.1)
27 (33.8)
0.363
63 (48.1)
14 (23.7)
0.003
Weekend admission (%)
65 (19.9)
26 (32.5)
0.023
37 (28.2)
13 (22.0)
0.471
Etiology (%)
0.021
0.003
Biliary
113 (34.6)
31 (38.8)
54 (41.2)
11 (18.6)
Hypertriglyceridemia
113 (34.6)
36 (45.0)
44 (33.6)
21 (35.6)
Alcoholic
27 (8.26)
7 (8.75)
5 (3.82)
8 (13.6)
Others
74 (22.6)
6 (7.50)
28 (21.4)
19 (32.2)
Peritoneal irritation (%)
74 (22.6)
31 (38.8)
0.005
27 (20.6)
32 (54.2)
< 0.001
GCS < 15 (%)
2 (0.61)
5 (6.25)
0.004
3 (2.29)
7 (11.9)
0.011
Tem (℃)
36.6 (36.4; 37.0)
37.2 (36.5; 38.2)
< 0.001
37.0 (36.8; 37.4)
38.0 (37.5; 38.4)
< 0.001
SBP (KPa)
17.2 (15.9; 18.7)
16.4 (14.9; 18.8)
0.119
16.3 (15.2; 18.4)
16.8 (14.7; 19.5)
0.949
HR (bpm)
82.0 (76.0; 94.5)
118 (99.5; 132)
< 0.001
89.0 (79.0; 102)
110 (100; 130)
< 0.001
RR (bpm)
20.0 (18.0; 20.0)
22.0 (20.0; 28.2)
< 0.001
20.0 (18.0; 21.0)
23.0 (20.5; 29.5)
< 0.001
WBC (× 109/L)
11.6 (8.74; 15.2)
14.2 (10.2; 18.7)
0.001
12.4 (8.70; 15.6)
16.4 (12.2; 21.1)
< 0.001
HCT (%)
40.8 (37.5; 44.8)
43.6 (36.9; 48.4)
0.022
41.3 (37.8; 45.0)
42.9 (36.5; 49.0)
0.218
PLT (×109/L)
221 (178; 276)
192 (156; 274)
0.03
232 (196; 278)
202 (166; 264)
0.025
K (mmol/L)
3.90 (3.60; 4.25)
3.90 (3.50; 4.30)
0.817
3.90 (3.50; 4.10)
4.00 (3.50; 4.55)
0.074
Na (mmol/L)
137 (134; 140)
137 (132; 140)
0.909
137 (133; 140)
137 (133; 140)
0.664
Ca (mmol/L)
2.26 (2.13; 2.34)
1.96 (1.74; 2.16)
< 0.001
2.22 (2.08; 2.32)
1.93 (1.77; 2.10)
< 0.001
Cr (μmol/L)
67.0 (55.0; 80.0)
90.0 (70.8; 142)
< 0.001
61.0 (52.0; 77.0)
90.0 (62.5; 227)
< 0.001
BUN (mmol/L)
5.00 (3.88; 6.34)
7.06 (5.26; 10.4)
< 0.001
4.66 (3.32; 5.84)
6.67 (4.91; 14.2)
< 0.001
Glu (mmol/L)
8.00 (6.50; 12.9)
11.0 (7.45; 16.3)
< 0.001
8.70 (6.80; 11.8)
11.3 (8.05; 17.2)
0.004
Local complication (%)
104 (31.8)
73 (91.2)
< 0.001
66 (50.4)
55 (93.2)
< 0.001
ICU admission (%)
12 (3.67)
37 (46.2)
< 0.001
12 (9.16)
38 (64.4)
< 0.001
ICU stays (d)
0.00 (0.00; 0.00)
0.00 (0.00; 10.0)
< 0.001
0.00 (0.00; 0.00)
5.00 (0.00; 9.00)
< 0.001
Hospital stays (d)
10.0 (7.00; 14.0)
17.5 (8.75; 25.0)
< 0.001
9.00 (4.00; 15.0)
23.0 (15.0; 30.0)
< 0.001
Mortality (%)
0 (0.00)
3 (3.75)
0.007
1 (0.76)
4 (6.78)
0.033
qSOFA
0.00 (0.00; 0.00)
1.00 (0.00; 1.00)
< 0.001
0.00 (0.00; 1.00)
1.00 (0.00; 1.00)
< 0.001
SIRS
1.00 (0.00; 2.00)
3.00 (2.00; 3.00)
< 0.001
1.00 (0.00; 2.00)
3.00 (2.00; 3.00)
< 0.001
BISAP
1.00 (0.00; 1.00)
2.00 (2.00; 3.00)
< 0.001
1.00 (0.00; 2.00)
2.00 (2.00; 3.00)
< 0.001
Table 2 Characteristics of non-acute respiratory distress syndrome and acute respiratory distress syndrome patients in derivation and validation cohorts.
Characteristic
Derivation cohort
P value
Validation cohort
P value
non-ARDS (n = 345)
ARDS (n = 62)
non-ARDS (n = 153)
ARDS (n = 37)
Age (yr)
49.0 (36.0; 64.0)
42.0 (33.2; 59.5)
0.062
40.0 (33.0; 57.0)
48.0 (41.0; 57.0)
0.082
Female (%)
138 (40.0)
20 (32.3)
0.312
67 (43.8)
10 (27.0)
0.093
Weekend admission (%)
73 (21.2)
18 (29.0)
0.228
39 (25.5)
11 (29.7)
0.751
Etiology (%)
0.024
0.187
Biliary
122 (35.4)
22 (35.5)
56 (36.6)
9 (24.3)
Hypertriglyceridemia
119 (34.5)
30 (48.4)
53 (34.6)
12 (32.4)
Alcoholic
28 (8.12)
6 (9.68)
8 (5.23)
5 (13.5)
Others
76 (22.0)
4 (6.45)
36 (23.5)
11 (29.7)
Peritoneal irritation (%)
80 (23.2)
25 (40.3)
0.007
39 (25.5)
20 (54.1)
0.002
GCS < 15 (%)
4 (1.16)
3 (4.84)
0.075
4 (2.61)
6 (16.2)
0.004
Tem (℃)
36.6 (36.4; 37.0)
37.5 (36.5; 38.2)
< 0.001
37.0 (36.8; 37.5)
38.0 (37.8; 38.5)
< 0.001
SBP (mmHg)
129 (119; 140)
123 (110; 141)
0.113
123 (116; 140)
119 (106; 147)
0.192
HR (bpm)
82.0 (76.0; 96.0)
120 (103; 134)
< 0.001
90.0 (80.0; 110)
115 (100; 140)
< 0.001
RR (bpm)
20.0 (18.0; 20.0)
23.0 (20.0; 30.0)
< 0.001
20.0 (18.0; 22.0)
24.0 (20.0; 29.0)
< 0.001
WBC (× 109/L)
11.9 (8.76; 15.3)
13.9 (9.91; 19.5)
0.006
12.5 (9.07; 16.3)
17.6 (13.6; 22.8)
< 0.001
HCT (%)
40.8 (37.5; 44.8)
45.2 (38.3; 48.7)
0.002
41.3 (37.4; 45.2)
43.9 (38.0; 49.4)
0.094
PLT (× 109/L)
218 (176; 275)
207 (167; 274)
0.26
228 (192; 278)
208 (168; 261)
0.112
K (mmol/L)
3.90 (3.60; 4.30)
3.90 (3.50; 4.20)
0.441
3.90 (3.50; 4.20)
3.90 (3.50; 4.40)
0.429
Na (mmol/L)
137 (134; 140)
137 (132; 140)
0.95
137 (133; 140)
139 (132; 144)
0.038
Ca (mmol/L)
2.25 (2.12; 2.34)
1.88 (1.65; 2.12)
< 0.001
2.16 (2.01; 2.29)
1.96 (1.77; 2.22)
0.002
Cr (μmol/L)
67.0 (55.0; 81.0)
86.5 (72.0; 134)
< 0.001
62.0 (53.0; 79.0)
96.0 (65.0; 230)
< 0.001
BUN (mmol/L)
5.03 (3.90; 6.44)
7.28 (5.25; 10.3)
< 0.001
4.80 (3.37; 6.48)
6.97 (5.37; 14.4)
< 0.001
Glu (mmol/L)
8.10 (6.50; 13.1)
11.0 (7.67; 16.1)
< 0.001
8.80 (6.90; 13.6)
11.3 (8.10; 16.8)
0.032
Local complication (%)
115 (33.3)
62 (100)
< 0.001
88 (57.5)
33 (89.2)
0.001
Ventilator (%)
3 (0.87)
4 (6.45)
0.012
0 (0.00)
23 (62.2)
< 0.001
ICU admission (%)
16 (4.64)
33 (53.2)
< 0.001
21 (13.7)
29 (78.4)
< 0.001
ICU stays (d)
0.00 (0.00; 0.00)
2.50 (0.00; 11.8)
< 0.001
0.00 (0.00; 0.00)
7.00 (1.00; 11.0)
< 0.001
Hospital stays (d)
10.0 (7.00; 15.0)
19.0 (11.0; 25.0)
< 0.001
10.0 (4.00; 19.0)
23.0 (15.0; 30.0)
< 0.001
Mortality (%)
1 (0.29)
2 (3.23)
0.062
1 (0.65)
4 (10.8)
0.005
Predictors and model construction
Four variables (HR, RR, Ca and BUN) were extracted as the predictors of SAP through LASSO regression. Supplementary Figure 1 and Supplementary Table 1 show this process in more detail. Multivariable logistic regression revealed that all four variables were independent predictors (Table 3). The probability (PA) of SAP could be calculated according to the following formula: PA = 1/{1 + exp [- (-6.42 + 0.05 × HR + 0.08 × RR - 1.30 × Ca + 0.14 × BUN)]}. Analysis of ARDS obtained similar results (Table 3). The following formula was used for ARDS: PA = 1/{1 + exp [- (-5.46 + 0.05 × HR + 0.10 × RR - 1.78 × Ca + 0.11 × BUN)]}. Two nomogram plots were displayed using prediction models (Figure 1).
Figure 1 Nomograms of new predictive models.
A: Nomogram of severe acute pancreatitis predictive model; B: Nomogram of acute respiratory distress syndrome prediction model. SAP: Severe acute pancreatitis; ARDS: Acute respiratory distress syndrome; HR: Heart rate; RR: Respiratory rate; Ca: Serum calcium concentration; BUN: Blood urea nitrogen.
Table 3 Multivariable logistic regression analysis for severe acute pancreatitis and acute respiratory distress syndrome prediction in derivation cohort.
Variable
SAP
ARDS
β
OR (95% CI)
P value
β
OR (95% CI)
P value
Intercept
-6.42
0.00 (0.00-0.05)
< 0.001
-5.46
0.00 (0.00-0.13)
0.002
HR
0.05
1.05 (1.04-1.07)
< 0.001
0.05
1.05 (1.03-1.07)
< 0.001
RR
0.08
1.08 (1.00-1.17)
0.047
0.10
1.10 (1.02-1.19)
0.014
Ca
-1.30
0.26 (0.09-0.73)
0.011
-1.78
0.17 (0.06-0.48)
0.001
BUN
0.14
1.15 (1.09-1.23)
< 0.001
0.11
1.12 (1.05-1.19)
< 0.001
Model performance
The new model indicated a great power of discrimination for SAP. Following 1000 interactions of bootstrapping to minimize the risk of overfitting to the original models, the AUC in the derivation cohort was estimated to 0.879 (95%CI: 0.830-0.928), which was significantly superior to that of SIRS (AUC = 0.808, 95%CI: 0.757-0.859, P = 0.002) and qSOFA (AUC = 0.730, 95%CI: 0.672-0.789, P < 0.001) and not inferior to that of the BISAP score (AUC = 0.888, 95%CI: 0.847-0.929, P = 0.6629) (Figure 2A, Table 4). In addition, the model indicated an optimal behavior in the validation cohort (AUC = 0.898, 95%CI: 0.848-0.949) (Figure 2B). The AUC of the new model in derivation was 0.892 (95%CI: 0.843-0.941) for ARDS prediction, which was superior to SIRS (AUC = 0.815, 95%CI: 0.766-0.864, P = 0.001) and qSOFA (AUC = 0.742, 95%CI: 0.678-0.807, P < 0.001) and not inferior to BISAP (AUC = 0.871, 95%CI: 0.827-0.916, P = 0.344) (Figure 2C, Table 4). Despite the assessment of the model in the validation cohort, its performance was moderate (AUC = 0.833. 95%CI: 0.754-0.912) (Figure 2D). When the cut-off value was set as PA > 25% for SAP prediction, the novel model suggested an optimal performance in the combined dataset (sensitivity 0.78, specificity 0.88) (Table 4). The best cut-off value was PA > 18% for ARDS prediction, with a sensitivity of 0.78 and a specificity of 0.85 (Table 4).
Figure 2 Receiver operating characteristic curves of different predictive models in derivation and validation cohort.
A: Receiver operating characteristic (ROC) curves of severe acute pancreatitis (SAP) predictive models in derivation cohort; B: ROC curves of SAP predictive models in validation cohort; C: ROC curves of acute respiratory distress syndrome (ARDS) predictive models in derivation cohort; D: ROC curves of ARDS predictive models in validation cohort. BISAP: Bedside index for severity in acute pancreatitis; qSOFA: Quick sequential organ failure assessment; SIRS: Systemic inflammatory response syndrome.
Table 4 Sensitivity, specificity, positive predictive value, negative predictive value for of the predictive models in combined cohort.
SAP
ARDS
PA
SE
SP
Youden index
PPV
NPV
PA
SE
SP
Youden index
PPV
NPV
> 10%
0.91
0.63
0.54
0.43
0.96
> 5%
0.94
0.48
0.42
0.26
0.98
> 20%
0.82
0.83
0.65
0.59
0.94
> 10%
0.85
0.71
0.56
0.37
0.96
> 25%
0.78
0.88
0.66
0.66
0.93
> 18%
0.78
0.85
0.63
0.51
0.95
> 40%
0.71
0.93
0.64
0.75
0.91
> 30%
0.67
0.93
0.59
0.65
0.93
> 60%
0.51
0.97
0.48
0.84
0.87
> 50%
0.46
0.96
0.43
0.71
0.90
> 80%
0.32
0.99
0.30
0.88
0.83
> 70%
0.29
0.98
0.27
0.71
0.87
> 95%
0.14
1.00
0.14
0.91
0.79
> 85%
0.12
1.00
0.12
0.86
0.85
Graphical assessment indicated a strong agreement between prediction and observation in both new models (Figure 3). The H-L test indicated that the difference between prediction and observation was not significant both in the derivation (χ2= 12.675, P = 0.124) and validation cohorts (χ2= 5.852, P = 0.664) with regard to SAP prediction. The model for ARDS prediction revealed improved performance with regard to the calibration in the derivation (χ2= 3.753, P = 0.879) and validation cohorts (χ2= 2.933, P = 0.939).
Figure 3 Calibration curves of new predictive models.
A: Calibration plot of severe acute pancreatitis (SAP) predictive model in derivation cohort; B: Calibration plot of SAP predictive model in validation cohort; C: Calibration plot of acute respiratory distress syndrome (ARDS) predictive model in derivation cohort; D: Calibration plot of ARDS predictive model in validation cohort.
DCA indicated that if the threshold PA was < 80%, using the new model to recognize and manage SAP had a positive net benefit compared with either the treat-all or treat-none (Figure 4A). In case the threshold probability was set to < 70%, the prediction and intervention for ARDS also produced net benefit (Figure 4B).
Figure 4 Decision curve analysis of new predictive models.
A: Decision curve analysis of severe acute pancreatitis predictive model in derivation and validation cohort; B: Decision curve analysis of acute respiratory distress syndrome predictive model in derivation and validation cohort.
DISCUSSION
In the current study, novel prediction models were established for SAP and ARDS in patients with AP. The models were also externally validated and exhibited remarkable discriminative power and high degree of consistency with the observation both in the derivation and external validation cohorts. These models suggested that patients with AP who manifested a higher heart rate, respiratory rate, blood urea nitrogen concentrations and lower serum calcium concentrations at admission exhibited a higher risk of developing SAP and ARDS.
AP is a major cause of acute abdomen. Patients with AP usually present with multiple organ dysfunction syndrome (MODS). Although organ dysfunction is mild and transient (< 48 h), approximately 20% of patients will proceed to consistent organ failure (> 48 h), leading to SAP and a high risk of mortality[27]. AP primarily affects the respiratory system and to a lesser extent the renal and cardiovascular systems[23]. ARDS is the critical event, which is noted during lung injury in AP[28]. Cyclooxygenase-2 inhibitors and enteral nutrition have been shown to prevent SAP, shorten the length of hospital stay and reduce infectious complications and mortality in patients with predicted SAP[11,12,29]. Administration of antiplatelet therapy, withdrawal of prehospital amiodarone treatment and administration of nebulized heparin may decrease the incidence of ARDS, inhibit the progression of lung injury and accelerate the recovery of patients at risk of developing ARDS[13-15,30]. Therefore, early identification of patients at risk of developing SAP and ARDS is clinically significant for improving the prognosis of AP.
Various models have been developed to predict SAP and organ dysfunction for AP management[16]. Although the majority of the models were deficient due to some limitations, such as small sample size, single center studies and lack of internal or external validation, several SAP models have been widely used and validated in different cohorts. These validations were performed using APACHE-II score, Ranson criteria, CTSI and BISAP[31-34]. BISAP contains only five variables and is simpler than APACHE-II score (18 items) and the Ranson criteria (11 items). However, their predictive power is equal[35]. In addition, ultrasound is preferred to CT as an efficient and nonradioactive examination used in the emergency department to initially evaluate potential development of AP. Therefore, pleural effusion and CT presentation could not be evaluated in this case to gain the BISAP and CTSI scores. Although LIPS is a popular model to predict ARDS for patients at risk, its calculation is considerably complicated[18]. In addition, its original developing cohort involved only a small part of patients with AP. LIPS had not been previously validated in patients with AP.
Therefore, a simple model with a low number of parameters and without radiology findings would be more practical. The novel predictive model reported in the current study involved only four parameters for both SAP and ARDS prediction and all these variables were routinely tested. The discriminatory power of the novel model was not inferior to that of BISAP. To the best of our knowledge, the prediction of SAP or ARDS for patients with AP has not been previously assessed by models that were as simple and accurate as this reported in the current study.
Using LASSO regression, calcium was identified as a predictor for both SAP and ARDS. Multiple logistic regression analysis indicated that serum calcium concentration was also the independent predictor. The models of the present study were easier to use than the APACHE-II, Ranson, CTSI and BISAP models and demonstrated improved efficacy than the other two simple models, suggesting their potential clinical significance. ARDS was a non-negligible manifestation of MODS in patients with AP. The majority of the models have mainly focused on the severity classification or mortality prediction of ARDS, whereas the identification of ARDS at an early stage is still challenging. Although LIPS was widely used to predict ARDS, the calculation of the LIPS score was complicated for patients admitted to the emergency department, since certain parameters may be unavailable[18]. The pathogenesis of ARDS involves the activation of signaling pathways, which include various cytokines and inflammatory mediators. Certain molecules, such as interleukin (IL)-6, IL-8, protein C, angiopoietin-2 and miRNAs and specific imaging examinations (X-ray and lung ultrasound) were also identified as predictors of ARDS in single or combined forms[8,36-38]. The data indicated that these new predictors seemed promising. However, the molecules and the examinations identified were not part of the routine clinical practice, which limited their clinical utility. Fei et al[10] used an artificial neural network algorithm to predict ARDS following SAP. The model by Fei et al[10] indicated high accuracy. However, the variable pancreatic necrosis rate was hard to assess when CT was not used and was not evident in the early course of AP.
The pathogenesis of SAP and ARDS involves a series of acute inflammatory reactions[2,39]. SIRS is widely adopted to assess the severity of diseases associated with acute inflammation. Both HR and RR are used in the SIRS model. Therefore, it is reasonable that both HR and RR were identified as predictors of SAP and ARDS. BUN has been shown to reflect volume depletion, renal function, the quality of resuscitates and even the ischemic injury of the pancreas during AP[40]. It has been reported that BUN can independently predict both SAP and the mortality of SAP[40-44]. Therefore, BUN was also involved in other predictive models of SAP, such as GCS, Ranson criteria and BISAP, in addition to our new model. The levels of BUN have not been used as a direct predictor of ARDS. However, this marker can be used as a predictor of pathogenesis in association with other risk factors, such as pancreatitis[39]. Calcium concentration has been closely associated with AP[45]. Hypocalcemia was common in the cohort of the present study and in other AP cohorts; notably in critically ill patients[46]. Elevated cytosolic calcium of pancreatic acinar cells causes premature trypsinogen activation, vacuolization and acinar cell death, which play critical roles in the pathogenesis of AP[47]. However, during the development of certain models for the prediction of SAP, serum calcium was excluded for a variety of factors[48]. Calcium is also involved in the pathogenesis of ARDS as a signaling molecule, leading to paracellular hyperpermeability through endothelial junction-cytoskeleton dissociation[49]. The current model suggested that calcium was an independent predictor of SAP and ARDS in AP, indicating the potential of developing novel drugs for the treatment of AP[50]. To the best of our knowledge, the model of the present study was the simplest used to predict SAP and ARDS within 24 h of AP admission. It is also the first model that involved serum calcium concentration to predict ARDS in AP.
The present study had several limitations. Firstly, organ dysfunction occurred mainly in the first week of AP, whereas accurate onset time was not available in the present study. Therefore, certain patients with SAP or ARDS may have been missed. Moreover, the exact onset time of ARDS was not recorded, so the new model only predicted the risk of ARDS during the whole admission (7-15 d) using the scores gained within 24 h of admission. It might not be appropriate, and the new model could not tell the clinicians when they should prepare for the possible onset of ARDS. Further studies are needed to verify the value of new models on a dynamic timescale. Secondly, the derivation cohort comprised tertiary and secondary hospitals from different regions of China. However, model validation was performed in a tertiary teaching hospital. Although the result of validation was also encouraging, it is hard to ignore that the incidence of SAP and ARDS in the validation cohort was considerably higher than that of the derivation cohort, which could reduce the generalizability in primary or secondary class hospitals, where MAP and MSAP exhibited high proportions. Moreover, certain laboratory examination technologies were different among four hospitals, which increased the systemic error of the data. It must be mentioned that mechanical ventilation will attenuate systemic inflammation of ARDS and the effect varies with patterns[51]. It is unavoidable that new scores to predict ARDS will also be affected. Unfortunately, detailed information of mechanical ventilation was not collected in this study, and further research is needed to investigate the influence of different ventilatory patterns on new models. Thirdly, selection bias was inevitable in a retrospective study. Furthermore, any missing value was deleted to obtain a complete-case dataset for analysis rather than imputating missing data with statistical methods (e.g., multi-imputation), which were not recommended. However, in the present study, only three individuals were removed due to missing data, accounting for a tiny part of the cohort. It was considered that complete data analysis would not affect the overall conclusion. Finally, other common predictive models, such as APACHE-II score, Ranson criteria, CTSI and LIPS were not evaluated due to lack of essential parameters. Therefore, direct comparison among different models was unavailable.
CONCLUSION
Novel models were developed containing only four items to predict SAP and ARDS in patients with AP, which were as accurate as BISAP but simpler. Serum calcium was identified as an important predictor, indicating a potential new strategy for management of AP. Further prospective studies are required to reveal whether early intervention based on novel prediction models could reduce the incidence of SAP and ARDS and finally improve the outcome of patients with AP.
ARTICLE HIGHLIGHTS
Research background
Acute respiratory distress syndrome (ARDS) is a major cause of death in patients with severe acute pancreatitis (SAP), but simple and credible predictive models are absent.
Research motivation
Use of models to predict and identify early patients with SAP and SAP-related ARDS, so that clinicians can manage these patients early to decrease mortality during admission.
Research objectives
To develop and verify new models to predict SAP and SAP-related ARDS.
Research methods
Clinical data from four centers were retrospectively collected. Items selected with least absolute shrinkage and selection operator regression method were involved in multiple logistic regression to develop the final model in development cohort. New models were than verified in validation cohort and assessed with C-index, calibration curve and decision-curve analysis.
Research results
New models could predict SAP and SAP-related ARDS with four easily available items, and performed well.
Research conclusions
We developed and verified simple and reliable models to predict SAP and SAP-related ARDS.
Research perspectives
To verify new models in a larger size of sample, and to investigate the questions raised by reviewer.
Footnotes
Provenance and peer review: Invited article; Externally peer reviewed.
Matta B, Gougol A, Gao X, Reddy N, Talukdar R, Kochhar R, Goenka MK, Gulla A, Gonzalez JA, Singh VK, Ferreira M, Stevens T, Barbu ST, Nawaz H, Gutierrez SC, Zarnescu NO, Capurso G, Easler J, Triantafyllou K, Pelaez-Luna M, Thakkar S, Ocampo C, de-Madaria E, Cote GA, Wu BU, Paragomi P, Pothoulakis I, Tang G, Papachristou GI. Worldwide Variations in Demographics, Management, and Outcomes of Acute Pancreatitis.Clin Gastroenterol Hepatol. 2020;18:1567-1575.e2.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 76][Cited by in F6Publishing: 70][Article Influence: 17.5][Reference Citation Analysis (0)]
Wu D, Lu B, Yang H, Li J, Qian J. [The three-category classification of severe acute pancreatitis: a single-center pilot study].Zhonghua Nei Ke Za Zhi. 2014;53:937-940.
[PubMed] [DOI][Cited in This Article: ]
Dixon B, Smith RJ, Campbell DJ, Moran JL, Doig GS, Rechnitzer T, MacIsaac CM, Simpson N, van Haren FMP, Ghosh AN, Gupta S, Broadfield EJC, Crozier TME, French C, Santamaria JD; CHARLI Study Group. Nebulised heparin for patients with or at risk of acute respiratory distress syndrome: a multicentre, randomised, double-blind, placebo-controlled phase 3 trial.Lancet Respir Med. 2021;9:360-372.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 18][Cited by in F6Publishing: 35][Article Influence: 11.7][Reference Citation Analysis (0)]
Gajic O, Dabbagh O, Park PK, Adesanya A, Chang SY, Hou P, Anderson H 3rd, Hoth JJ, Mikkelsen ME, Gentile NT, Gong MN, Talmor D, Bajwa E, Watkins TR, Festic E, Yilmaz M, Iscimen R, Kaufman DA, Esper AM, Sadikot R, Douglas I, Sevransky J, Malinchoc M; U. S. Critical Illness and Injury Trials Group: Lung Injury Prevention Study Investigators (USCIITG-LIPS). Early identification of patients at risk of acute lung injury: evaluation of lung injury prediction score in a multicenter cohort study.Am J Respir Crit Care Med. 2011;183:462-470.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 400][Cited by in F6Publishing: 449][Article Influence: 32.1][Reference Citation Analysis (0)]
Liang M, He M, Tang J, He X, Liu Z, Feng S, Chen P, Li H, Xue Y, Bai T, Ma Y, Zhang J. Novel risk scoring system for predicting acute respiratory distress syndrome among hospitalized patients with coronavirus disease 2019 in Wuhan, China.BMC Infect Dis. 2020;20:960.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 9][Cited by in F6Publishing: 11][Article Influence: 2.8][Reference Citation Analysis (0)]
Anan K, Ichikado K, Ishihara T, Shintani A, Kawamura K, Suga M, Sakagami T. A Scoring System with High-Resolution Computed Tomography to Predict Drug-Associated Acute Respiratory Distress Syndrome: Development and Internal Validation.Sci Rep. 2019;9:8601.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 5][Cited by in F6Publishing: 5][Article Influence: 1.0][Reference Citation Analysis (0)]
Seymour CW, Liu VX, Iwashyna TJ, Brunkhorst FM, Rea TD, Scherag A, Rubenfeld G, Kahn JM, Shankar-Hari M, Singer M, Deutschman CS, Escobar GJ, Angus DC. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3).JAMA. 2016;315:762-774.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 2092][Cited by in F6Publishing: 2356][Article Influence: 294.5][Reference Citation Analysis (0)]
Bone RC, Balk RA, Cerra FB, Dellinger RP, Fein AM, Knaus WA, Schein RM, Sibbald WJ. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine.Chest. 1992;101:1644-1655.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 6211][Cited by in F6Publishing: 6402][Article Influence: 200.1][Reference Citation Analysis (0)]
Banks PA, Bollen TL, Dervenis C, Gooszen HG, Johnson CD, Sarr MG, Tsiotos GG, Vege SS; Acute Pancreatitis Classification Working Group. Classification of acute pancreatitis--2012: revision of the Atlanta classification and definitions by international consensus.Gut. 2013;62:102-111.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 4134][Cited by in F6Publishing: 3988][Article Influence: 362.5][Reference Citation Analysis (41)]
Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, Vickers AJ, Ransohoff DF, Collins GS. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration.Ann Intern Med. 2015;162:W1-73.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 2833][Cited by in F6Publishing: 3000][Article Influence: 333.3][Reference Citation Analysis (0)]
Shi N, Liu T, de la Iglesia-Garcia D, Deng L, Jin T, Lan L, Zhu P, Hu W, Zhou Z, Singh V, Dominguez-Munoz JE, Windsor J, Huang W, Xia Q, Sutton R. Duration of organ failure impacts mortality in acute pancreatitis.Gut. 2020;69:604-605.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 48][Cited by in F6Publishing: 63][Article Influence: 15.8][Reference Citation Analysis (0)]
Petrov MS, van Santvoort HC, Besselink MG, van der Heijden GJ, Windsor JA, Gooszen HG. Enteral nutrition and the risk of mortality and infectious complications in patients with severe acute pancreatitis: a meta-analysis of randomized trials.Arch Surg. 2008;143:1111-1117.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 154][Cited by in F6Publishing: 121][Article Influence: 7.6][Reference Citation Analysis (0)]
Karnatovskaia LV, Festic E, Gajic O, Carter RE, Lee AS; US Critical Illness and Injury Trials Group: Lung Injury Prevention Study Investigators (USCIITG-LIPS). Prehospital amiodarone may increase the incidence of acute respiratory distress syndrome among patients at risk.J Crit Care. 2012;27:447-453.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 7][Cited by in F6Publishing: 7][Article Influence: 0.6][Reference Citation Analysis (0)]
Ranson JH, Rifkind KM, Roses DF, Fink SD, Eng K, Localio SA. Objective early identification of severe acute pancreatitis.Am J Gastroenterol. 1974;61:443-451.
[PubMed] [DOI][Cited in This Article: ]
Singh VK, Wu BU, Bollen TL, Repas K, Maurer R, Johannes RS, Mortele KJ, Conwell DL, Banks PA. A prospective evaluation of the bedside index for severity in acute pancreatitis score in assessing mortality and intermediate markers of severity in acute pancreatitis.Am J Gastroenterol. 2009;104:966-971.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 153][Cited by in F6Publishing: 161][Article Influence: 10.7][Reference Citation Analysis (0)]
Reilly JP, Wang F, Jones TK, Palakshappa JA, Anderson BJ, Shashaty MGS, Dunn TG, Johansson ED, Riley TR, Lim B, Abbott J, Ittner CAG, Cantu E, Lin X, Mikacenic C, Wurfel MM, Christiani DC, Calfee CS, Matthay MA, Christie JD, Feng R, Meyer NJ. Plasma angiopoietin-2 as a potential causal marker in sepsis-associated ARDS development: evidence from Mendelian randomization and mediation analysis.Intensive Care Med. 2018;44:1849-1858.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 88][Cited by in F6Publishing: 85][Article Influence: 14.2][Reference Citation Analysis (0)]
Pando E, Alberti P, Mata R, Gomez MJ, Vidal L, Cirera A, Dopazo C, Blanco L, Gomez C, Caralt M, Balsells J, Charco R. Early Changes in Blood Urea Nitrogen (BUN) Can Predict Mortality in Acute Pancreatitis: Comparative Study between BISAP Score, APACHE-II, and Other Laboratory Markers-A Prospective Observational Study.Can J Gastroenterol Hepatol. 2021;2021:6643595.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 9][Cited by in F6Publishing: 9][Article Influence: 3.0][Reference Citation Analysis (0)]
Koutroumpakis E, Wu BU, Bakker OJ, Dudekula A, Singh VK, Besselink MG, Yadav D, Mounzer R, van Santvoort HC, Whitcomb DC, Gooszen HG, Banks PA, Papachristou GI. Admission Hematocrit and Rise in Blood Urea Nitrogen at 24 h Outperform other Laboratory Markers in Predicting Persistent Organ Failure and Pancreatic Necrosis in Acute Pancreatitis: A Post Hoc Analysis of Three Large Prospective Databases.Am J Gastroenterol. 2015;110:1707-1716.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 74][Cited by in F6Publishing: 102][Article Influence: 11.3][Reference Citation Analysis (0)]
Serafini M, Cordero-Sanchez C, Di Paola R, Bhela IP, Aprile S, Purghè B, Fusco R, Cuzzocrea S, Genazzani AA, Riva B, Pirali T. Store-Operated Calcium Entry as a Therapeutic Target in Acute Pancreatitis: Discovery and Development of Drug-Like SOCE Inhibitors.J Med Chem. 2020;63:14761-14779.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 9][Cited by in F6Publishing: 14][Article Influence: 3.5][Reference Citation Analysis (0)]
