Machine learning for modeling and identifying risk factors of pancreatic fistula

Abstract

BACKGROUND

Pancreatic fistula is the most common complication of pancreatic surgeries that causes more serious conditions, including bleeding due to visceral vessel erosion and peritonitis.

AIM

To develop a machine learning (ML) model for postoperative pancreatic fistula and identify significant risk factors of the complication.

METHODS

A single-center retrospective clinical study was conducted which included 150 patients, who underwent pancreatoduodenectomy. Logistic regression, random forest, and CatBoost were employed for modeling the biochemical leak (symptomless fistula) and fistula grade B/C (clinically significant complication). The performance was estimated by receiver operating characteristic (ROC) area under the curve (AUC) after 5-fold cross-validation (20% testing and 80% training data). The risk factors were evaluated with the most accurate algorithm, based on the parameter “Importance” (Im), and Kendall correlation, P < 0.05.

RESULTS

The CatBoost algorithm was the most accurate with an AUC of 74%-86%. The study provided results of ML-based modeling and algorithm selection for pancreatic fistula prediction and risk factor evaluation. From 14 parameters we selected the main pre- and intraoperative prognostic factors of all the fistulas: Tumor vascular invasion (Im = 24.8%), age (Im = 18.6%), and body mass index (Im = 16.4%), AUC = 74%. The ML model showed that biochemical leak, blood and drain amylase level (Im = 21.6% and 16.4%), and blood leukocytes (Im = 11.2%) were crucial predictors for subsequent fistula B/C, AUC = 86%. Surgical techniques, morphology, and pancreatic duct diameter less than 3 mm were insignificant (Im < 5% and no correlations detected). The results were confirmed by correlation analysis.

CONCLUSION

This study highlights the key predictors of postoperative pancreatic fistula and establishes a robust ML-based model for individualized risk prediction. These findings contribute to the advancement of personalized perioperative care and may guide targeted preventive strategies.

Key Words: Pancreatoduodenectomy; Postoperative pancreatic fistula; Risk factors; Machine learning; Precision oncology

Core Tip: The study provides a machine-learning approach with a focus on medical data evaluation, based on algorithm selection. The best algorithm was CatBoost. Young age and large tumor size were associated with fistula B/C and biochemical leak development. Blood and drain amylase level increases and biochemical leak were the major risk factors of subsequent fistula B/C development.

INTRODUCTION

According to World Health Organization statistics, pancreatic cancer is associated with poor prognosis and is considered one of the most common causes of mortality in oncological patients[1,2]. The main step in pancreatic cancer treatment is surgery: Pancreatoduodenectomy (PD) and distal pancreatic resection. Pancreatic surgery requires high-level techniques, and is associated with a high risk of complications. Postoperative pancreatic fistula (POPF) is the most common complication that causes more serious conditions, including bleeding due to visceral vessel erosion and peritonitis. Severe fistula sometimes requires reoperation. POPF increases hospitalization time and cost and leads to poor outcomes of cancer treatment. During the last decades, surgical techniques have improved, but the pancreatic fistula incidence rate is still around 65%[3-7].

The main fistula criterion is an increase of more than 3-fold in pancreatic amylase level in drain fluid, compared to the normal blood level. The International Group of Studying Pancreatic Fistula (IGSPF) classified pancreatic fistula into 3 grades: Biochemical leak (grade A fistula before 2016), grade B and C fistulas[8,9]. Biochemical leak is an isolated increase in pancreatic amylase drain level, considered clinically insignificant. Fistulas B and C are characterized by clinical symptoms with different severity. Fistulas are usually diagnosed within the first 5 days after surgery; however, fistulas that develop up to 30 days after surgery are also common[5,6,9-11].

The IGSPF identified other intraoperative risk factors: Soft pancreatic texture, severe hemorrhage, long duration of surgery, and tumor morphology. The criteria were included in the fistula risk score calculator[5,12-14]. There were several attempts to develop a wider scale for pancreatic fistula risk, but the quality of results was insufficient[7,15]. Other potential risk factors may be evaluated before and after surgery and include clinical, demographic parameters, and surgical techniques. Possible risk factors are high body mass index (BMI), age, and male sex, comorbidities, and surgical techniques[6,16,17]; however, their significance is controversial. Pancreatic duct diameter is an important predictor of POPF; however, different studies consider high or low values a risk factor[16,17]. In addition, Villodre et al[18] in 2023 suggested tumor vascular invasion as a risk factor of pancreatic fistula.

High drain amylase level should be considered not only the diagnostic criterion for biochemical leak but also one of the main postoperative risk factors for subsequent pancreatic fistula B/C[12,19,20]. Nevertheless, the significance of the early increase in drain amylase level (within 5 days after PD) was validated in association with other possible risk factors, applying the evidence-based approach[21,22].

We believe that the implementation of a personal approach in pancreatic cancer surgery will help to predict pancreatic fistula and correct the treatment strategy[23,24]. Machine learning (ML) as a group of artificial intelligence (AI) methods is successfully implemented in various branches of industry and science[25]. Applied to clinical data, ML technologies provide the ability to create prognostic models of complication development and allow physicians to personalize treatment and diagnostic strategies[26]. Such models are usually used for individual prognosis[27-30]. There have been several attempts at AI-based POPF modeling. Lee et al[31] developed a deep learning based model of fistula after PD that included computed tomography and clinical data. Other studies presented a ML model based on preoperative data[32,33] and a drain-removal prediction model[34]. The latter included in the analysis, only classification ML algorithms without logistic regression. Selection of an optimal algorithm from various types will help to develop the best model[34]. We believe that by presenting the ML-based solution to a crucial problem in surgery, we will translate the effectiveness of the approach for precision medicine[30].

This study aimed to develop a predictive ML model for POPF and identify the main risk factors of the complication. By incorporating a comprehensive dataset including preoperative, intraoperative, and postoperative parameters, we sought to develop an accessible and accurate risk prediction tool for clinical practice.

MATERIALS AND METHODS

Study design

We performed a single-center retrospective clinical study, which included 150 patients who underwent PD in FSBI NMRRC from 2018 to 2023. The inclusion criteria were age 18-75 years, verified diagnosis of neoplasm in the pancreaticoduodenal region, PD, and stage of the disease: Early, locally advanced, and marginally resectable. The non-inclusion/exclusion criteria were: Severe comorbidity, age less than 18 and more than 75 years, performance status Eastern Cooperative Oncology Group 2 or more before surgery, metastases, and inability to perform radical operation. All the patients, meeting the inclusion criteria without non-inclusion/exclusion criteria were included in the study. We performed initial diagnosis and treatment in association with NCCN guidelines[35]. The patient list was obtained from the clinic database. All the data, including results of postoperative laboratory tests, were collected from medical records, according to the guidelines for retrospective studies[11].

All the patients underwent surgery via the standard surgical protocol of PD. This protocol involves the en bloc resection of the pancreatic head, duodenum, proximal jejunum, common bile duct, and gallbladder. The resection is followed by meticulous reconstruction to restore gastrointestinal continuity. The protocol included standard lymphadenectomy and vascular resection and reconstruction in case of vascular involvement. Patient data were collected from electronic medical records and included demographic characteristics, comorbidities, intraoperative parameters, and postoperative outcomes.

The patients were divided into 3 groups: “no fistula”, “biochemical leak”, and “fistula B/C”. Pancreatic fistula was diagnosed using IGSPF criteria[9]. Patients, included in the "fistula B/C” group had increased drain amylase levels 3 times greater than normal blood levels with clinical symptoms of fistula B/C. Biochemical leak was defined as an increased drain amylase level without clinical symptoms. Patients with “no fistula” had drain amylase levels no more than 3 times the normal blood level without clinical symptoms of pancreatic fistula. Due to the considered clinical insignificance of biochemical leak, we studied joined groups: “Fistula B/C and biochemical leak” and “no fistula and biochemical leak”. The fraction of patients with biochemical leak was 12.3% and with fistula B/C was 14.2%.

Mathematical modeling and parameters

ML mathematical modeling includes several steps: Model feature selection, model training, quality check, algorithm selection, and risk factor evaluation. We developed two types of models: Binary and 3-dimensional. The binary model included two classes: Patients with no fistula and patients with biochemical leak or fistula B/C. The 3-dimensional model included 3 classes: No fistula, biochemical leak, and fistula B/C. The models were developed with the pre- and intraoperative data and with additional clinical information, collected 3-5 days after the surgery.

In the first step, we selected features for ML modeling. To develop an accurate ML model, suitable for implementation in clinical practice, evaluation of clinical parameters is required. We selected 25 potential risk factors, based on the latest scientific data and available medical records[3,5,12]. Next, from 25 clinical parameters we selected 14 features for optimal model development (Table 1) with simulated annealing, which is a randomized algorithm of suboptimal optimization. The algorithms make small random changes in a subset of features, optimizing the function of POPF. According to the latest research, this step is crucial for high-quality ML model development[29,36-38].

Table 1 Clinical parameters.

Groups parameters	All patients	No fistula	No fistula and biochemical leak	Fistula B/C and biochemical leak	Biochemical leak	Fistula B/C
Qualitative disease and treatment parameters (%)
Blood vessel resection	15.3	19.8	16.8	3.3	7.1	0
Perineural invasion	46.9	55.6	52.6	23.3	14.3	31.3
Vascular invasion	48.7	59.3	54.7	20	14.3	25
Enterostomy	23.1	76.2	24.7	25	50	14.3
Roux anastomosis	36.3	87.9	37.7	20	0	28.6
Tumor morphology (%)
Benign tumor	2.02	1.4	2.3	4	10	0
Neuroendocrine cancer	10.1	6.8	9.5	20	30	13.3
Epithelial malignant tumor	87.9	91.9	88.1	76	60	86.7
Disease stage (%)
I	17	16.9	16.5	17.2	14.3	20
IIa	24	22.5	22.4	27.6	21.4	33.3
IIb	30	29.6	30.6	31	35.7	26.7
III	20	21.1	21.2	17.2	21.4	13.3
IV	9	9.9	9.4	6.9	7.1	6.7
Quantitative parameters, median (interquartile range)
Age	61 (53.75, 66)	62 (54.75, 68)	60.5 (54, 66)	56.5 (46, 62.75)	61 (50.75, 64.75)	54.5 (46, 59.5)
Blood amylase 3-5 days, mM	50 (29.5, 91.7)	44 (24, 70)	51 (30, 91.6)	81 (37, 138)	45.5 (32.5, 97.15)	91.8 (66.5, 246)
Drain amylase 3-5 days, mM	81 (17.75, 1243)	23 (13.5, 44.5)	81 (19.25, 1341)	2445 (628.2; 3725)	932.5 (370.25, 1653.75)	4065 (3707, 5000)
Leukocytes 2-3 days, 109/L	13.1(9.25, 17.45)	12.4 (9.225, 15.7)	12.8 (9.3, 17.2)	16.7 (11.85, 20.35)	16.7 (11.7, 20.8)	14.85 (12.1, 19.025)
Tumor size, mm3	12.5 (5.1, 32)	5 (3, 7.25)	12.5 (5.25, 36.26)	3 (2.5, 4.375)	3 (2.5, 3.75)	3 (2.95, 4.5)
BMI, kg/m2	25.26 (23.1, 28.21)	25.3 (23, 28.1)	25.3 (23, 28.1)	25.8 (23, 28.1)	24.5 (21.6, 27.5)	25.9 (23.9, 28.2)
Pancreatic duct diameter, mm	4.5 (2.5, 7)	5 (3, 7.25)	4.5 (2.5, 7)	3 (2.5, 4.36)	3 (2.5, 3.75)	3 (2.95, 4.5)

BMI: Body mass index.

The parameters included were demographic and clinical characteristics (age, BMI, and disease stage I-IV), operative approaches (applied Roux anastomosis, enterostomy, blood vessel resection), instrumental values (tumor volume, vascular, perineural invasion, and diameter of pancreatic duct), and tumor type (benign, neuroendocrinal, and epithelial malignancies). Laboratory values included were blood and drain amylase levels, measured 3-5 days after surgery, and blood leukocyte level at 2-3 days. Tumor morphology and vascular and perineural invasion were initially studied before surgery with biopsy and the morphological diagnosis was confirmed after the operation. Qualitative parameters were transformed into numerical equivalents. We classified the tumor morphological types by clinical prognosis: Benign: 0, neuroendocrinal malignancies: 1, and epithelial malignancies: 2. We studied pancreatic fistula as the primary outcome of PD which was selected as the target variable for all the models.

We developed logistic regression, random forest, and CatBoost algorithms to select the most accurate one. Logistic regression is a linear algorithm that is effective for modeling qualitative parameters. Random forest and CatBoost are non-linear ML algorithms, based on decision trees[38,39] and the latter includes ordered gradient boosting. CatBoost is considered unbiased and effective in classification models with categorical variables[40-42]. Thus, we selected one linear and two non-linear algorithms to select the most accurate one suitable to the dataset.

To prevent overtraining, we performed a 5-fold cross-validation[43]. We tested the model quality with metric receiver operating characteristic (ROC) area under the curve (AUC)[43,44]. ROC is a function between specificity and 1-sensitivity and ROC AUC is an integral of the parameters. To improve the model qualities we searched for the best hyperparameters with the GridSearch algorithm[45]. Hyperparameter tuning is a procedure of search for the optimal configurable parameters of the ML model. The most accurate algorithm was selected for risk factor evaluation.

The code was developed in Jupyter Notebook, Anaconda 3, Python 3.7. We developed the algorithms and selected hyperparameters with the sklearn library. Cross-validation was performed with sklearn.model_selection.StratifiedKFold, and hyperparameters were selected with sklearn.model_selection.GridSearchCV[45].

The risk factors were evaluated with standard approaches available in the sklearn library. Logistic regression evaluates risk factors by the coefficients and their significance. In the case of the decision tree algorithm, the parameter “Importance” (Im) is estimated, which is a relative value of Gini impurity change, corrected on standard deviation. Gini impurity measures how often a randomly chosen element of a set would be incorrectly labeled if it were labeled randomly and independently according to the distribution of labels in the set[39].

Statistical analysis

To evaluate possible risk factors, associated with POPF, we combined ML modeling with Kendall correlation analysis, P < 0.05. In cases of multiplicity, we employed family-wise error rate Bonferroni correction to adjust P values and prevent correlations from incorrectly appearing to be statistically significant[46]. The statistical analysis was performed in Statistica 13.2 and RStudio with the programming language R.

RESULTS

Correlation analysis

We detected a significant positive correlation between blood and drain amylase level increase and subsequent development of biochemical leak and pancreatic fistula B/C (r = 0.25, P = 0.017 and r = 0.74, P = 8.37 × 10^-9), including joint groups: “Fistula B/C and biochemical leak” (r = 0.24, P = 0.02 and r = 0.77, P = 1.56 × 10^-7) and “no fistula and biochemical leak” (r = 0.24, P = 0.02 and r = 0.77, P = 1.56 × 10^-7). The same results were received for young age (r = -0.23, -0.21, -0.24, P = 0.021, 0.031, 0.014) (Table 1 and Table 2). In addition, perineural and vascular invasion were negatively correlated with biochemical leak and fistula B/C (r = -0.31, P = 0.002 and r = -0.32, P = 0.001), including the joint group “fistula B/C and biochemical leak” (r = -0.33, P = 0.001 and r = -0.35, P = 0.0003).

Table 2 Correlation matrix.

Correlated groups parameters	No fistula-0; biochemical leak-1; fistula-2		No fistula and biochemical leak- 0; fistula B/C-1		No fistula-0; biochemical leak and fistula B/C-1
	r value	P value	r value	P value	r value	P value
Age	-0.23	0.021a	-0.24	0.014b	-0.21	0.031a
Enterostomy	-0.05	0.637	-0.06	0.536	-0.04	0.676
Roux anastomosis	-0.2	0.064	-0.09	0.409	-0.22	0.04
Blood vessel resection	-0.13	0.226	-0.2	0.071	-0.12	0.288
Disease stage	-0.12	0.251	-0.11	0.3	-0.12	0.259
Blood amylase 3-5 days	0.25	0.017b	0.24	0.024a	0.24	0.02a
Drain amylase 3-5 days	0.74	8.37 × 10-9b	0.51	0.0004b	0.7	1.56 × 107b
Leukocytes 2-3 days	0.16	0.121	0.09	0.371	0.17	0.1
Tumor morphology	0.16	0.127	0.12	0.222	0.16	0.117
Tumor size, mm3	-0.01	0.934	-0.06	0.571	0.004b	0.975
Perineural invasion	-0.31	0.002b	-0.18	0.062	-0.33	0.001b
Vascular invasion	-0.32	0.001b	-0.17	0.088	-0.35	0.0003b
BMI	-0.04	0.239	0.001	0.672	-0.04	0.382
Pancreatic duct diameter	-0.08	0.437	0.03	0.743	-0.11	0.293

^aP < 0.05.

^bP < 0.0167 (Bonferroni correction, P < α/3).

BMI: Body mass index. The qualitative parameters were studied in numerical equivalent. For disease stages I, IIa, IIb, III, and IV the values were considered 1, 2, 2.5, 3, and 4, respectively. Tumor morphological types were divided into numerical classes by prognosis, from positive to negative: Benign tumor-0, neuroendocrine tumor-1, and adenocarcinoma-2[12,57].

ML modeling

According to the IGSPF, fistula B/C is a clinically significant complication in contrast to biochemical leak[8]. However, biochemical leak may increase the risk of fistula B/C development[12]. Therefore, at the first step of the study, we developed binary ML models (logistic regression, random forest, and CatBoost) that distinguish patients with no fistula from patients with fistula B/C or biochemical leak. We developed ML models, based on pre- and intraoperative data, and additional data, collected 3-5 days after surgery. After cross-validation and hyperparameters selection, CatBoost was detected as the most accurate algorithm, with the best ROC AUC (Table 3).

Table 3 Quality of binary models, mean ± SD.

Algorithm model	Logistic regression	Random forest	CatBoost
5-fold cross validation ROC AUC1
Pre- and intraoperative data model	0.71 ± 0.11	0.77 ± 0.08	0.74 ± 0.1
3-5 days after surgery	0.86 ± 0.09	0.8 ± 0.11	0.87 ± 0.07
5-fold cross validation ROC AUC2
Pre- and intraoperative data model	0.72	0.74	0.74
3-5 days after surgery	0.81	0.81	0.86

¹mean ± SD receiver operating characteristic area under the curve of each fold of cross-validation, n = 20% of the initial dataset.

²Integral receiver operating characteristic area under the curve calculated with all false positive rates obtained in 5-fold cross-validation. True positive rate was obtained from the initial dataset.

ROC: Receiver operating characteristic; AUC: Area under the curve.

We developed a three-dimensional model, distinguishing no fistula, fistula B/C, and biochemical leak. The model was employed in two ways: To predict fistula B/C in general, considering biochemical leak as no fistula (fistula B/C vs other classes) and to predict fistula B/C in patients with an increased amylase drain level according to the IGSPF criteria (fistula B/C vs biochemical leak). The quality of each prediction was evaluated separately (Table 4).

Table 4 Quality of three-dimensional models, mean ± SD.

Algorithm model	Logistic regression	Random forest	CatBoost
5-fold cross validation ROC AUC1
Pre- and intraoperative data model	0.71 ± 0.088	0.67 ± 0.09	0.68 ± 0.07
3-5 days after surgery	0.77 ± 0.145	0.726 ± 0.145	0.74 ± 0.16
5-fold cross validation ROC AUC2
Pre- and intraoperative data model: Fistula B/C vs other classes	0.77	0.73	0.85
3-5 days after surgery: Fistula B/C vs other classes	0.76	0.77	0.86
Pre- and intraoperative data model: Fistula B/C vs biochemical leak	0.73	0.45	0.74
3-5 days after surgery: Fistula B/C vs biochemical leak	0.39	0.44	0.47

¹mean ± SD receiver operating characteristic area under the curve of each fold of cross-validation, n = 20% of the initial dataset.

²Integral receiver operating characteristic area under the curve calculated with all false positive rates obtained in 5-fold cross-validation. True positive rate was obtained from the initial dataset. These metrics were calculated for the model’s ability to distinguish different classes.

ROC: Receiver operating characteristic; AUC: Area under the curve.

In the case of the three-dimensional model, CatBoost was also the most accurate algorithm with the best ROC AUC. We suggest that the result is associated with the inclusion of qualitative parameters with different classes as boosting algorithms are good for such data analysis[30,42,47].

All the models had good quality, but the lowest ROC AUC was detected for logistic regression which may indicate non-linear data. Random forest and CatBoost are quite similar algorithms, based on decision trees; therefore it is enough to employ the best one in accordance with the ROC AUC, for risk factor evaluation. Thus, we selected the CatBoost algorithm for risk factor evaluation, based on the quality of binary and three-dimensional algorithms (Table 3 and Table 4). All further predictions and risk factor analysis were based on CatBoost ML models. To measure the importance of potential risk factors, we calculated the relative parameter Im[42].

There are various ways to select the main predictors from an ML model. Our study aimed to scale the possible risk factors, and at the same time select the main risk factors considering the highest difference between Ims as the border, and employing cut-offs of Im (5%)[48-51].

According to the binary CatBoost model, developed using pre- and intraoperative data, ROC AUC = 74% (Table 3 and Figure 1A), the main risk factors of fistula B/C and biochemical leak were tumor vascular invasion, age, and BMI, Im = 24.8, 18.6, and 16.4, respectively (Figure 2A). Risk factors with lower significance were tumor size, disease stage, and Roux anastomosis Im = 10, 8.8, and 5.9, respectively. Tumor morphology, enterostomy, pancreatic duct diameter, perineural invasion, and blood vessel resection Ims were below 5%.

Figure 1 Receiver operating characteristic curve. A and B: Binary model of pancreatic fistula (A: Model, based on the pre- and intraoperative data; B: Model, developed with additional 3-5 postoperative days data). The model was developed with the CatBoost algorithm. We considered two classes: No fistula and patients with biochemical leak or fistula B and C; C-F: Three-dimensional model of pancreatic fistula. The model was developed with the CatBoost algorithm. We considered three classes: No fistula, patients with biochemical leak, and fistula B/C [C: Prediction of fistula B/C, considering biochemical leak as no fistula, based on pre- and intraoperative data; D: Prediction of fistula B/C in patients with increased amylase drain level according to the International Group of Studying Pancreatic Fistula (IGSPF) criteria, based on pre- and intraoperative data; E: Prediction of fistula B/C, considering biochemical leak as no fistula at 3-5 postoperative days; F: Prediction of fistula B/C in patients with increased amylase drain level according to the IGSPF criteria at 3-5 postoperative days]. The plot presents the quality of the model receiver operating characteristic (ROC) area under the curve. The area under the ROC curve is the quality metric, presented in values (0, 1). ROC curve is a function. True positive rates are shown along the ordinate and false positive rates - along the abscissa. False positive rates received in 5-fold cross-validation. True positive rates received from the initial dataset. ROC: Receiver operating characteristic.

Figure 2 Feature importance. A: Binary pre- and intraoperative model of pancreatic fistula; B: Binary postoperative model of pancreatic fistula; C: Three-dimensional pre- and intraoperative model of pancreatic fistula; D: Three-dimensional postoperative model of pancreatic fistula. Importances of the parameters (%), are presented at the abscissa. The features are ranked by importance. Im: Importance; BMI: Body mass index.

The binary model, based on the data known at 3-5 days after surgery, ROC AUC = 86% (Table 3 and Figure 1B), detected the main risk factors: Drain and amylase levels on 3-5 days after surgery, blood leukocytes, measured at 2-3 days, Im = 21.6%, 16.4%, and 11.2%, respectively. According to the IGSPF, the main diagnostic criterion of pancreatic fistula is an increase in the amylase drain level more than 3 times that in the blood; therefore, the results support the validity of the model. BMI, tumor vascular invasion, and age were also important risk factors, Im = 10.3%, 9.5%, and 9%, respectively. The Im of tumor size was 6.7%, and the Ims of the remaining parameters were less than 5% (Figure 2B).

The same results were obtained from the correlation analysis. Tumor vascular invasion was more common in patients with no fistula in comparison to patients with biochemical leak, fistula B/C, and the joint group “biochemical leak and fistula B/C” (Table 1). Patients with fistula B/C and patients in the joint group “biochemical leak and fistula B/C” were younger, compared to the “no fistula” group. Tumors with morphology associated with worse prognosis, were more common in patients with pancreatic fistula. All the parameters were significantly correlated with fistula B/C and biochemical leak, especially the amylase drain level (Table 2).

According to the three-dimensional postoperative model, ROC AUC = 85% (Figure 1C and Table 4), the main general risk factors of pancreatic fistula B/C were BMI, age, tumor vascular invasion, and tumor size, Im = 20.7%, 17.9%, 14.3%, and 10%, respectively. Disease stage, tumor morphology, and pancreatic duct diameter were also important factors, Im = 9.3%, 8.5%, and 6%, respectively. The Ims of other parameters were below 5% (Figure 2C).

Similar results were obtained from the correlation analysis: Young age was more common in patients with biochemical leak and pancreatic fistula B/C development (Table 1 and Table 2). The model was also effective in distinguishing between the risk of pancreatic fistula B/C and biochemical leak, ROC AUC = 74% (Figure 1D).

Based on the 3-5 postoperative days data, the quality of fistula B/C prediction was higher, ROC AUC = 86% (Figure 1E). Similar to the binary model, the main risk factors were drain and blood amylase level 3-5 days after surgery, blood leukocytes, measured at 2-3 days, Im = 25%, 13.4%, and 12.4%, respectively. Age and BMI were also important factors, Im = 9.8% and 8.6%. Vascular invasion, tumor size, and disease stage were the next parameters sorted at the significance level, Im = 6%, 5.4%, and 5.4%, respectively. Importances, calculated for tumor morphology, pancreatic duct diameter, perineural invasion, enterostomy, Roux anastomosis, and blood vessel resection were less than 5% (Figure 2D).

We obtained low quality in the model to distinguish biochemical leak and fistula B/C after surgery, ROC AUC = 47% (Figure 1F). Therefore at 3-5 days after PD, biochemical leak and fistula B/C have the same predictors and a significant increase in pancreatic amylase drain level leads to subsequent fistula B/C.

DISCUSSION

We employed logistic regression, random forest, and CatBoost algorithms to develop models of POPF. Based on the ROC AUC metrics, the CatBoost ML algorithm was the most effective. The CatBoost model predicted fistula B/C and biochemical leak with high quality (> 70%)[28,38,52], ROC AUC = 74%-86%, based on pre- and intraoperative data with additional data, obtained 3-5 days after surgery. The quality is enough for risk factor detection and clinical implementation[30,42,43]. We provide results of the first implementation of the ML approach, based on the selection of algorithms and their hyperparameters for pancreatic fistula prediction and risk factor detection. Previous studies that aimed to extend ISGPF classification and scale risk factors, did not employ hyperparameter selection[15]. Usually, ML is used for predictive modeling; however, in our study, we also provide risk factor analysis.

Our study scaled 14 clinical parameters as risk factors for pancreatic fistula. We selected the main pre- and intraoperative prognostic factors of all the fistulas: Tumor vascular invasion, age, and BMI, ROC AUC = 74%. The ML model confirmed biochemical leak, blood and drain amylase level, and blood leukocytes as crucial predictors for subsequent fistula B/C, ROC AUC = 86%. Surgical techniques, tumor size, morphology, and pancreatic duct diameter less than 3 mm were insignificant.

Increases in amylase drain and blood levels after surgery were the main risk factors for the subsequent development of pancreatic fistulas B/C. Since 2016, the IGSPF has considered biochemical leak as a clinically insignificant condition[9,10]. We included biochemical leak in our novel model and showed that the increase in pancreatic amylase drain level and biochemical leak were associated with the development of fistulas with clinical symptoms. Thus, we consider it important to timely detect biochemical leak early after surgery to predict and prevent fistula grade B and C. These results prove the crucial role of pancreatic blood and drain amylase levels from an evidence-based point of view[14,21,53].

Young patient age was a significant risk factor for fistula development which was also confirmed in previously published results[7]. BMI and large tumor size were also significant in the models. There are a few published studies, dedicated to the significance of tumor size as a pancreatic fistula risk factor; however, similar results were obtained in a retrospective observational study by Mostafa et al[54]. The role of BMI is contradictory in different studies[16], and we proved its significance. Thus, we consider these parameters as preoperative risk factors.

According to the ML models and correlation analysis, perineural, and vascular invasion were associated with a better prognosis of pancreatic fistula. Similar results regarding vascular invasion were obtained in the retrospective multicenter observational study by Villodre et al[18]. We verified the clinical significance of these parameters and suggest perineural and vascular invasion should be evaluated in biopsy material, as prognostic factors of POPF. We did not find any publications dedicated to preoperative evaluation of perineural and vascular invasion, and these parameters are not highlighted in the clinical guidelines. Thus, we believe that our results will demonstrate their significance.

ML modeling results were confirmed by classical statistics, and correlation analysis, whereas some literature data are contradictory. Different studies consider high or low pancreatic duct diameter a risk factor[16,17]; the observations on BMI and surgical techniques are controversial[6,16,55]. Being important for ML model development, BMI was not correlated with pancreatic fistula (Kendall correlation, P < 0.05). Considering the controversial literature data[6,16], we suggest a complicated association between these parameters that is difficult to detect with the classical statistical analysis employed in previous studies. At the same time, ML modeling may stay challenging without a big dataset that is difficult to collect for pancreatic cancer[56,57].

Novelty and limitations

The novelty of the study includes the implementation of a ML approach with a focus on medical data evaluation, based on algorithm selection. There are no previous publications describing three-dimensional ML models of POPF and the study of biochemical leak. We provide the results of ML modeling, based on pre- and intraoperative data with additional clinical information, 3-5 days after surgery.

This study is limited by its retrospective design, which may introduce selection bias. Additionally, the single-center nature of the dataset limits the generalizability of the findings. While the model demonstrated robust performance in internal validation, external validation in multicenter datasets is necessary to confirm its utility across diverse populations. We included only patients, who underwent PD and excluded patients with distal pancreatic cancer who may have a different prognosis. From a technical point of view, the workflow for risk factor analysis included Ims evaluation. If the Im of a parameter, such as the drain amylase level increases with a gap, all other Im values decrease about the gap[39] which may lead to bias.

CONCLUSION

This study highlights the key predictors of POPF and establishes a robust ML-based model for individualized risk prediction. The results of the risk factor analysis were confirmed by correlation analysis and indicated the main risk factors for this complication. Thus, young age and large tumor size are associated with fistula B/C and biochemical leak development. Blood and drain amylase level increases and biochemical leak are the major risk factors of subsequent fistula B/C development. By identifying high-risk individuals, the developed model has the potential to guide personalized management strategies for patients undergoing PD.