Establishment and application of three predictive models of anastomotic leakage after rectal cancer sphincter-preserving surgery

Abstract

BACKGROUND

Anastomotic leakage (AL) occurs frequently after sphincter-preserving surgery for rectal cancer and has a significant mortality rate. There are many factors that influence the incidence of AL, and each patient’s unique circumstances add to this diversity. The early identification and prediction of AL after sphincter-preserving surgery are of great significance for the application of clinically targeted preventive measures. Developing an AL predictive model coincides with the aim of personalised healthcare, enhances clinical management techniques, and advances the medical industry along a more precise and intelligent path.

AIM

To develop nomogram, decision tree, and random forest prediction models for AL following sphincter-preserving surgery for rectal cancer and to evaluate the predictive efficacy of the three models.

METHODS

The clinical information of 497 patients with rectal cancer who underwent sphincter-preserving surgery at Jincheng People’s Hospital of Shanxi Province between January 2017 and September 2022 was analyzed in this study. Patients were divided into two groups: AL and no AL. Using univariate and multivariate analyses, we identified factors influencing postoperative AL. These factors were used to establish nomogram, decision tree, and random forest models. The sensitivity, specificity, recall, accuracy, and area under the receiver operating characteristic curve (AUC) were compared between the three models.

RESULTS

AL occurred in 10.26% of the 497 patients with rectal cancer. The nomogram model had an AUC of 0.922, sensitivity of 0.745, specificity of 0.966, accuracy of 0.936, recall of 0.987, and accuracy of 0.946. The above indices in the decision tree model were 0.919, 0.833, 0.862, 0.951, 0.994, and 0.955, respectively and in the random forest model were 1.000, 1.000, 1.000, 0.951, 0.994, and 0.955, respectively. The DeLong test revealed that the AUC value of the decision-tree model was lower than that of the random forest model (P < 0.05).

CONCLUSION

The random forest model may be used to identify patients at high risk of AL after sphincter-preserving surgery for rectal cancer owing to its strong predictive effect and stability.

Key Words: Cancer of rectum; Anastomotic leakage; Nomogram; Decision tree; Random forest

Core Tip: Anastomotic leakage (AL) is a very dangerous complication of rectal cancer surgery, which not only increases the recurrence rate of the tumor but also lowers the quality of life of affected patients. We examined the clinical data of 497 patients with rectal cancer to determine variables that influence AL. We established nomogram, decision tree, and random forest models to identify a prediction model tool for forecasting AL after rectal cancer surgery.

INTRODUCTION

According to the most recent data from 2020, colorectal cancer has become the third most common cancer worldwide. In China, colorectal cancer has the third highest incidence and fatality rate among all malignancies. Cases of rectal cancer account for 39% of total colorectal cancer cases, making it a serious public health issue in China[1,2]. Radical surgery remains the first choice of treatment for rectal cancer, both for primary and secondary tumors[3]. With advances in medical technology, the prognosis of patients undergoing rectal cancer surgery has significantly improved, and the sphincter preservation rate has also continuously improved[4].

Nevertheless, anastomotic leakage (AL) remains the most common complication following sphincter-preserving surgery for rectal cancer. The perioperative mortality of patients with AL is as high as 18.6%, and these patients are more likely to experience other complications[5,6]. The early identification and prediction of AL after sphincter-preserving surgery are of great significance for the application of clinically targeted preventive measures. At present, a logistic regression analysis of the factors influencing AL in clinical practice is performed[7,8]; however, it cannot intuitively show the importance of each factor to the outcome. Therefore, this study developed nomogram, decision tree, and random forest models for predicting AL following rectal cancer sphincter-preserving surgery. The predictive power of the models were evaluated to identify a tool that would enable the identification of high-risk patients.

MATERIALS AND METHODS

Patient characteristics

For this retrospective analysis, we collected clinical data from 497 patients with rectal cancer who underwent sphincter-preserving surgery at Jincheng People’s Hospital of Shanxi Province between January 2017 and September 2022. The inclusion criteria were: Rectal cancer diagnosed by colonoscopy or anal biopsy; tumor within 12 cm of the anal margin; age ≥ 18 years old; and complete clinical data. The exclusion criteria were: Extensive tumor metastasis or the presence of other malignant tumors; a history of rectal surgery, anal stenosis or anal fistula; and conversion to laparotomy. The patients were divided into two groups, those with AL and those without (no AL). The study was approved by the Jincheng People’s Hospital of Shanxi Province (JCPH.No20230407001) and written informed consent was obtained from all study participants or their legal guardians.

Study variables

The following variables were analyzed: Sex, age, body mass index, diabetes mellitus, hypertension, smoking history, neoadjuvant treatment, hemoglobin level, albumin (Alb) level, tumor size, tumor-node-metastasis stage, American Society of Anesthesiologists score, tumor location, surgical approach, operative time, and blood loss.

Definition of AL

The diagnosis of AL was based on clinical manifestations (pain, persistent body temperature of > 38 °C, peritonitis, watery fecal matter, food residue in fecal matter, or pus in the drainage fluid), laboratory tests (elevated white blood cell count and neutrophil percentage), and imaging findings (computed tomography following an enema with a liquid, gas, or water-soluble contrast agent)[9].

Statistical analysis

Statistical analyses were performed using SPSS for Windows version 26.0 (IBM Corp., Armonk, NY, United States). All continuous variable data are presented as mean ± SD, and student’s t-tests were used to compare differences. Data from discrete variables are presented as numbers and percentages, and the χ² test was used to assess differences between groups. Variables associated with AL were identified using univariate and multivariate logistic regression analyses.

The prediction models were constructed using the R software, and the data were randomly divided between a training set and a verification set in a 7:3 ratio. The nomogram was created using the ‘rms’ package, the decision tree with the ‘rpart’ package, and the random forest using the ‘random Forest’ package. The model with the best predictive effect was selected by comparing the sensitivity, specificity, accuracy, recall rate, precision rate, and area under the receiver operating characteristic curve (AUC). AUCs were compared using the DeLong test. The statistical significance level was set at P < 0.05.

RESULTS

Patient characteristics

There were 271 men and 226 women among the 497 patients. The incidence of AL was 10.26% (51/497). Patients in the AL group had a mean age of 60.98 ± 10.83 years. The no AL group included 446 patients with a mean age of 59.27 ± 10.76 years.

Univariate analysis

Univariate analysis revealed statistically significant differences between the AL and no AL groups for the following variables: Sex, diabetes mellitus, smoking history, neoadjuvant treatment, Alb level, tumor size, and tumor location (Table 1).

Table 1 Single factor analysis of anastomotic leakage, n (%).

Patient characteristics	AL (n = 51)	No AL (n = 446)	t/χ²	P value
Sex			6.921	0.009
Male	33 (64.71)	244 (54.71)
Female	18 (35.29)	202 (45.29)
Age, year (mean ± SD)	60.98 ± 10.83	59.27 ± 10.76	1.073	0.284
BMI, kg/m² (mean ± SD)	22.30 ± 2.91	21.45 ± 3.06	1.896	0.058
Diabetes mellitus			14.164	< 0.001
No	21 (41.18)	302 (67.71)
Yes	30 (58.82)	144 (32.29)
Hypertension			0.232	0.630
No	16 (31.37)	155 (34.75)
Yes	35 (68.63)	291 (65.25)
Smoking history			6.970	0.008
No	14 (27.45)	209 (46.86)
Yes	37 (72.55)	237 (53.14)
Neoadjuvant treatment			7.973	0.005
No	16 (31.37)	233 (52.24)
Yes	35 (68.63)	213 (47.76)
Hb, g/L (mean ± SD)	135.60 ± 10.46	136.75 ± 10.41	0.746	0.456
Alb, g/L (mean ± SD)	33.23 ± 6.59	37.13 ± 7.25	3.664	< 0.001
Tumor size, cm (mean ± SD)	4.73 ± 1.22	3.42 ± 1.26	7.009	< 0.001
Tumor location, cm (mean ± SD)	4.32 ± 1.28	6.13 ± 1.30	9.378	< 0.001
TNM stage			0.010	0.995
I	32 (62.75)	281 (63.00)
II	12 (23.53)	106 (23.77)
III	7 (13.72)	59 (13.23)
ASA score			0.289	0.866
I	34 (66.67)	282 (63.23)
II	12 (23.53)	111 (24.89)
III	5 (9.80)	53 (11.88)
Surgical approach			1.676	0.195
Open	13 (25.49)	154 (34.53)
Laparoscopic	38 (74.51)	292 (65.47)
Operation time, min (mean ± SD)	182.19 ± 6.25	181.87 ± 5.79	0.378	0.705
Blood loss, mL (mean ± SD)	230.45 ± 17.62	232.74 ± 20.58	0.761	0.447

AL: Anastomotic leakage; Alb: Albumin; ASA: American Society of Anesthesiologists; BMI: Body mass index; Hb: Hemoglobin; TNM: Tumor-node-metastasis.

Influencing factors of AL

Whether the patient had AL after surgery (not occurring = 0, occurring = 1) was used as the dependent variable, and the statistically significant factors identified by univariate analysis and shown in Table 1 (sex, diabetes mellitus, smoking history, neoadjuvant treatment, Alb level, tumor size, and tumor location) were used as independent variables. Table 2 lists the assignments of the indicators. A multivariate logistic regression analysis revealed sex [odds ratio (OR) = 3.656, 95% confidence interval (CI): 1.538-8.264, P = 0.003), diabetes mellitus (OR = 5.669, 95%CI: 2.455-13.092, P < 0.001), Alb level (OR = 0.898, 95%CI: 0.846-0.953, P < 0.001), tumor size (OR = 2.604, 95%CI: 1.840-3.684, P < 0.001), and tumor location (OR = 0.272, 95%CI: 0.180-0.413, P < 0.001) as factors that influence AL in patients with rectal cancer following sphincter-preserving surgery (Table 3).

Table 2 Assignment of each factor.

Factor	Assignment
Sex	Female = 0, male = 1
Diabetes mellitus	No = 0, yes = 1
Smoking history	No = 0, yes = 1
Neoadjuvant treatment	No = 0, yes = 1
Albumin level	Enter the original value
Tumor size	Enter the original value
Tumor location	Enter the original value

Table 3 Analysis of influencing factors of anastomotic leakage.

Factor	β	SE	Wald χ²	P value	OR (95%CI)
Sex	1.271	0.429	8.778	0.003	3.656 (1.538-8.264)
Diabetes mellitus	1.735	0.427	16.504	< 0.001	5.669 (2.455-13.092)
Smoking history	1.758	0.967	3.309	0.069	5.801 (0.873-38.572)
Neoadjuvant treatment	-0.947	0.940	1.015	0.314	0.388 (0.062-2.448)
Albumin level	-0.108	0.030	12.627	< 0.001	0.898 (0.846-0.953)
Tumor size	0.957	0.177	29.204	< 0.001	2.604 (1.840-3.684)
Tumor location	-1.300	0.212	37.699	< 0.001	0.272 (0.180-0.413)

CI: Confidence interval; OR: Odds ratio.

Nomogram model

According to the results presented in Table 3, five variables (sex, diabetes mellitus, Alb level, tumor size, and tumor location) were used to construct a nomogram model for predicting AL after sphincter-preserving surgery for rectal cancer (Figure 1).

Figure 1 Nomogram for predicting anastomotic leakage following rectal cancer surgery.

Decision tree model

A decision tree prediction model for AL after sphincter-preserving surgery for rectal cancer was also constructed, and four explanatory variables were screened: Tumor location, tumor size, Alb level, and sex. The results of the model showed that tumor location was the first-level factor influencing AL in patients with rectal cancer after sphincter-preserving surgery. The incidence of AL was 100% in patients with a distance of the tumor from the anal verge < 2.7 cm, and 78% in male patients with tumor location < 6.3 cm, Alb level < 41 g/L, and tumor size ≥ 5.1 cm (Figure 2).

Figure 2 Decision tree for predicting anastomotic leakage following rectal cancer surgery. AL: Anastomotic leakage.

Random forest model

According to the change in the overall prediction accuracy of the best model, the variables affecting AL in patients with rectal cancer after sphincter-preserving surgery were tumor location, tumor size, diabetes mellitus, sex, and Alb level (Figure 3).

Figure 3 Importance of variables in the random forest model.

Evaluation of the prediction efficacy of the three models

In the training dataset, the overall performance of the random forest model in predicting AL after sphincter-preserving surgery for rectal cancer was comparable to that of the decision tree model. The AUC of the random forest model was significantly higher than that of the decision tree model (Z = -2.836, P = 0.004) (Table 4 and Figure 4). In the validation dataset, the overall effectiveness of the three models was equivalent (Table 5 and Figure 5).

Figure 4 Receiver operating characteristic curves of the three models in the training set. A: Nomogram; B: Decision tree; C: Random forest. AUC: Area under the receiver operating characteristic curve; CI: Confidence interval.

Figure 5 Receiver operating characteristic curves of the three models in the validation set. A: Nomogram; B: Decision tree; C: Random forest. AUC: Area under the receiver operating characteristic curve; CI: Confidence interval.

Table 4 Prediction efficiency of the three model training sets.

Model	Sensitivity	Specificity	Accuracy	Recall	Precision	AUC (95%CI)
Nomogram	0.745	0.966	0.936	0.987	0.946	0.922 (0.883-0.961)
Decision tree	0.833	0.862	0.951	0.994	0.955	0.919 (0.863-0.975)
Random forest	1.000	1.000	0.951	0.994	0.955	1.000 (1.000-1.000)

AUC: Area under the receiver operating characteristic curve; CI: Confidence interval.

Table 5 Prediction efficiency of the three model validation sets.

Model	Sensitivity	Specificity	Accuracy	Recall	Precision	AUC (95%CI)
Nomogram	0.867	0.909	0.927	1.000	0.921	0.950 (0.908-0.992)
Decision tree	0.836	0.864	0.951	0.994	0.955	0.882 (0.797-0.968)
Random forest	0.984	0.727	0.893	1.000	0.889	0.934 (0.883-0.985)

AUC: Area under the receiver operating characteristic curve; CI: Confidence interval.

DISCUSSION

Rectal carcinoma is a prevalent cancer of the digestive system. Currently, the treatment of this disease is mainly surgical[10]. AL is one of the most common and dangerous complications associated with rectal cancer surgery. According to studies, the incidence of AL after rectal cancer surgery ranges from 2.6% to 19.0%[11]. AL not only affects recovery from surgery, but also leads to a variety of complications, such as intra-abdominal abscesses, diffuse peritonitis, and sepsis, and can even cause tumor recurrence. In severe cases, secondary surgery is required, which worsens patient survival rates[12]. Therefore, it is critical to identify the factors that influence the development of AL in patients with rectal cancer after sphincter-preserving surgery and provide targeted interventions to reduce its occurrence.

The incidence of postoperative AL was analyzed in 497 patients with rectal cancer admitted to our hospital. A total of 51 patients developed postoperative AL, representing an incidence of 10.26%. This is similar to the incidence of AL reported by Degiuli et al[13] in 5398 patients with rectal cancer (10.2%), but lower than that reported by Peltrini et al[14] in 367 patients with rectal cancer (17.4%). These differences may be related to factors such as inclusion criteria and different populations. AL after sphincter-preserving surgery for rectal cancer results from multiple factors, including patient characteristics, tumor status, and operation-related factors. The results of this study showed that sex, diabetes mellitus, Alb level, tumor size, and tumor location influence AL.

According to this study, males were 3.656 times more likely to have postoperative AL than women, which is consistent with the results of most studies[15,16]. As the male pelvis is narrow, the visual field is not fully exposed during the operation, making surgery more challenging; this may cause the rectal stump to retract and cause secondary injury[17].

Penna et al[18] confirmed that diabetes mellitus is an independent risk factor for AL after transanal total mesorectal excision. Additionally, this study reported that individuals with diabetes had an AL risk 5.669 times greater than that of non-diabetic patients. Diabetes mellitus can affect the anastomotic blood supply as uncontrolled hyperglycemia leads to vascular damage, reduced blood flow, and cellular accumulation of toxic glucose-derived metabolites, resulting in a significant decrease in anastomotic healing and the ability to fight infection[19].

According to the findings of this study, which are consistent with those of Shimura et al[20], there is a direct association between preoperative Alb levels and the risk of developing postoperative AL. This may be due to a low perioperative nutritional status leading to reduced immune function and a greater risk of infection and the spread of infection[21]. Yasui et al[22] found that patients with tumor sizes ≥ 4 cm were more likely to develop AL. This study also found that the probability of postoperative AL increases with tumor size. As more tissue has to be removed during surgery, more damage occurs and the chance of developing AL subsequently increases[23].

Both domestically and internationally, there is consensus on the impact of tumor location on AL following low anterior resection of rectal cancer; that is, the closer the tumor is to the anal margin, the higher the risk of AL[24-26]. The same conclusion was reached in the present study. This may be because the closer the tumor is to the anus, the larger the wound during resection. In addition, intraoperative electrocoagulation damage to the tissue and blood vessels causes exudation and bleeding, which reduces the blood supply to the anastomosis and increases the risk of postoperative AL[27].

Currently, there is no consensus regarding whether neoadjuvant therapy increases the incidence of AL. Arezzo et al[12] analyzed the effects of short- and long-term radiotherapy on AL and showed that the risk of postoperative AL significantly increased in patients receiving short-term radiotherapy. However, Chang et al[28] found no difference in the incidence of AL between patients with rectal cancer who underwent chemoradiotherapy before surgery, and those who did not. The present study also found that neoadjuvant therapy had no effect on AL occurrence.

With the development of computer software and artificial intelligence, machine learning has become a new direction for medical research. Studies have used machine learning models to predict the risk of anti-tuberculosis drug-induced liver injury in inpatients with tuberculosis[29], and machine learning can also predict the risk of essential hypertension[30]. In this study, five indicators with statistically significant differences in the multivariate analysis were used to establish three prediction models of AL after sphincter-preserving surgery for rectal cancer using machine learning algorithms. In the training dataset, the sensitivity, specificity, accuracy, recall rate, and precision rate of the nomogram were lower than those of the random forest and decision tree models, and the prediction efficacy of the random forest model was better than that of the decision tree. But the sensitivity, specificity, and AUC of the random forest model reached 1, indicating that the random forest model may have overfitting or the generalization effect may be poor due to insufficient data in the training set. In the validation dataset, the three models exhibited similar prediction performances. Although the random forest model has better predictive performance, it also has some disadvantages. As a result, in practice, each of the three prediction models has benefits and limitations, and the most appropriate method should be chosen based on the situation.

As this was a single-center retrospective study, it has several limitations. The representativeness of single-center research is limited, and there may be some bias owing to time constraints. This study did not analyze additional risk factors for AL after sphincter-preserving surgery for rectal cancer, and the constructed model may have been overfitted. Future research will use a larger and more comprehensive sample set and multicenter studies to verify and build a more complete prediction model.

CONCLUSION

Overall, AL is a serious complication of rectal cancer surgery, with a high incidence rate. In this study, nomogram, random forest, and decision tree prediction models of AL after sphincter-preserving surgery for rectal cancer were established using machine learning algorithms. The random forest model was found to have excellent predictive effect and stability, and might serve as a reference for the clinical identification of high-risk groups for AL following sphincter-preserving surgery for rectal cancer.

ARTICLE HIGHLIGHTS

Research background

With advances in medical technology, the success rate of sphincter-preserving surgery in patients with rectal cancer is increasing. However, anastomotic leakage (AL) remains a devastating complication.

Research motivation

AL significantly lowers patients’ quality of life. This study examines the elements that influence AL and establishes models to help doctors predict whether patients will develop AL, allowing the timely adoption of preventive measures.

Research objectives

This study aimed to identify the characteristics that influence AL and utilize these factors to build a prediction model for AL after sphincter-preserving surgery for rectal cancer.

Research methods

The clinical data of patients with rectal cancer who underwent sphincter-preserving surgery at our institution in the past five years were examined to analyze the factors influencing AL; nomogram, decision tree, and random forest prediction models were established; and the predictive efficacy of the three models was compared.

Research results

The factors influencing AL after sphincter-preserving surgery for rectal cancer were sex, diabetes mellitus, albumin level, tumor size, and tumor location. To predict the probability of postoperative AL, we constructed nomogram, decision tree, and random forest models.

Research conclusions

This study compared the predictive efficacy of the three prediction models. The random forest model performed the best and may be a useful alternative tool for predicting patients at a high risk of AL.

Research perspectives

Future research will include larger and more comprehensive cohorts across multiple centers, and build a more complete prediction model.