Flap failure prediction in microvascular tissue reconstruction using machine learning algorithms

Abstract

BACKGROUND

Microvascular tissue reconstruction is a well-established, commonly used technique for a wide variety of the tissue defects. However, flap failure is associated with an additional hospital stay, medical cost burden, and mental stress. Therefore, understanding of the risk factors associated with this event is of utmost importance.

AIM

To develop machine learning-based predictive models for flap failure to identify the potential factors and screen out high-risk patients.

METHODS

Using the data set of 946 consecutive patients, who underwent microvascular tissue reconstruction of free flap reconstruction for head and neck, breast, back, and extremity, we established three machine learning models including random forest classifier, support vector machine, and gradient boosting. Model performances were evaluated by the indicators such as area under the curve of receiver operating characteristic curve, accuracy, precision, recall, and F1 score. A multivariable regression analysis was performed for the most critical variables in the random forest model.

RESULTS

Post-surgery, the flap failure event occurred in 34 patients (3.6%). The machine learning models based on various preoperative and intraoperative variables were successfully developed. Among them, the random forest classifier reached the best performance in receiver operating characteristic curve, with an area under the curve score of 0.770 in the test set. The top 10 variables in the random forest were age, body mass index, ischemia time, smoking, diabetes, experience, prior chemotherapy, hypertension, insulin, and obesity. Interestingly, only age, body mass index, and ischemic time were statistically associated with the outcomes.

CONCLUSION

Machine learning-based algorithms, especially the random forest classifier, were very important in categorizing patients at high risk of flap failure. The occurrence of flap failure was a multifactor-driven event and was identified with numerous factors that warrant further investigation. Importantly, the successful application of machine learning models may help the clinician in decision-making, understanding the underlying pathologic mechanisms of the disease, and improving the long-term outcome of patients.

Key Words: Machine learning, Flap failure, Microvascular procedure, Random forest, Risk factors

Core Tip: Flap failure is a rare but severe event in microvascular tissue reconstruction. It is generally associated with the additional economic burden and mental stress to the patients. Therefore, identifying the risk factors and screening high-risk patients carries a significant value in the clinical practice. Machine learning is an artificial intelligence based on the computer learning to learn from data and thus automatically make decisions. This retrospective study applied machine learning for the risk factor analysis of flap failure during microvascular tissue reconstruction.

INTRODUCTION

Microvascular reconstruction surgery is a commonly used technique to treat various defects, including remodeling after mastectomy[1], head and neck trauma repair[2], and profound burn tissue remodeling[3]. However, even though this technique is quite robust, many adverse complications arise after the reconstruction procedure, such as postoperative incision infection and reoperation[4,5]. Among these complications, the failure of flap transplantation after microvascular tissue reconstruction is the most important event, as it is associated with the arterial blood supply, ischemia-reperfusion, and venous return of the flap[6,7]. Despite the rare occurrence of flap failure, it can result in devastating consequences for patients, such as permanent scarring of the face and breast. Moreover, it increases complication in postoperative care, length of hospital stays, financial burden, and mental stress to the patients[8,9]. Therefore, it is important to identify the relevant factors and screen out high-risk patients before surgery, which might result in flap failure.

Previous studies have analyzed the multifactorial aspect of flap failure[10,11]. Associated preoperative risk factors include, but are not limited to age, smoking, diabetes, hypertension, and obesity[12,13]. Related intraoperative factors included the surgeon’s lack of experience and the choice of free flaps[14]. However, these studies were based on traditional logistic regression methods and were limited to nonlinearity and variable set[15]. Of note, precise, logistic regression analysis assumes that variables are linearly correlated, and therefore potential nonlinear interactions can compromise the outcome[15,16]. Moreover, only a small number of variables could be included in the analysis, overlooking the many potentially relevant factors[16]. These deficiencies in the analytical methods needs to be addressed using an advanced algorithm. Therefore, the recently emerging algorithms of machine learning might be a better option for the data analysis.

Machine learning, a branch of artificial intelligence (AI), literally meaning where machines can understand and learn from data to make decisions like humans[17,18]. In 2017, an AI called AlphaGo won worldwide attention by beating the international GO master Li Shiming. Due to its advantages in computational capacity and problem-solving techniques, machine learning has been widely used in medicine for many purposes, including the interpretation of test results[19], diagnosis of skin diseases[20], pathology[21], prediction of adverse complications[22], and the prognosis of cancer patients[23]. However, in plastic surgery the use of clinical application of machine learning is still rare[24]. Therefore, this study aimed to apply AI in the field of plastic surgery, assessing the factors associated with the prognosis of microvascular tissue reconstruction for identifying high-risk patients with flap failure.

MATERIALS AND METHODS

Study population

A total of 946 consecutive patients were recruited in the study from January 1, 2006 to December 12, 2020. These recruited patients underwent microvascular tissue reconstruction of free flap for head and neck, breast, back, and extremity at the Department of Plastic Surgery of Affiliated Hospital in Guangdong Medical University. Exclusion criteria included: patients with more than 30% data loss and who refused surgical treatment. Inclusion criteria/variables included: (1) Preoperative variables such as sex, age, body mass index (BMI), smoking, alcohol use, blood pressure, medication history, complications, laboratory findings, preoperative chemotherapy, preoperative radiotherapy, free flap location, and recipient surgical site; and (2) Intraoperative variables like duration of operation, duration of anesthesia, hypotensive events, use of vasoactive agents, duration of flap ischemia, number of vascular anastomoses, use of venous grafts, and surgeon’s experience measured as the time since the flap procedures.

Statistical analysis

Open-source software Python (version 3.6) and Scikit-learn package (https://scikit-learn.org/) were used for the data processing and analysis. Univariate analyses were done using the χ² and Fisher’s exact tests for categorical variables, whereas the t-test and Mann-Whitney U tests were used for the continuous variables. A subset of data was usually selected from the entire database for model training to train a suitable algorithm. The rest of the subset was used for the performance test of the model. Conceptually, the whole data set was divided into a training and testing subset according to the ratio of 5:5. Then, GridSearch was performed with the 5-fold cross-validation, where the training data set was further split into five parts and five repetitions. At each repetition, there were four random parts that served as the training set, whereas the remaining part served as the testing set. Multivariable regression was performed for the most critical variables in the random forest model to identify the risk factors in the traditional logistic regression model. A P value less than 0.05 was considered statistically significant.

Machine learning technique

We employed the following machine learning methods:

Random forest: The random forest classifier, one of the most used techniques in the data mining or automatic learning, was developed from the training data set using the python programming software. Random forest, introduced by Ishwaran, was used as decision tool based on a binary tree. It uses a branching structure like a binary tree to form a decision model and analyze possible results. Each node in these binary tree structures represents a decision (based on selected variables), whereas the two branches of the node represent the two kinds of classification results. Each branch produces two leaf nodes and other subtrees, depending on the classification when the variable is analyzed. For assessing the variables importance, variables in the random forest are determined by the average distance of the branching nodes in the tree structure from the roots. Thus, the higher a variable is in an inverted binary tree, the closer it would be to the root, with the higher ranking.

Support vector machine: Support vector machine is an algorithm for creating nonlinear discriminative classifier, governed by an optimal hyperplane that separates examples of different classes (the notable kernel trick).

Gradient boosting machine: Gradient boosting is a supervised machine learning technique for solving regression and classification problems that yield predictive models in the form of an ensemble of weak predictive models (e.g., decision trees). Through pooling weak predictive models into a more powerful and reliable prediction model, the gradient tree boosting technique incorporated in the eXtreme Gradient Boosting system becomes a robust machine learning classifier.

Performance evaluation

The model evaluation used performance indicators used in the machine learning. The primary evaluation method was the receiver operating characteristic curve and the area under the curve (AUC) score. Other relevant indicators included accuracy, precision, recall, and F1 score. The higher value indicators represented the better predictive performance of the model.

RESULTS

Patient characteristics

A total of 946 patients who underwent free flap transplantation for head and neck (40.2%), breast (38.3%), and extremity reconstruction (21.5%) were recruited. Overall, 58.3% of the recruited population was female, with an average age of 42 years (range: 13-65 years). The average BMI of the studied population was 24.9 ± 6.3 (mean ± standard deviation). Other potential factors for flaps failure were obesity (23.4%), smoking (30.3%), diabetes (6.3%), insulin (1.3%), hypertension (16.2%), preoperative tumor chemotherapy (25.3%), and preoperative tumor radiotherapy (19.2%). Table 1 showed the clinical characteristics of the patients in the training and the test sets. However, no significant statistical difference was observed between the two subsets.

Table 1 Patient characteristics.

Variables	Training set	Test set	P value
Patient population, n	473	473
Age (yr)	41 (13-64)	43 (15-65)	0.115
Male, n (%)	274 (57.9)	278 (58.8)	0.258
BMI (kg/m2)	25.3(16.9-32.8)	25.9 (16.7-35.5)	0.079
Systolic blood pressure	119 (87-165)	121(85-177)	0.658
Smoking, n (%)	142 (30.0)	145 (30.7)	0.583
Alcohol, n (%)	163 (34.5)	172 (36.4)	0.158
Diabetes, n (%)	34 (7.2)	26 (5.5)	0.098
Insulin, n (%)	8 (1.7)	4 (0.8)	0.059
Hypertension, n (%)	73 (15.4)	80 (16.9)	0.113
Preoperative chemotherapy, n (%)	117 (24.7)	122 (25.8)	0.358
Preoperative radiotherapy, n (%)	100 (21.1)	82 (17.3)	0.663
Obesity, n (%)	112 (23.7)	109 (23.0)	0.487
WBC (× 103/µL)	7.5 (3.2-14.3)	7.2 (3.1-15.9)	0.226
Hemoglobin (mg/dL)	12.6 (9.8-16.6)	12.9 (10.1-16.9)	0.460
PLT (× 103/µL)	156 (102-253)	165 (113-267)	0.115
Creatinine (mg/dL)	0.89 (0.69-1.20)	0.83 (0.65-1.15)	0.328
Glucose (mg/dL)	10.5(5.1-16.5)	11.3 (4.4-18.8)	0.085
Cholesterol (mg/dL)	159.2 (137.3-195.3)	144.0 (127.4-199.8)	0.075
Beta blockers, n (%)	51 (10.8)	55 (11.6)	0.165
Aspirin, n (%)	43 (9.1)	47 (9.9)	0.392
Flap ischemia time (min)	123 (108-145)	117 (101-153)	0.558
Hypotensive events, n (%)	11 (2.3)	15 (3.2)	0.663

BMI: Body mass index; PLT: Platelet; WBC: White blood cell.

Clinical outcomes

Major complications after flap transplantation were hematoma in 69 cases (7.3%), infection in 49 cases (5.2%), and damaged flap circulation in 65 cases (6.9%). Salvage measures were implemented for 95 cases (10%), where 61 cases were successfully saved, with a success rate of 64.2%. Finally, 34 patients (3.6%) had flap failure, with the most common cause of postoperative infection followed by hematoma formation.

Model performance

We developed three machine learning-based models based on the various preoperative and intraoperative data for analyzing the potential risk factors associated with the flap failure after microvascular tissue reconstruction. A total of 473 patients and 16 events were included in the training set, while a total of 473 patients and 18 events were included in the test set. The receiver operating characteristic graph was drawn based on model sensitivity and specificity, whereas the random forest model yielded the highest AUC score in the test set (AUC = 0.770, 95% confidence interval: 0.726-0.854) (Figure 1). The random forest model maintained a very high predictive ability for predicting the flap failure events, indicating that the classification model based on the binary tree could accurately divide the samples into with and without flap failure events. Other model indicators in the random forest were: (1) The value of precision based on the true positive divided by the sum of true positive and false positive was 0.82; (2) The values of recall obtained by dividing true positive by the sum of the true positive and false negative was 0.69; and (3) The values of the F1 score obtained by the precision-recall curve was 0.75 (Table 2).

Figure 1 Receiver operating characteristic curve of the machine learning models in the testing set. AUC: Area under the curve; CI: Confidence interval.

Table 2 The model performance of the machine learning classifiers for predicting flap failure.

	Accuracy	Precision	Recall	F1 score	AUC
Random forest	0.78	0.82	0.69	0.75	0.770
Support vector machine	0.71	0.79	0.58	0.67	0.720
Gradient boosting	0.68	0.76	0.53	0.65	0.707

AUC: Area under the curve.

Importance rank

Figure 2 represents the importance ranking of tested variables in the random forest model to predict flap failure. The variables were ranked based on the average distance from the split branch to the tree root in the binary tree. The line length measured the variable importance in the random forest model (Figure 2). The top ten variables in the random forest model were age, BMI, ischemia time, smoking, diabetes, experience, prior chemotherapy, hypertension, insulin, and obesity.

Figure 2 Ranked variable value of the random forest algorithm. The variables were ranked based on the average distance from the split branch to the tree root in the binary tree. The line length in the graph measures the variable importance in the random forest model. The top ten variables in the random forest model were age, body mass index, ischemia time, smoking, diabetes, experience, prior chemotherapy, hypertension, insulin, and obesity. BMI: Body mass index.

Multivariate regression

Table 3 outlines the statistical analysis results of the top ten variables of the random forest model in the traditional logistic regression analysis. Of note, among the top ten variables, only age, BMI, and ischemic time were significantly associated with the outcomes of the multivariate analysis. For the remaining seven variables, P values for diabetes and prior chemotherapy were 0.06 and 0.07, respectively. Interestingly, surgeon’s experience was not found to be statistically significant in the multivariate analysis.

Table 3 Multivariate logistic regression model for top 10 variables in random forest.

Variables	Odds ratio (95%CI)	P value
Age	1.56 (0.57-5.87)	0.04
Body mass index	2.83 (0.68-5.54)	0.02
Ischemia time	1.98 (0.53-3.24)	0.001
Smoking	1.13 (0.28-2.89)	0.87
Diabetes	1.15 (0.53-3.28)	0.06
Experience	0.86 (0.18-4.87)	0.79
Prior chemotherapy	1.15 (0.56-2.68)	0.07
Hypertension	1.08 (0.25-2.64)	0.28
Insulin	1.27 (0.64-3.21)	0.54
Obesity	1.09 (0.57-2.95)	0.13

CI: Confidence interval.

DISCUSSION

Free flap transplantation is a robust technique, ensuring the success of tissue reconstruction even with various postoperative complications through timely rescue attempts[25]. However, though the incidence of flap failure is relatively low, once it occurs it is generally detrimental for the patient, as it results in the permanent scarring of the graft area, especially at the region of the face and breast[26]. Therefore, in this study a random forest model based on machine learning was used for a series of preoperative and intraoperative variables, aiming to assess and analyze the risk factors associated with the flap failure after microvascular tissue reconstruction and to screen out the high-risk groups in clinical practice. To best of our knowledge, this is the first report about the application of the random forest model for flap failure after microvascular tissue reconstruction.

When the event-to-variable ratio was greater than 10, an ideal prediction model in multivariate logistic regression analysis was successfully constructed[27]. However, owing to the low incidence of flap failure, the event-to-variable ratio in this study was approximately 1. Therefore, even reducing the variables of the analysis could not achieve the ideal ratio value. Moreover, the traditional logistic regression could not consider the nonlinear relationship between the variables[15]. Therefore, in this study, due to the potential overfitting phenomenon, the utility of the prediction model based on the traditional multivariate analysis might be compromised. The phenomenon partially explains that only three factors, including age, BMI, and ischemia time, were considered statistically significant for flap failure using the multivariate analysis.

Other research has widely explored the comparison between the traditional logistic regression and emerging machine learning for the data analysis. In 2018, Lee et al[15] published a study for predicting acute kidney injury after liver transplantation. Their research indicated that the AUC score of the prediction model based on machine learning could reach up to 0.90, while that of the logistic regression model was only 0.61[15]. In 2020, Arkin et al[28] analyzed 30-d survival prediction of cancer patients by comparing the machine learning and logistic regression to understand the better statistical methods for the relatively accurate prediction of survival. Their results showed that the machine learning-based artificial neural network yielded a higher AUC score of 0.86 than the AUC score of 0.76 in the logistic regression model[28]. Considering the abovementioned pitfall of event-to-variable ratio, several techniques, such as GridSearch, 5-fold cross-validation, and oversampling to avoid potential overfitting defects were used in the current study. We found the adopted machine learning models achieved the highest AUC score of 0.772 in the random forest classifier, as to the interesting outcome of flap failure. The effectiveness of our machine learning model was similar to that of the predictive models demonstrated in other studies. Formeister et al[29] yielded a decision tree model that could correctly classify outcomes with an accuracy ranging from 65% to 75%. O’Neill et al[30] achieved an AUC of 0.95 in the training set and 0.67 in the testing set for 2012 patients within microvascular breast reconstruction.

The potential risk factors involved in the development of flap failure and the contribution ranking in a random forest classifier is shown in Figure 2. These observations are suggestive that the occurrence of flap failure is a multifactor-driven event with the identified numerous factors. Reported preoperative risk factors included BMI, ischemia time, and limited surgical experience[31-33]. Specifically, it is widely accepted that there was an increase in the postoperative complications for the free flap transplantation in the obese patients[34]. At the same time, Chang et al[35] recommended that the microvascular tissue reconstruction should be performed with full discretion of patients with high BMI. Additionally, prolonged ischemia time of the free flap and subsequent ischemia-reperfusion injury can also increase the risk of postoperative complications and eventually flap failure[36].

There were some limitations for our study. First, this was a single-center retrospective study. Thus, although the model achieved high accuracy, relatively few factors and limited cases were included. Second, we did not perform the external validation of the samples from other institutions, so there might be differences that occur in the results obtained from other institutions and larger data sets. Third, we only developed one machine learning model of the random forest classifier, which may become more efficient if we would have used more algorithms during the data analysis. Finally, although we showed the importance of ranking variables in the random forest models, the black-box effect of the predictive models and the analytical decision on the samples remain ambiguous.

CONCLUSION

In our study, the random forest machine learning technique predicted the flap failure in patients following the microvascular tissue reconstruction for head and neck, breast, and extremities. We also identified the relevant risk factors of the outcome and further analysis in the traditional multivariate logistic regression. The findings of our study will help plastic surgeons to identify the potential risk factors associated with the flap failure and in screening high-risk events. These observations will eventually assist the clinician in decision-making by understanding the underlying pathologic mechanisms of the disease and improving the long-term outcome of the patients. Future multicentric research is required to develop an AI-based, big-data-driven clinical decision support system with a larger sample size.

ARTICLE HIGHLIGHTS

Research background

Microvascular tissue reconstruction is a well-established technique used for the wide variety of tissue defects. However, still a risk of experiencing flap failure exist that eventually results in additional hospital stays, financial burden, and mental stress of the patients.

Research motivation

The application of the machine learning technique in flap failure events remains an underestimated area.

Research objectives

The objective of the current study was to develop machine learning-based predictive models for the flap failure to identify potential factors and screening the high-risk patients.

Research methods

To establish machine learning classifiers, we used a data set with 945 consecutive patients who underwent microvascular tissue reconstruction. Model performances were evaluated by the indicators including area under the receiver operating characteristic curve, accuracy, precision, recall, and F1 score. A multivariable regression analysis was also performed for the essential variables in the random forest model.

Research results

The flap failure event occurred in 152 patients (1.9%) after the operation. The random forest classifier based on various preoperative and intraoperative variables performed the best, with an area under the curve score of 0.770 in the test set. The top variables in the random forest were age, body mass index, ischemia time, smoking, diabetes, experience, prior chemotherapy, hypertension, insulin, and obesity.

Research conclusions

Machine learning models were successfully developed for identifying the potential factors and screening out the high-risk patients for the interesting outcome of flap failure.

Research perspectives

In our study, the machine learning technique correctly predicted flap failure in the patients who followed microvascular tissue reconstruction. Results from our research will help the clinician in decision-making by better understanding the underlying pathologic mechanisms of the disease and improving the long-term outcome of patients.