Development of a random forest model for hypotension prediction after anesthesia induction for cardiac surgery

Abstract

BACKGROUND

Hypotension after the induction of anesthesia is known to be associated with various adverse events. The involvement of a series of factors makes the prediction of hypotension during anesthesia quite challenging.

AIM

To explore the ability and effectiveness of a random forest (RF) model in the prediction of post-induction hypotension (PIH) in patients undergoing cardiac surgery.

METHODS

Patient information was obtained from the electronic health records of the Second Affiliated Hospital of Hainan Medical University. The study included patients, ≥ 18 years of age, who underwent cardiac surgery from December 2007 to January 2018. An RF algorithm, which is a supervised machine learning technique, was employed to predict PIH. Model performance was assessed by the area under the curve (AUC) of the receiver operating characteristic. Mean decrease in the Gini index was used to rank various features based on their importance.

RESULTS

Of the 3030 patients included in the study, 1578 (52.1%) experienced hypotension after the induction of anesthesia. The RF model performed effectively, with an AUC of 0.843 (0.808-0.877) and identified mean blood pressure as the most important predictor of PIH after anesthesia. Age and body mass index also had a significant impact.

CONCLUSION

The generated RF model had high discrimination ability for the identification of individuals at high risk for a hypotensive event during cardiac surgery. The study results highlighted that machine learning tools confer unique advantages for the prediction of adverse post-anesthesia events.

Key Words: Anesthesia, Hypotension prediction, Cardiac surgery, Random forest, Machine learning

Core Tip: This was a retrospective study intended to develop a prediction model for hypotensive events after anesthesia during cardiac surgery. A random forest machine learning technique was used to establish a predictive algorithm using preoperative data. “Features ranked by importance” were also identified in this study. This novel prediction model can be used to predict hypotension events and help to avoid the occurrence of any potential adverse events.

INTRODUCTION

Post-induction hypotension (PIH), defined as hypotension after the induction of anesthesia, is a common event that is often associated with unfavorable patient outcomes. In certain cases, it might cause adverse consequences including longer hospital stays and even death[1]. In cardiac surgery, PIH is known to be associated with postoperative adverse events[2]. During surgical procedures, anesthesiologists take all necessary precautions to ensure careful administration of medication, however, PIH can occur without warning, and is usually followed by a series of adverse events including acute kidney injury, myocardial infarction, and sometimes mortality within 30 d[3]. Therefore, devising effective strategies for early detection of intraoperative hypotension along with preventive measures could help anesthesiologists to improve the outcomes of anesthesia and surgical procedures. A variety of complex factors are known to contribute to PIH, including age, preinduction systolic blood pressure (SBP), comorbidities, and preoperative medication[4]. The existence of interlocking relationships between different factors limits the utility of simple algorithms in predicting PIH during cardiac surgery. Alternatively, machine learning might offer a more sophisticated modeling technique to enable anesthesiologists to predict PIH, by combining various data characteristics.

In the last 30-40 years, advances in machine learning have revolutionized the field of medicine. Additionally, recent progress in the field of computing power and big-data processing ensured breakthroughs in clinical data analysis[5]. In general, machine learning approaches are focused on utilizing multidimensional data and computational methods to generalize predictions about individuals. The random forest (RF) model, first proposed by Breiman, is a robust model algorithm of machine learning that is capable of synthesizing and analyzing all kinds of datasets[6]. Complex nonlinear interactions among variables are managed by computers to minimize the inconsistencies between the observed and the predicted results, thereby ensuring increased accuracy of disease prediction[7]. The formation of multiple decision trees allows the transfer of all patient features to the trees, and a final classification is generated in terms of “voting” by decision trees. A prediction model based on an RF algorithm has been previously applied to several fields of clinical medicine. In particular, it has been used for the prediction of cardiovascular disease[8], the clinical outcome after aneurysm rupture at the time of discharge[9], diagnosis of polycystic ovary syndrome[10], and the effect of chemotherapy on patient tumors[11]. However, very few studies have reported the use of RF for the prediction of PIH in patients undergoing cardiac surgery. This study aimed to use the RF algorithm as a powerful tool to learn feature presentations and establish a prediction model for PIH in patients subject to cardiac surgery. The study involved the construction of a predictive RF model for PIH events based on patient characteristics. An RF classifier was used to identify key characteristics, and a ranking structure was established to define the importance of various features using the algorithm rules. The accuracy and superiority of the generated diagnostic RF model were evaluated by analysis of the area under the curve (AUC).

MATERIALS AND METHODS

Study design and population

A flow chart of the study protocol is shown in Figure 1. The electronic medical records of a total of 3030 consecutive patients treated from December 2007 to January 2018 were screened. Those who were younger than 18 years of age, had abandoned surgical treatment, and those with incomplete preoperative data were excluded. The patients included in the study database were divided into two subsets, 70% were allocated to the RF model training subset and 30% were allocated to the testing subset for model validation. An RF algorithm was used to establish the PIH prediction model. The definition of PIH included an SBP threshold of < 90 mmHg or a mean blood pressure (MBP) of < 65 mmHg.

Figure 1 Research protocol flow chart. A total of 4504 patients who underwent cardiac surgery were screened for inclusion; 1474 were excluded. The 3030 who were eligible and included in the study were divided into a training set and a testing set. The random forest algorithm was applied to the training set for the modeling process and the parameters were debugged. The model was then validated in the test set.

Data collection

We collected preoperative and perioperative variables from electronic health records. Table 1 summarizes the data of the training and test sets. Preoperative variables included age, sex, body mass index (BMI), underlying disease, EuroSCORE I, and the American Society of Anesthesiologists (ASA) score. Experimental findings included hemoglobin, serum creatinine, and total bilirubin. Data on the patient’s preoperative medications, such as the use of beta blockers, insulin, and aspirin were also included. Intraoperative medications were introduced after entering the operating room until 10 min after the induction of anesthesia. Data on perioperative blood pressure were extracted from the anesthesia information management system. The obtained data were identical to those in the electronic health records.

Table 1 Patient characteristics.

Variable	Training set	Test set	P value
Patient population, n	2121	909
Age (yr)	63 (53-70)	62 (53-69)	0.323
Male, n (%)	1444 (68.1)	591 (65.0)	0.307
BMI (kg/m2)	24.8 (17.8-29.8)	24.8 (17.3-28.9)	0.956
Smoking, n (%)	775 (36.5)	328 (36.1)	0.937
Surgery type, n (%)
CABG only	750 (35.4)	378 (41.6)	0.128
Valve surgery only	1113 (52.5)	455 (50.0)	0.560
CABG + valve surgery	258 (12.2)	76 (8.4)	0.156
Previous cardiac surgery, n (%)	267 (12.6)	166 (18.3)	0.052
Preoperative ventilator support, n (%)	148 (7.0)	54 (5.9)	0.603
Preoperative IABP, n (%)	45 (2.1)	40 (4.5)	0.096
Diabetes mellitus, n (%)	687 (32.4)	284 (31.2)	0.756
Dyslipidemia, n (%)	800 (37.7)	365 (40.1)	0.564
Cerebrovascular accident, n (%)	123 (5.8)	54 (5.9)	0.926
Myocardial infarction within 90 d, n (%)	329 (15.5)	135 (14.9)	0.879
Serum creatinine (mg/dL)	0.91 (0.71-1.16)	0.90 (0.70-1.14)	0.128
Hemoglobin (g/dL)	13.2 (11.4-14.6)	13.1 (11.4-14.3)	0.403
Preoperative EF (%)	62.0 (35.0-70.0)	61.0 (30.0-71.3)	0.323
ASA score, n (%)
Class I	21 (1.5)	14 (1.5)	0.994
Class II	380 (17.9)	157 (17.3)	0.856
Class III	1525 (71.9)	666 (73.3)	0.708
Class IV	172 (8.1)	58 (6.4)	0.455
EuroSCORE I	3.2 (0.83-8.95)	2.96 (0.67-9.65)	0.355
Beta blocker use, n (%)	959 (45.2)	383 (42.1)	0.455
Aspirin use, n (%)	575 (27.1)	243 (26.7)	0.926
Dobutamin use, n (%)	19 (0.9)	14 (1.5)	0.460
Insulin use, n (%)	99 (4.7)	76 (8.4)	0.059
Systolic blood pressure	111 (88-154.8)	119 (95-165.5)	0.658
Diastolic blood pressure	73 (55-84)	75 (58-89)	0.537
Mean arterial pressure	93 (71-119)	108 (68-121)	0.437

ASA: American Society of Anesthesiologists; BMI: Body mass index; CABG: Coronary artery bypass graft; EF: Ejection fraction; IABP: Intra-aortic balloon pump.

RF model

The study used an RF algorithm, which is a supervised machine learning algorithm. Briefly, RF is based on multiple decision trees that are merged to obtain a stable and accurate prediction[12]. To develop trees, RF uses a resampling technique. The classification was generated based on voting by trees, as per the specific value for the variable. Finally, the model was predicted based on majority votes[13]. Similar to a tree structure, the RF model consisted of roots, branches, and leaves that reflected the derivation of the relationship of features and events. During the construction of the RF model, the database was randomly divided into N subsets, each of which developed a tree-like structure based on the incorporated variables. Following the definition of information gain, variables were given an entropy value after being processed within the tree. As shown in Figure 2, the variables that achieved peak values were placed at the top node location, closest to the root of the tree[8]. The value of a variable was indicated by the mean decrease in the Gini (MDG) index[14]. It has been established that the higher the index value, the more important is the value given by the variable. To evaluate the performance of the RF model, the AUC of the receiver operating characteristic (ROC) curve was calculated. The Scikit-learn package (https://scikit-learn.org/stable/) was used for the development of the RF model.

Figure 2 Illustration of the random forest built from a collection of trees. The random forest adapted the bootstrap sampling method. All participants were randomly selected from the dataset to form the N subset. By forcing each split to consider only a subset of predictors, all predictors were allowed a chance to reveal their importance.

Data analysis

Python 2.7 was used for the data analysis. Continuous variables were reported as means ± SD, and between-group differences were evaluated by a t-tests. Categorical variables were reported as n (%) with and differences were evaluated by Fisher’s exact tests. P < 0.05 was considered to be the threshold of statistical significance.

RESULTS

Patient characteristics

The records of 3030 patients who underwent cardiac surgery between December 2007 and January 2018, were obtained from the electronic medical record system. The patients were 62 ± 8.9 years of age, 2035 (67.2%) were men, 1103 (36.4%) were smokers, 818 (27.0%) took aspirin, 971 (32.0%) had diabetes, and 175 (5.8%) used insulin. Coronary artery bypass graft surgery was performed in 1128 patients (37.2%), valve replacement in 1568 (51.7%), and coronary artery bypass plus valve surgery in 334 (11.0%) patients. The patients were randomly assigned to training and testing subsets in a ratio of 7:3. The baseline characteristics for the training and testing sets are summarized in Table 1. Importantly, no statistically significant differences between the two sets were recorded (P > 0.05).

PIH morbidity

The data obtained from the anesthesia information management system revealed that PIH was reported in 1578 patients (52.1%) during anesthesia before surgery. The incidence of PIH was 51.3% in the training set (1088/2121) and 53.9% in the test set.

RF development

For the prediction of PIH, an RF model was generated using a total of 2121 training samples, with all the variables used as input variables. A schema of the construction of the RF model is shown in Figure 2. Briefly, the RF model was an integrated algorithm composed of multiple decision trees. It was characterized by anti-overfitting and anti-noise abilities. During training, a 10-fold cross-validation strategy was adopted. The RF model adopted the bootstrap sampling method in which N participants were randomly selected from the dataset and included in the training subset. The remaining participants were categorized as the testing subset. Each split was forced to consider only a subset of predictors, which allowed all predictors to reveal their importance. The final determination of the model predictions was based on the majority votes.

Model performance

To evaluate the effectiveness of the prediction model, 909 participants from the testing subset were used. The accuracy of the RF model was 83.1%, the sensitivity was 78.8%, and the specificity was 85.6%. ROC curve analysis was used to evaluate the effectiveness of the prediction model. The horizontal and vertical coordinates of this curve represented the sensitivity and 1 − specificity, respectively. For the ROC curve, the AUC is 0.50 when calculated by random prediction. An AUC value of 1 represents 100% identification, and an AUC > 0.8 indicates that the model is characterized by a high degree of discrimination. As a general rule, a higher value of AUC is associated with better performance of the model. In this study, an RF-based risk assessment model was developed for PIH prediction in patients, during cardiac surgery. The RF model could assist the doctors in the identification of patients at increased risk of experiencing PIH during surgery. The ROC-AUC coupled with sensitivity and specificity is shown in Figure 3. In terms of the effectiveness of the model to predict PIH, RF showed a high discrimination capacity (AUC = 0.843; 95%CI: 0.808-0.877).

Figure 3 The receiver operating characteristic curve of the random forest model. AUC: Area under the curve.

Importance of various patient features

The ranked importance of features in the RF model was determined by the MDG index. The variables were ranked both by the mean decrease in accuracy and the Gini index. As shown in Figure 4, the 10 most important variables were MBP, age, BMI, ejection fraction, ASA score, preoperative use of an intra-aortic balloon pump, diabetes mellitus, EuroSCORE I, serum creatinine, and beta blocker use. Among the variables, MBP had the most significant impact, followed by age. Interestingly, the indicator of the pumping capacity of the heart ranked fourth.

Figure 4 Ranked importance of the random forest model. The importance of each variable was ranked by the mean decrease in the Gini index. ASA: American Society of Anesthesiologists; BMI: Body mass index; EF: Ejection fraction; IABP: Intra-aortic balloon pump; MBP: Mean blood pressure.

DISCUSSION

Several machine learning approaches have been previously developed and used in the field of anesthesia, particularly for postoperative pain management[15,16] and assessment of patient use of analgesic pumps[17]. In this retrospective study, the application of the RF model in predicting PIH events during cardiac surgery was investigated using preoperative and perioperative variables. The final model, generated using an RF algorithm, had a strong differentiation ability in the testing set (AUC = 0.843, 95%CI: 0.808-0.877). Machine learning models have been previously shown to be superior to traditional logistic regression methods for the prediction of mortality[18] and aortic aneurysm surgery[19]. Compared with the RF method, logistic regression methods presume a linear dependence during data analysis, which might be responsible for the low prediction precision of those methods[20]. Given that machine learning has a powerful data processing capacity, the result of this investigation is not surprising, namely that machine learning methods have high predictive performance because they combine numerous variables into a unified algorithm.

PIH is a common event, with a prevalence of approximately 20%[21]. PIH is known to be associated with significant postoperative adverse events. In a previous study, Maheshwari et al[22] reported the occurrence of PIH after induction of anesthesia, which resulted in an increased postoperative hospital stay and even death. Interestingly, despite the use of a significantly high threshold value for hypotensive criteria, hypotensive events were reported in 1578 participants (52.1%). Such a high incidence of hypotensive events might have been the result of continuous monitoring by invasive blood pressure measurements during cardiac surgery. In comparison, blood pressure measurements are normally performed every 3 to 5 min during noncardiac surgery.

Ranking of the importance of various features showed that MBP was the most significant contributor to the prediction of PIH in cardiac surgery. The importance of MBP in this study seems reasonable, as the anesthesiologist usually tends to predict the blood pressure via monitoring of circulatory system dynamics during anesthesia induction. Patient age was found to be the second most important predictor, which might have reflected the high vulnerability of old individuals to hypotensive events compared with young patients. Among the other characteristics, EF ranked fourth in the impact on PIH prediction. It is physiologically plausible for patients who underwent cardiac surgery to have an impairment in heart pump function. Given that machine learning has taken the form of a black box in the prediction of PIH, results of the present study might not provide convincing evidence for anesthesiologists in predicting PIH events. However, the results were identical to those reported by Kendale et al[23] who found that MBP and age were the most significant contributors of hypotensive events. In another study, Sudfeld et al[4] used multivariate logistic regression and identified MBP and age as significantly associated with PIH events.

The RF model integrated various characteristic features of individuals, and thus displayed an effective ability to identify patients at increased risk for PIH events. It is well established that effective monitoring of vital signs, maintenance of circulatory stability, and rapid response along with intervention are effective measures that help to reduce adverse complications associated with surgery. However, despite the use of sophisticated modern equipment by experienced anesthesiologists, the monitoring of hemodynamic changes in blood flow and early detection of poor performance is not easy. Therefore, it is clinically important to develop a diagnostic alternative tool that leverages the available information to provide accurate judgments that assist in clinical decision making. In the past few years, machine learning has gained immense attention in the field of clinical and medical research. Recent studies that utilized machine learning, reported a relationship between neural networks and hemodynamic changes[23-25]. The widely established electronic medical record system includes a large amount of clinically relevant information about each individual and provides a rich database for the development of a prediction model. The application of an RF-based model ensures effective processing and analysis of such large datasets. Furthermore, in the era of big data, electronic health records make real-time monitoring of patients feasible[26]. All these results highlight the suitability of the RF algorithm and other machine learning tools as a powerful model for data analysis, which can assist anesthesiologists and other clinicians in decision making, and thus aid in improving post-anesthesia patient outcomes.

This study had some limitations. The retrospective nature of the study makes it vulnerable to selection bias, and also makes it impossible to identify a causal link. As the quality of data obtained from the database might affect the prediction performance of the RF model, some factors might have been excluded from the analysis, which might have changed the prediction results. The study only included patients who underwent cardiac surgery with total intravenous anesthesia administered by an infusion pump. There might thus be some discrepancies in the results obtained from patients with noncardiac surgery. The study did not include external validation of the generated model. Future research should focus on the integration of the model with patient electronic health data, medical information, and imaging examination to construct a real-time clinical decision-support system.

CONCLUSION

Several previous studies have reported better performance of RF models compared with traditional models, particularly in the prediction of mortality in cardiac surgery[27]. In this study, RF modeling had a high competence in predicting PIH in cardiac surgery. The incorporation of perioperative variables into machine learning modeling might improve the ability to identify patients at increased risk of PIH during cardiac surgery. In this era of personalized medicine, precise machine learning modeling based on accessible individual characteristics might provide an opportunity for early intervention for PIH management by anesthesiologists.

ARTICLE HIGHLIGHTS

Research background

Hypotension, which most often occurs on the induction of anesthesia, is known to lead to the development of adverse events and poor outcomes in patients following surgery. However, risk scores based on conventional logistic regression analysis have a low ability to discriminate characteristics that influence the development of post-induction hypotension (PIH) events.

Research motivation

Recently, a model based on machine learning techniques was reported to effectively predict or actively monitor events of interest by using variables in medical records datasets.

Research objectives

We attempted to construct a random forest (RF) model for the prediction of PIH events by using electronic information in a patient records dataset.

Research methods

Data were acquired from the electronic dataset of the Second Affiliated Hospital of Hainan Medical University. A RF model based on an up-to-date machine learning algorithm was used to predict post-anesthesia hypotension in patients during cardiac surgery.

Research results

Of the 3030 patients analyzed, 1578 (52.1%) experienced hypotensive events after anesthesia. The RF model had a high predictive performance, with an AUC of 0.843 (0.808-0.877). The most important variable attributing to the accuracy of hypotension prediction after anesthesia in the RF model was the mean blood pressure, followed by age and body mass index.

Research conclusions

RF technology can accurately predict PIH in patients following cardiac surgery.

Research perspectives

In the era of individualized medicine, precise machine learning modeling based on accessible patient information may offer anesthesiologists an opportunity for early intervention in PIH events.