Published online Nov 26, 2022. doi: 10.4330/wjc.v14.i11.565
Peer-review started: June 15, 2022
First decision: August 1, 2022
Revised: September 18, 2022
Accepted: October 18, 2022
Article in press: October 18, 2022
Published online: November 26, 2022
Processing time: 161 Days and 13 Hours
Risk stratification tools exist for patients presenting with chest pain to the emergency room and have achieved the recommended negative predictive value (NPV) of 99%. However, due to low positive predictive value (PPV), current stratification tools result in unwarranted investigations such as serial laboratory tests and cardiac stress tests (CSTs).
To create a machine learning model (MLM) for risk stratification of chest pain with a better PPV.
This retrospective cohort study used de-identified hospital data from January 2016 until November 2021. Inclusion criteria were patients aged > 21 years who presented to the ER, had at least two serum troponins measured, were subsequently admitted to the hospital, and had a CST within 4 d of presentation. Exclusion criteria were elevated troponin value (> 0.05 ng/mL) and missing values for body mass index. The primary outcome was abnormal CST. Demographics, coronary artery disease (CAD) history, hypertension, hyperlipidemia, diabetes mellitus, chronic kidney disease, obesity, and smoking were evaluated as potential risk factors for abnormal CST. Patients were also categorized into a high-risk group (CAD history or more than two risk factors) and a low-risk group (all other patients) for comparison. Bivariate analysis was performed using a χ2 test or Fisher’s exact test. Age was compared by t test. Binomial regression (BR), random forest, and XGBoost MLMs were used for prediction. Bootstrapping was used for the internal validation of prediction models. BR was also used for inference. Alpha criterion was set at 0.05 for all statistical tests. R software was used for statistical analysis.
The final cohort of the study included 2328 patients, of which 245 (10.52%) patients had abnormal CST. When adjusted for covariates in the BR model, male sex [risk ratio (RR) = 1.52, 95% confi
The XGBoost MLM achieved a PPV of 24.33% for an abnormal CST, which is better than current stratification tools (13.00%-17.50%). This highlights the beneficial potential of MLMs in clinical decision-making.
Core Tip: For patients with chest pain, current stratification tools result in unwarranted investigations due to low (13.0%-17.5%) positive predictive values (PPVs). This retrospective cohort study aimed to create a machine learning model (MLM) for risk stratification of patients with chest pain with a better PPV. Demographics, coronary artery disease history, hypertension, hyperlipidemia, diabetes mellitus, chronic kidney disease, obesity, and smoking were the covariates. The XGBoost MLM achieved a PPV of 24.33% for an abnormal cardiac stress test, which is better than current stratification tools. This model highlights the potential use of MLMs in clinical decision-making.
- Citation: Shafiq M, Mazzotti DR, Gibson C. Risk stratification of patients who present with chest pain and have normal troponins using a machine learning model. World J Cardiol 2022; 14(11): 565-575
- URL: https://www.wjgnet.com/1949-8462/full/v14/i11/565.htm
- DOI: https://dx.doi.org/10.4330/wjc.v14.i11.565
The annual cost of cardiovascular disease and stroke has been estimated to be more than $200 billion in the United States[1]. Chest pain, in particular, led to over 40 million hospital visits from 2006-2016 and had an associated cost of $6.2 billion from 2014-2016[2]. For each hospital admission, the average cost has been reported to be $11700 per patient in the United States[3]. This cost represents a significant burden on the patients and the healthcare system and emphasizes the need for a better stratification tool to safely identify patients who present with chest pain for early discharge without unnecessary testing.
Patients who present with chest pain are considered to be low risk if their probability of a major adverse cardiac event is < 1%, according to clinical practice guidelines of the American College of Cardiology and American Heart Association[4]. The two most widely used stratification tools that calculate the probability of major adverse cardiac event include the history, electrocardiogram (ECG), age, risk factors, and initial troponin pathway accelerated diagnostic protocol (HEART pathway-ADP) and the emergency department assessment of chest pain score (EDACS)-ADP[5-8]. The HEART pathway-ADP and EDACS-ADP have demonstrated excellent sensitivity and a negative predictive value (NPV) of 99%, as recommended by the American College of Cardiology and the American Heart Association[5-8]. However, both the HEART pathway-ADP and EDACS-ADP have low positive predictive values (PPVs) of 13.0% and 17.5%, respectively[7,9]. This low PPV leads to unnecessary additional testing, such as serial laboratory tests and cardiac stress tests (CSTs). Furthermore, the main focus of these two stratification tools has been the 30-d outcome of acute myocardial infarction (AMI) and all-cause mortality.
Many medical devices are already using artificial intelligence and machine learning algorithms[10]. However, these domains are underutilized in clinical decision-making despite their huge potential. The National COVID Cohort Collaborative facilitated the creation of machine learning models (MLMs) for accurate prediction of coronavirus disease 2019 severity with proven efficacy[11]. The same concept of machine learning can be applied to patients who present with chest pain in the emergency room (ER), and the use of patient data points (such as demographics and risk factors) can build predictive models with better specificity and PPV.
Stewart et al[12] reported in a recent systematic review of 23 studies that MLM outperformed traditional risk stratification scores for chest pain. Although the outcome varied among the included studies, data and the specifics of the MLMs, such as hyperparameters, were not shared, which complicates replication and validation[12]. Most recently, Doudesis et al[13] published the myocardial-ischemic-injury-index (MI3) algorithm with high PPV (70.4%) and NPV (99.8%) for myocardial infarction. They have not provided the specifics of their MLM either.
The HEART pathway-ADP, the EDACS-ADP, and other recently developed MLMs include both normal and abnormal troponin levels in their stratification[7,8,12,14,15]. However, risk stratification is more challenging when the troponin level is normal. In addition, the HEART pathway-ADP and the EDACS-ADP do not address the risk of non-obstructive coronary artery disease (CAD) or the risk of AMI over an extended period of time (over a year). Lastly, recently developed MLMs claim to have better stratification, but essential information about their models have not been shared, limiting the potential for reproducibility. Given these challenges to risk stratification among patients with chest pain and the need to improve patient outcomes and reduce unnecessary healthcare costs, we hypothesized that an MLM could be created to better predict abnormal CST among patients who present with chest pain and have normal troponin values. CST can identify wide spectrum of CAD, including non-obstructive CAD, and it provides risk assessment for 1 year rather than just 30 d. Due to the current enhanced computing capabilities, MLMs have the potential to achieve better PPV and lower false-positive rates, which can reduce unnecessary testing. MLMs can gain the reproducibility needed to build trust through data sharing and transparency, which can further improve risk stratification and deliver the most cost-effective healthcare.
This study aimed to create an MLM that can use patient characteristics to provide risk stratification for further clinical intervention for patients who present with chest pain and have normal troponin values. The hypothesis of this study was that patient characteristics can be used to create MLMs for risk stratification for patients who present to the ER with chest pain and have normal troponin values with a PPV of 25% or more and an NPV of at least 99%.
This study used a retrospective cohort design involving de-identified data available in the i2b2 common data model repository of the University of Kansas Medical Center. Database queries for patients observed in the health system between January of 2016 and November of 2021 were conducted using Healthcare Enterprise Repository for Ontological Narration, a search discovery tool that allows cohort building for observational research using de-identified data[16,17]. Institutional Review Board approval was not required because the data was de-identified. Identification and/or definitions of computable phenotypes used in this study are provided in the Supplementary material.
Inclusion criteria: All patients aged 21 years or older who presented to the ER, had their first troponin test carried out within the first 6 h of arrival to the ER and at least one troponin test completed after 6 h, were subsequently admitted to the hospital, and had a nuclear CST carried out within 4 d of presentation to the ER were included in the study.
Exclusion criteria: Patients with elevated troponin levels (> 0.05 ng/mL) and those with missing body mass index values were excluded. Patients with elevated troponin levels are considered high risk, and hospital admission with further clinical intervention is more appropriate for this class of patients. Without body mass index data, obesity could not be defined as one of the risk factors/covariates in this study.
CAD history, diabetes mellitus, hypertension, hyperlipidemia, chronic kidney disease, obesity, and smoking history were the risk factors included in this study. For comparison, patients included in the final cohort were also categorized into a high-risk group (CAD history or > 2 risk factors) and a low-risk group (no prior CAD history and ≤ 2 risk factors).
Only the first encounter was included in the study for patients with more than one ER encounter that met the above inclusion and exclusion criteria.
The outcome of the study was the incidence of abnormal CST. Abnormal CST was defined as CST followed by cardiac catheterization and/or coronary artery bypass graft within 30 d. Cardiac catheterization and coronary artery bypass graft were used as a surrogate to identify abnormal CST in this study.
Bivariate analysis: Except for age, the association between the incidence of abnormal CST and sex, race, and risk factors was assessed by χ2test or Fisher’s exact test. High-risk and low-risk groups were also compared using a χ2test, and the risk ratio was calculated. Age was compared by t test. Alpha criterion was set at 0.05.
Binomial regression: Binomial regression (BR) was used to adjust for confounding factors and infer the degree of association between the risk factors and the outcome. All BR assumptions were assessed, including assumptions of no multi-collinearity and no outliers. To ensure model adequacy, Hosmer-Lemeshow goodness-of-fit test was also performed. Alpha criterion both for BR as well as Hosmer-Lemeshow goodness-of-fit test was set at 0.05.
MLMs: BR was also used for prediction, and it used the same predictor variables as for inference. Besides BR, the random forest and XGBoost MLMs were also used. In the random forest MLM, proximity and importance were set as “True”. In order to minimize overfitting, the number of trees was set at 25 in training and testing. In the XGBoost MLM, hyperparameters were set as: Booster = “gbtree”; objective = “binary:logistic”; eval_metric = “auc”; eta = 0.1; max.depth = 10; gamma = 0; min_child_ weight = 1; and colsample_bytree = 1. In order to minimize overfitting, 25 rounds were used in the XGBoost MLM.
Bootstrapping with replacement was used for internal validation of all the above models. Data were randomly split into training (75%) and testing (25%) during each iteration of bootstrapping. The model with training data was first fitted, and then testing data were applied to assess internal validation during each iteration of bootstrapping. Finally, the results were averaged. In order to produce more precise estimates, 500 iterations were used in bootstrapping for all models, and 95% confidence intervals (CIs) were created. Prediction cutoff values were calibrated manually, and the value with the best metrics (sensitivity, specificity, PPV, and NPV) was then selected for each model. All statistical analyses were performed using R software (version 4.1.2).
The final cohort sample included 2328 unique patients, of which 245 (10.52%) patients had abnormal CST requiring cardiac catheterization and/or coronary artery bypass graft (Figure 1). There were 196 duplicate encounters, which were removed, and only the patient’s first encounter was included in the study. The basic demographic characteristics of the patients are presented in Table 1. Male sex and Caucasian race were significantly associated with the incidence of abnormal CST.
| Characteristics | Abnormal cardiac stress test requiring cardiac catheterization and/or CABG | Degree of association (95%CI) | P value | |
| Yes, n = 245 | No, n = 2083 | |||
| Age, mean ± SD | 63.02 ± 11.67 | 61.99 ± 12.46 | Mean different: 1.03 (-0.53, 2.59) | 0.200 |
| Sex male | 153 (62.4%) | 965 (46.3%) | RR: 1.8 (1.41, 2.30) | < 0.001 |
| Race | ||||
| Caucasian | 163 (66.5%) | 1182 (56.7%) | RR: 1.8 (1.08, 3.00) | 0.020 (combined) |
| African American | 65 (26.5%) | 650 (31.2%) | RR: 1.35 (0.79, 2.32) | |
| Asian | 2 (0.8%) | 43 (2.1%) | RR: 0.66 (0.16, 2.79) | |
| Other | 15 (6.1%) | 208 (10.0%) | Reference | |
In the bivariate analysis, obesity was not significantly associated with abnormal CST. Smoking history was significantly associated with abnormal CST, but the association was weak. As shown in Table 2, all other risk factors were significantly associated with abnormal CST. High-risk patients were found to have a risk ratio of 5.31 (95%CI: 2.75-10.24) for abnormal CST when compared to low-risk patients (Table 3).
| Risk factors | Abnormal cardiac stress test requiring cardiac catheterization and/or CABG | Risk ratio (95%CI) | P value | |
| Yes, n = 245 | No, n = 2083 | |||
| CAD history, yes | 216 (88.2%) | 1063 (51.0%) | 6.11 (4.18, 8.92) | < 0.001 |
| Obesity, yes | 136 (55.5%) | 1121 (53.8%) | 1.06 (0.84, 1.35) | 0.620 |
| Diabetes mellitus, yes | 112 (45.7%) | 760 (36.5%) | 1.41 (1.11, 1.78) | 0.005 |
| Hypertension, yes | 232 (94.7%) | 1783 (85.6%) | 2.77 (1.61, 4.78) | < 0.001 |
| Hyperlipidemia, yes | 236 (96.3%) | 1602 (76.9%) | 6.99 (3.62, 13.50) | < 0.001 |
| CKD history, yes | 88 (35.9%) | 540 (25.9%) | 1.52 (1.19, 1.94) | < 0.001 |
| Smoking history, yes | 153 (62.4%) | 1133 (54.4%) | 1.35 (1.05, 1.72) | 0.020 |
| Risk category | Abnormal cardiac stress test requiring cardiac catheterization and/or CABG | Risk ratio (95%CI) | P value | |
| Yes, n = 245 | No, n = 2083 | |||
| High risk | 236 (96.3%) | 1700 (81.61%) | 5.31 (2.75, 10.24) | < 0.001 |
| Low risk | 9 (3.7%) | 383 (18.39%) | ||
Age, race, and obesity were removed from the final BR model because they were not statistically significant, did not have any confounding relationship with other covariates, and their contribution to the prediction was not significant. These three covariates together resulted in an increased area under the receiver operating characteristic curve (AUC) by only 1.35%. The final model met all assumptions of BR, including the assumption of no multi-collinearity and no outliers, and appeared to fit the data adequately (Hosmer-Lemeshow goodness-of-fit test, P = 0.9). Only male sex, CAD history, and hyperlipidemia were statistically significant when adjusted for covariates in the final BR model. Covariates that were included in the final BR model and their estimated risk ratios are shown in Table 4.
| Covariate | Risk ratio | 95%CI | P value |
| Sex, male | 1.52 | 1.2, 1.94 | < 0.001 |
| CAD history, yes | 4.46 | 3.08, 6.72 | < 0.001 |
| Hypertension, yes | 1.35 | 0.82, 2.44 | 0.280 |
| Hyperlipidemia, yes | 3.87 | 2.12, 8.12 | < 0.001 |
| Diabetes mellitus, yes | 1.06 | 0.83, 1.34 | 0.650 |
| CKD history, yes | 1.02 | 0.80, 1.30 | 0.860 |
| Smoking history, yes | 1.06 | 0.84, 1.35 | 0.620 |
BR: The same covariates used for inference in the BR model were also used for prediction. These covariates included sex, CAD history, hypertension, hyperlipidemia, diabetes mellitus, chronic kidney disease, and smoking history. First, all data were used together, which yielded an AUC of 74.51% (Figure 2A). The internal validation of the BR model yielded similar results (Figure 2B). A prediction cutoff value of 0.2 provided a sensitivity of 45.06%, specificity of 80.46%, a PPV of 21.34%, and an NPV of 92.55%.
Random forest: In the random forest model, all covariates were included. This model showed a much better fit for all data combined (Figure 2C). During internal validation of the random forest model, the AUC dropped significantly (Figure 2D). A prediction cutoff value of 0.18 provided a sensitivity of 13.92%, specificity of 93.66%, a PPV of 20.55%, and an NPV of 90.24 % for the random forest model.
XGBoost: All covariates were used in the XGBoost model. It provided a better AUC for all data combined compared to BR (Figure 2E). Like the random forest model, the AUC dropped significantly during internal validation of the XGBoost model (Figure 2F). A prediction cutoff value of 0.27 was used for the XGBoost model, and it yielded a sensitivity of 30.54%, specificity of 88.51%, a PPV of 24.33%, and an NPV of 91.34%. A comparison of the three models is presented in Table 5.
| Feature | BR | RF | XGBoost |
| Prediction cutoff value | 0.20 | 0.18 | 0.27 |
| Sensitivity (95%CI) | 45.06 (44.23, 45.88) | 13.92 (13.50, 14.33) | 30.54 (29.30, 31.79) |
| Specificity (95%CI) | 80.46 (80.14, 80.79) | 93.66 (93.53, 93.80) | 88.51 (88.15, 88.86) |
| PPV (95%CI) | 21.34 (21.09, 21.60) | 20.55 (20.05, 21.04) | 24.33 (23.46, 25.20) |
| NPV (95%CI) | 92.55 (92.42, 92.69) | 90.24 (90.14, 90.35) | 91.34 (91.12, 91.56) |
This study found statistically significant associations between abnormal CST and several factors, including male sex, CAD history, and hyperlipidemia, among patients with chest pain who presented to the hospital, had normal troponin tests, and completed a CST. The incidence rate of abnormal CST among low-risk patients was only 2.30%, while it was 12.19% among high-risk patients. NPV for all MLMs in this study did not reach the recommended value of 99%. However, the XGBoost MLM in this study provided a much better PPV of 24.33%.
Both the HEART pathway-ADP and EDACS-ADP have excellent sensitivity and NPV (99%), but the PPV (13.0% and 17.5%, respectively) is considerably low. This likely leads to unnecessary clinical intervention[7,8]. Despite the retrospective nature and limited explanatory variables in modeling in this study, the XGBoost MLM resulted in an increased PPV (24.33%). Patients who had CAD with the need for revascularization and who did not follow up within 30 d could not be identified in the HEART pathway-ADP and EDACS-ADP studies. Likewise, further clinical intervention was not completed for low-risk patients, who were subsequently discharged in the HEART pathway-ADP and EDACS-ADP studies. Therefore, no objective data exists to suggest that those discharged patients did not have CAD (such as non-obstructive CAD). Nevertheless, in our study, nuclear CST was part of the inclusion criteria to ensure there was objective data on CAD for all patients. Therefore, the inclusion criteria were stricter for our study.
In the HEART pathway-ADP and EDACS-ADP, ECG and all troponin tests (normal and abnormal) were predictor variables[7,8]. If a patient has elevated troponin values, then the resulting clinical decision is to recommend further clinical intervention the majority of the time. However, it is challenging to identify patients who have normal troponin values and need further clinical intervention. This study undertook the challenge of including only patients with normal troponin values, which is another reason that the selection criteria for this study cohort was restricted. However, it mirrors real-life clinical practice and the challenges that come with it. ECG was not included in this study and is a significant limitation. Since it was a retrospective study based on de-identified data, access to actual ECG data was not possible. Machine readings of the ECG are available. However, due to the poor PPV of machine readings for abnormal findings on ECG, machine readings were not included[18,19]. Incorporating incorrect interpretations of ECG machine readings could have resulted in erroneous results and conclusions.
There has been increased interest in using MLMs and artificial intelligence in clinical decision-making in the past 8-10 years[12,14,15]. For patients who presented with chest pain to the ER, Stewart et al[12] conducted a systematic review of 23 studies that used MLMs to stratify these patients. There is significant heterogeneity in the studies included in this systematic review with differences noted in study design, type of MLM used, selected outcomes, comparisons made, data sharing, and validation. Despite the limitations, Stewart et al[12] reported that MLMs outperformed traditional risk stratification scores. Using high-sensitivity serum troponins, Doudesis et al[13] recently developed the MI3 using gradient boosting MLM with sex, age, serial serum troponin concentrations, and the time interval between the serum troponin sampling as their parameters. This model also included both normal and abnormal troponin assays. MI3 has reported a PPV of 70.4% for myocardial infarction if the MI3 score was > 49.6 and an NPV of 99.8% if the MI3 score was < 1.6. The specifics of their gradient boosting MLM have not been shared.
Zhang et al[14] conducted a study using MLMs for chest pain stratification, which shares some similarities in methodology to our study. The retrospective arm of their study was based on data from the electronic medical records. They excluded patients who had no follow-up data and did not include ECG for technical reasons (with no further explanation). The main differences in the study by Zhang et al[14] and our study included different outcomes (30-d AMI and all-cause mortality vs abnormal CST, respectively), different number of explanatory variables (14 vs 10, respectively), troponin values (all values vs only normal values, respectively), and the use of different MLMs.
Zhang et al[14] reported a robust sensitivity, specificity, and PPV of 92.90%, 88.50%, and 90.80%, respectively, for the 30-d risk of AMI during internal validation[14]. They observed similar sensitivity, specificity, and PPV for 30-d risk of all-cause mortality (77.50%, 99.99%, and 90.80%, respectively)[14]. This represents one of the best models for risk stratification of patients who present to the ER with chest pain. In comparison, our study had a small sample size, used fewer explanatory variables, and included only normal troponins values, which likely led to lower predictive performance. Despite the robust performance of their MLM, Zhang et al[14] have not shared the MLM specifications in order to reproduce and externally validate their model. The heterogeneity of MLMs, lack of data sharing, and current lack of external validation are the major obstacles to widespread adoption of MLMs for risk stratification for patients presenting with chest pain.
Digitization of health records and exceptional computing power have enabled the use of artificial intelligence and MLMs in medicine. Taking advantage of these resources, our study created an MLM and is the first study to our knowledge that used an MLM for risk stratification of abnormal CST. Since abnormal CST can be used as a surrogate for the entire spectrum of CAD, this study promotes the idea that MLMs can be used to risk stratify for all CAD spectrums.
There are limitations to this study. First, ECG was not included in the study. ECG can improve both the PPV and NPV. Second, given the clear definitions of the computable phenotypes, the probability of missing a true risk factor or outcome is very low, but it is not zero. Third, we included only ten explanatory variables in this study, which is likely one of the reasons for lower NPV. Similar MLM studies with better predictive performance have incorporated 14 or more explanatory variables. Lastly, this study was a retrospective single-center study. Future studies replicating this study will strengthen the external validity of the current findings.
The current study achieved a better PPV for the entire spectrum of CAD (not just AMI) compared to the currently used risk stratification tools. These results highlight the potential of using MLMs in clinical decision-making. The results of this study also advanced the idea that a well-designed prospective study, which incorporates ECG and ensures proper follow-up, can achieve a much better PPV than currently used stratification tools (i.e., the HEART pathway-ADP and the EDACS-ADP) while simultaneously maintaining an NPV of 99% for chest pain presentation to the ER. Data sharing and external validation of these prospective trials will be crucial to the recognition and adoption of MLMs.
Risk stratification tools exist for patients presenting with chest pain to the emergency room and have achieved the recommended negative predictive value (NPV) of 99%. However, the current stratification tools result in unnecessary clinical interventions due to a low positive predictive value (PPV).
Healthcare costs are astronomical in the United States, including for patients who present with chest pain. These costs emphasize the need for a better stratification tool to safely identify patients who present with chest pain for early discharge without unnecessary testing.
This study aimed to create a machine learning model (MLM) for risk stratification of chest pain patients with a better PPV while maintaining an NPV of 99%.
This retrospective cohort study used demographics, coronary artery disease history, hypertension, hyperlipidemia, diabetes mellitus, chronic kidney disease, obesity, and smoking as the covariates for the prediction of an abnormal cardiac stress test (CST). Binomial regression (BR), random forest, and XGBoost MLMs were used for prediction. Bootstrapping was used for the internal validation of the prediction models.
The XGBoost MLM had the best PPV of 24.33%, with an NPV of 91.34% for abnormal CST. The BR MLM had a PPV of 21.34% and an NPV of 92.55%. The random forest MLM had a PPV of 20.55% and an NPV of 90.24%.
The XGBoost MLM provided a better PPV than currently used stratification tools (24.33% vs 13.00%-17.50%). Though the NPV from the XGBoost MLM remained lower than the recommended value of 99%, it highlights the potential use of MLMs in clinical decision-making.
Data sharing and external validation of the MLMs will be crucial for their recognition and widespread adoption.
The University of Kansas Medical Center has an established i2b2-based Healthcare Enterprise Repository for Ontological Narration. The Healthcare Enterprise Repository for Ontological Narration is supported by the Clinical and Translational Science Award from the National Center for Advancing Translational Sciences, which has been awarded to the University of Kansas Clinical and Translational Science Institute. The authors are grateful to the Department of Clinical Informatics at the University of Kansas Medical Center for their help in providing de-identified data using the Healthcare Enterprise Repository for Ontological Narration.
Provenance and peer review: Unsolicited article; Externally peer reviewed.
Peer-review model: Single blind
Specialty type: Cardiac and cardiovascular systems
Country/Territory of origin: United States
Peer-review report’s scientific quality classification
Grade A (Excellent): 0
Grade B (Very good): B
Grade C (Good): C, C
Grade D (Fair): 0
Grade E (Poor): 0
P-Reviewer: Byeon H, South Korea; Muneer A, Malaysia S-Editor: Wang JJ L-Editor: A P-Editor: Wang JJ
| 1. | Roger VL, Go AS, Lloyd-Jones DM, Adams RJ, Berry JD, Brown TM, Carnethon MR, Dai S, de Simone G, Ford ES, Fox CS, Fullerton HJ, Gillespie C, Greenlund KJ, Hailpern SM, Heit JA, Ho PM, Howard VJ, Kissela BM, Kittner SJ, Lackland DT, Lichtman JH, Lisabeth LD, Makuc DM, Marcus GM, Marelli A, Matchar DB, McDermott MM, Meigs JB, Moy CS, Mozaffarian D, Mussolino ME, Nichol G, Paynter NP, Rosamond WD, Sorlie PD, Stafford RS, Turan TN, Turner MB, Wong ND, Wylie-Rosett J; American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Heart disease and stroke statistics--2011 update: a report from the American Heart Association. Circulation. 2011;123:e18-e209. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 3692] [Cited by in F6Publishing: 3680] [Article Influence: 283.1] [Reference Citation Analysis (0)] |
| 2. | Aalam AA, Alsabban A, Pines JM. National trends in chest pain visits in US emergency departments (2006-2016). Emerg Med J. 2020;37:696-699. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 9] [Cited by in F6Publishing: 9] [Article Influence: 2.3] [Reference Citation Analysis (0)] |
| 3. | Liang L, Moore B, Soni A. National Inpatient Hospital Costs: The Most Expensive Conditions by Payer, 2017. 2020 Jul 14. In: Healthcare Cost and Utilization Project (HCUP) Statistical Briefs [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2006 Feb-. [PubMed] [Cited in This Article: ] |
| 4. | Writing Committee Members; Gulati M, Levy PD, Mukherjee D, Amsterdam E, Bhatt DL, Birtcher KK, Blankstein R, Boyd J, Bullock-Palmer RP, Conejo T, Diercks DB, Gentile F, Greenwood JP, Hess EP, Hollenberg SM, Jaber WA, Jneid H, Joglar JA, Morrow DA, O'Connor RE, Ross MA, Shaw LJ. 2021 AHA/ACC/ASE/CHEST/SAEM/SCCT/SCMR Guideline for the Evaluation and Diagnosis of Chest Pain: Executive Summary: A Report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice Guidelines. J Am Coll Cardiol. 2021;78:2218-2261. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 13] [Cited by in F6Publishing: 66] [Article Influence: 22.0] [Reference Citation Analysis (0)] |
| 5. | Boyle RSJ, Body R. The Diagnostic Accuracy of the Emergency Department Assessment of Chest Pain (EDACS) Score: A Systematic Review and Meta-analysis. Ann Emerg Med. 2021;77:433-441. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 7] [Cited by in F6Publishing: 7] [Article Influence: 2.3] [Reference Citation Analysis (0)] |
| 6. | Frisoli TM, Nowak R, Evans KL, Harrison M, Alani M, Varghese S, Rahman M, Noll S, Flannery KR, Michaels A, Tabaku M, Jacobsen G, McCord J. Henry Ford HEART Score Randomized Trial: Rapid Discharge of Patients Evaluated for Possible Myocardial Infarction. Circ Cardiovasc Qual Outcomes. 2017;10. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 19] [Cited by in F6Publishing: 20] [Article Influence: 2.9] [Reference Citation Analysis (0)] |
| 7. | Mahler SA, Lenoir KM, Wells BJ, Burke GL, Duncan PW, Case LD, Herrington DM, Diaz-Garelli JF, Futrell WM, Hiestand BC, Miller CD. Safely Identifying Emergency Department Patients With Acute Chest Pain for Early Discharge. Circulation. 2018;138:2456-2468. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 74] [Cited by in F6Publishing: 116] [Article Influence: 23.2] [Reference Citation Analysis (0)] |
| 8. | Than M, Flaws D, Sanders S, Doust J, Glasziou P, Kline J, Aldous S, Troughton R, Reid C, Parsonage WA, Frampton C, Greenslade JH, Deely JM, Hess E, Sadiq AB, Singleton R, Shopland R, Vercoe L, Woolhouse-Williams M, Ardagh M, Bossuyt P, Bannister L, Cullen L. Development and validation of the Emergency Department Assessment of Chest pain Score and 2 h accelerated diagnostic protocol. Emerg Med Australas. 2014;26:34-44. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 131] [Cited by in F6Publishing: 145] [Article Influence: 14.5] [Reference Citation Analysis (0)] |
| 9. | Flaws D, Than M, Scheuermeyer FX, Christenson J, Boychuk B, Greenslade JH, Aldous S, Hammett CJ, Parsonage WA, Deely JM, Pickering JW, Cullen L. External validation of the emergency department assessment of chest pain score accelerated diagnostic pathway (EDACS-ADP). Emerg Med J. 2016;33:618-625. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 36] [Cited by in F6Publishing: 36] [Article Influence: 4.5] [Reference Citation Analysis (0)] |
| 10. | Benjamens S, Dhunnoo P, Meskó B. The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database. NPJ Digit Med. 2020;3:118. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 418] [Cited by in F6Publishing: 419] [Article Influence: 104.8] [Reference Citation Analysis (0)] |
| 11. | Bennett TD, Moffitt RA, Hajagos JG, Amor B, Anand A, Bissell MM, Bradwell KR, Bremer C, Byrd JB, Denham A, DeWitt PE, Gabriel D, Garibaldi BT, Girvin AT, Guinney J, Hill EL, Hong SS, Jimenez H, Kavuluru R, Kostka K, Lehmann HP, Levitt E, Mallipattu SK, Manna A, McMurry JA, Morris M, Muschelli J, Neumann AJ, Palchuk MB, Pfaff ER, Qian Z, Qureshi N, Russell S, Spratt H, Walden A, Williams AE, Wooldridge JT, Yoo YJ, Zhang XT, Zhu RL, Austin CP, Saltz JH, Gersing KR, Haendel MA, Chute CG; National COVID Cohort Collaborative (N3C) Consortium. Clinical Characterization and Prediction of Clinical Severity of SARS-CoV-2 Infection Among US Adults Using Data From the US National COVID Cohort Collaborative. JAMA Netw Open. 2021;4:e2116901. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 154] [Cited by in F6Publishing: 162] [Article Influence: 54.0] [Reference Citation Analysis (0)] |
| 12. | Stewart J, Lu J, Goudie A, Bennamoun M, Sprivulis P, Sanfillipo F, Dwivedi G. Applications of machine learning to undifferentiated chest pain in the emergency department: A systematic review. PLoS One. 2021;16:e0252612. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 28] [Cited by in F6Publishing: 13] [Article Influence: 4.3] [Reference Citation Analysis (1)] |
| 13. | Doudesis D, Lee KK, Yang J, Wereski R, Shah ASV, Tsanas A, Anand A, Pickering JW, Than MP, Mills NL; High-STEACS Investigators. Validation of the myocardial-ischaemic-injury-index machine learning algorithm to guide the diagnosis of myocardial infarction in a heterogenous population: a prespecified exploratory analysis. Lancet Digit Health. 2022;4:e300-e308. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 21] [Cited by in F6Publishing: 19] [Article Influence: 9.5] [Reference Citation Analysis (0)] |
| 14. | Zhang PI, Hsu CC, Kao Y, Chen CJ, Kuo YW, Hsu SL, Liu TL, Lin HJ, Wang JJ, Liu CF, Huang CC. Real-time AI prediction for major adverse cardiac events in emergency department patients with chest pain. Scand J Trauma Resusc Emerg Med. 2020;28:93. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 16] [Cited by in F6Publishing: 29] [Article Influence: 7.3] [Reference Citation Analysis (0)] |
| 15. | Than MP, Pickering JW, Sandoval Y, Shah ASV, Tsanas A, Apple FS, Blankenberg S, Cullen L, Mueller C, Neumann JT, Twerenbold R, Westermann D, Beshiri A, Mills NL; MI3 collaborative. Machine Learning to Predict the Likelihood of Acute Myocardial Infarction. Circulation. 2019;. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 108] [Cited by in F6Publishing: 113] [Article Influence: 22.6] [Reference Citation Analysis (0)] |
| 16. | Murphy SN, Weber G, Mendis M, Gainer V, Chueh HC, Churchill S, Kohane I. Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2). J Am Med Inform Assoc. 2010;17:124-130. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 554] [Cited by in F6Publishing: 619] [Article Influence: 44.2] [Reference Citation Analysis (0)] |
| 17. | Waitman LR, Warren JJ, Manos EL, Connolly DW. Expressing observations from electronic medical record flowsheets in an i2b2 based clinical data repository to support research and quality improvement. AMIA Annu Symp Proc. 2011;2011:1454-1463. [PubMed] [Cited in This Article: ] |
| 18. | Park JH, Moon SW, Kim TY, Ro YS, Cha WC, Kim YJ, Shin SD. Sensitivity, specificity, and predictive value of cardiac symptoms assessed by emergency medical services providers in the diagnosis of acute myocardial infarction: a multi-center observational study. Clin Exp Emerg Med. 2018;5:264-271. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 1] [Cited by in F6Publishing: 1] [Article Influence: 0.2] [Reference Citation Analysis (0)] |
| 19. | Schläpfer J, Wellens HJ. Computer-Interpreted Electrocardiograms: Benefits and Limitations. J Am Coll Cardiol. 2017;70:1183-1192. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 149] [Cited by in F6Publishing: 158] [Article Influence: 22.6] [Reference Citation Analysis (0)] |