Validation of a pediatric bedside tool to predict time to death after withdrawal of life support

Abstract

AIM: To evaluate the accuracy of a tool developed to predict timing of death following withdrawal of life support in children.

METHODS: Pertinent variables for all pediatric deaths (age ≤ 21 years) from 1/2009 to 6/2014 in our pediatric intensive care unit (PICU) were extracted through a detailed review of the medical records. As originally described, a recently developed tool that predicts timing of death in children following withdrawal of life support (dallas predictor tool [DPT]) was used to calculate individual scores for each patient. Individual scores were calculated for prediction of death within 30 min (DPT30) and within 60 min (DPT60). For various resulting DPT30 and DPT60 scores, sensitivity, specificity and area under the receiver operating characteristic curve were calculated.

RESULTS: There were 8829 PICU admissions resulting in 132 (1.5%) deaths. Death followed withdrawal of life support in 70 patients (53%). After excluding subjects with insufficient data to calculate DPT scores, 62 subjects were analyzed. Average age of patients was 5.3 years (SD: 6.9), median time to death after withdrawal of life support was 25 min (range; 7 min to 16 h 54 min). Respiratory failure, shock and sepsis were the most common diagnoses. Thirty-seven patients (59.6%) died within 30 min of withdrawal of life support and 52 (83.8%) died within 60 min. DPT30 scores ranged from -17 to 16. A DPT30 score ≥ -3 was most predictive of death within that time period, with sensitivity = 0.76, specificity = 0.52, AUC = 0.69 and an overall classification accuracy = 66.1%. DPT60 scores ranged from -21 to 28. A DPT60 score ≥ -9 was most predictive of death within that time period, with sensitivity = 0.75, specificity = 0.80, AUC = 0.85 and an overall classification accuracy = 75.8%.

CONCLUSION: In this external cohort, the DPT is clinically relevant in predicting time from withdrawal of life support to death. In our patients, the DPT is more useful in predicting death within 60 min of withdrawal of life support than within 30 min. Furthermore, our analysis suggests optimal cut-off scores. Additional calibration and modifications of this important tool could help guide the intensive care team and families considering DCD.

Key Words: Death; Organ donation; Children; Donation after circulatory death

Core tip: Donation after circulatory death (DCD) has gained acceptance as a way of increasing the number of organs available for transplantation. In order for DCD to occur, organs must be harvested within 30 or 60 min of withdrawal of support. A tool that predicts time of death after withdrawal of support in children has been created but not validated by an external source. In this study, we apply the newly created Dallas Predictor Tool to an external pediatric sample and show it to be an accurate predictor of death within 60 min of withdrawal of support. The tool would require additional calibration to be a good predictor of death within 30 min.

INTRODUCTION

In the United States, the number of patients awaiting organ transplantation far exceeds the number of organs available from living or brain dead donors[1,2]. In August 2015, there were 133661 patients waiting for organ transplantation in the United States, yet there were only 29532 transplants performed during the previous year with organs from 14413 donors, highlighting the marked disparity between need and supply[3]. That disparity also exists in the pediatric age range, considering that there are currently 2036 patients younger than 18 years of age awaiting transplants, despite 1795 transplants performed in 2014[3].

In donation after circulatory death (DCD), organs are recovered from a donor that dies during controlled withdrawal of life support[4-6]. Typically, in the United States, a potential DCD donor is a terminally ill patient with a clear advance directive or a surrogate decision maker who, in conjunction with the medical team, believes that the best course of action is withdrawal of life support. Should there be agreement on the opportunity for DCD and proper consent, life support is withdrawn under controlled conditions and organs (usually kidneys and liver) can be harvested upon declaration of death by cardiopulmonary criteria following a pre-determined time interval. Although DCD has become more frequent in the past decade, still the majority of transplanted organs are recovered after donation following neurological death (DND)[3]. Various studies have shown that outcomes following DCD transplants are similar to those following DND transplants[6-8]. Therefore, increasing utilization of DCD is one mechanism to increase the availability of organs for patients on the transplant wait list and decrease waiting time[9].

Organ viability from DCD donors is predicated on a minimal interval between withdrawal of support and organ removal. If excessive time elapses between withdrawal of support and circulatory death, the donor will become ineligible. Although no evidence-based consensus on what constitutes “excessive time” exists, an organ typically is no longer considered transplantable if time from withdrawal to death is greater than 30 min for a liver and 60 min for kidneys[10]. The uncertainty of suitability of organs relative to the time of death in addition to usual end-of-life considerations may lead to undue stress on the donor’s family, potential transplant recipients and medical teams. Therefore, improved ability to predict the amount of time from withdrawal of support to circulatory death could enhance the DCD process and facilitate increased donation rates.

A tool that predicts the likelihood of death within the organ recovery window has been developed and used in adult patients for several years[11]. The Wisconsin DCD Evaluation Tool predicted suitability for DCD in adults 83.7% of the time within 60 min after withdrawal of support[11]. More recently, a pediatric tool was developed through analysis of 518 deaths at Children’s Medical center dallas, referred here henceforth as the dallas predictor tool (DPT)[1]. The DPT was created using data from a single institution and external validation has not been reported. Validation of this tool could help physicians determine a priori whether a pediatric patient might be eligible for organ donation following withdrawal of life support and help inform families considering this type of organ donation.

The objective of this study is to characterize the process of death following withdrawal of support in the pediatric intensive care unit (PICU) of an academic children’s hospital and evaluate the performance of the DPT in predicting time to death after withdrawal of support in this remote pediatric sample.

MATERIALS AND METHODS

After obtaining Institutional Review Board approval, we performed a detailed retrospective chart analysis of all deaths that occurred after withdrawal of support in the PICU at UH Rainbow Babies and Children’s Hospital from January 1^st 2009 to June 30^th 2014. The inclusion criteria for this study were all patients 21 years of age or younger admitted to PICU who died after withdrawal of life support. Patients were excluded if they were older than 21 years of age, died during active resuscitation or were declared brain dead.

Patients were identified using our own PICU database. Medical records for these patients were reviewed to evaluate whether they met the inclusion criteria. Data from patients meeting the inclusion criteria were abstracted into a protected spreadsheet for subsequent analysis. Extracted data included demographic information, time of admission, time of withdrawal of support, time of death, diagnoses, co-morbidities, vital signs, support modalities (e.g., mechanical ventilation, renal replacement therapies, extracorporeal life support), vasoactive and inotropic support, results from laboratory testing related to renal and hepatic function, infectious status and mechanism of death. The dataset was then used to externally validate the accuracy of the existing pediatric DCD tool (DPT)[1] for this remote sample.

We utilized the DPT to calculate scores for likelihood of death within 30 min (DPT30) and 60 min (DPT60) for each patient with clinical data obtained just prior to withdrawal of life support using the criteria specified in Table 1. The score was then used to predict the likelihood of a patient dying within 30 min or 60 min from withdrawal of life support. The predictive accuracy of the DPT tool was calculated by correlating the scores and the actual time interval between withdrawal of life support and death.

Table 1 Dallas predictor tool score calculation[1].

	30 min predictor (DPT30)	60 min predictor (DPT60)
Model parameter	Point score	Point score
Age 1 mo or younger	-9	-9
Norepi, epi or phenyl > 0.2 mcg/kg per minute	11	10
ECMO	11	17
PEEP > 10 cmH2O	5	11
Spontaneous ventilation	-8	-12
Lowest possible score	-17	-21
Highest possible score	27	38

DPT: Dallas predictor tool; Norepi: Norepinehrine; Epi: Epinephrine; Phenyl: Phenylephrine; ECMO: Extracorporeal membrane oxygenation; PEEP: Positive end-expiratory pressure.

Data were treated with descriptive statistics using dedicated software (SigmaPlot, Systat Software Inc, San Jose, CA). Receiver operating characteristic (ROC) curves were created using a dedicated web-based ROC analysis calculator[11]. ROC data were used to determine the optimal cut point in the range of DPT scores for best sensitivity and specificity. This optimal cut point was then used to determine the overall classification accuracy, which we defined as the added percentage of patients correctly predicted to be dead or alive at 30 min and 60 min by using the optimal score cut point.

RESULTS

During the 66-mo study period, there were 8829 admissions to the PICU resulting in 132 deaths and a mortality rate of 1.5% (Figure 1). Death followed withdrawal of life support in 70 patients (53%). Of those, 8 patients were excluded from the data analysis for not having sufficient data for retrospective calculation of the DPT scores. Therefore, 62 patients who died following withdrawal of life support were included in the data analysis. Among the remaining 62 patients for whom support was not withdrawn, 37 deaths (28%) occurred during attempts to resuscitate (failed CPR), 16 (12%) patients met brain death criteria, and 9 (7%) deaths occurred in patients with a “do not attempt resuscitation” (DNAR) order but without active withdrawal of life support.

Figure 1 Pediatric intensive care unit admissions and deaths. PICU: Pediatric intensive care unit; CPR: Cardiopulmonary resuscitation; DNAR: Do not attempt resuscitation; WS: Withdrawal of support.

The mean age of patients analyzed in our sample was 5.3 years (SD: 6.9 years). The median time to death after withdrawal of life support was 25 min (range: 7 min to 16 h 54 min). Thirty-seven patients (59.6%) died within 30 min of withdrawal of life support and 52 (83.8%) died within 60 min. Common diagnoses included respiratory failure (32.2%), hypoxic-ischemic encephalopathy (19.3%), cardiorespiratory arrest (16.1%), congenital heart disease (16.1%) and shock (14.5%).

Death within 30 min after withdrawal of life support (DPT30) scores ranged from -17 to 16 (Table 2). A DPT30 score ≥ -3 was most predictive of death, with sensitivity of 0.76, specificity of 0.52, area under curve (AUC) of 0.69 and an overall classification accuracy of 66.1% (Table 2 and Figure 2). Death within 60 min after withdrawal of life support (DPT60) scores ranged from -21 to 28 (Table 3). A DPT60 score ≥ -9 was most predictive of death, with sensitivity of 0.75, specificity of 0.8, AUC of 0.847 and an overall classification accuracy of 75.8% (Table 3 and Figure 3). Organs were actually donated after circulatory arrest following withdrawal of life support (DCD) from 2 patients in our sample. The interval time to death after withdrawal of life support in those patients was 35 min and 38 min, with liver and kidneys harvested in those procedures. There was also 1 case of attempted but unsuccessful DCD, where parents consented to donation but the child died after the 60-min time limit following withdrawal of life support.

Table 2 Scores for dallas predictor tool 30 min.

Cutpoint	Dead (1)	Alive (0)	Total	Sensitivity	Specificity
-17	1	1	2	1.0000	0.00
-9	1	0	1	0.9730	0.04
-8	7	12	19	0.9459	0.04
-3	1	0	1	0.7568	0.52
0	10	7	17	0.7297	0.52
2	2	1	3	0.4595	0.8
3	1	0	1	0.4054	0.84
5	0	2	2	0.3784	0.84
7	1	0	1	0.3784	0.92
8	0	1	1	0.3514	0.92
11	7	1	8	0.3514	0.96
16	6	0	6	0.1622	1.00
Total	37	25	62

Figure 2 Receiver operating characteristic for prediction of death within 30 min of withdrawal of life support (DPT30). Area under the curve (AUC) = 0.69.

Table 3 Scores for dallas predictor tool 60 min.

Cutpoint	Dead (1)	Alive (0)	Total	Sensitivity	Specificity
-21	1	1	2	1.0000	0.0
-12	12	7	19	0.9808	0.1
-9	1	0	1	0.7500	0.8
-1	1	0	1	0.7308	0.8
0	15	2	17	0.7115	0.8
1	1	0	1	0.4231	1.0
5	1	0	1	0.4038	1.0
8	2	0	2	0.3046	1.0
10	7	0	7	0.3462	1.0
11	2	0	2	0.2115	1.0
16	1	0	1	0.1731	1.0
17	1	0	1	0.1538	1.0
19	1	0	1	0.1346	1.0
21	3	0	3	0.1154	1.0
28	3	0	3	0.0577	1.0
Total	52	10	62

Figure 3 Receiver operating characteristic for prediction of death within 60 min of withdrawal of life support (DPT60). Area under the curve (AUC) = 0.87.

DISCUSSION

Data from the Organ Procurement and Transplant Network show that the number of patients awaiting an organ transplants continues to grow every year[2]. A national survey of donor hospitals identified 1330 eligible pediatric organ donors with consent rates of nearly 69%[12]. However, within that group there were only 37 pediatric DCD donors yielding 103 transplanted organs[12]. Another study evaluating the potential for DCD at a children’s hospital showed that 5.5% of all patients who died in a PICU would have been potential candidates for organ donation through DCD[13]. However, that figure is higher (58%) when considering only those patients not receiving CPR or without a contraindication for donation, such as target organ dysfunction[13]. A more recent study involving children in the neonatal, cardiovascular and pediatric intensive care units found that the number of pediatric potential candidates for DCD was significantly larger than the number of potential candidates for donation after neurologic determination of death, but that the actual donation rate was significantly lower[9]. With external validation of the DPT and its increased use, it might be possible for pediatric intensivists to identify with a greater degree of certainty which patients might be eligible for DCD following withdrawal of life support and counsel the family accordingly. This tool may also help minimize the stress, frustration, and inefficient use of resources associated with donation failure by enrolling patients found to be highly likely to die within 60 min following withdrawal of life support.

The criteria used to calculate the DPT score are simple and intuitive. It is reasonable to expect that critically-ill patients who are incapable of producing spontaneous respirations, those who require significant mechanical ventilator support, high doses of vasoactive or inotropic drugs, or extracorporeal life support would have a shorter interval between withdrawal of life support and circulatory death. The original development and application of the DPT has an overall classification accuracy of 74.5% and 87.3% for death within 30 and 60 min after withdrawal of life support[1]. However, those figures were obtained by applying the DPT to the very sample used to develop it. While the DPT score has shown promise in that initial publication, it had not been validated through a remote sample until the current study.

Our data suggest that the DPT is clinically relevant in predicting time from withdrawal of life support to death. We note that the DPT60 score has higher classification accuracy than the DPT30 score and a more robust AUC. In general, the classification accuracy in our data was lower than that noted in the original Dallas study[1]. The overall classification accuracy for DPT30 at our institution was 66.1% while the accuracy in the Dallas study was 74.5%[1]. Similarly, the accuracy for DPT60 at our institution was 75.8% compared to the accuracy of 87.3% noted in the Dallas study[1]. Despite these differences, we believe that the DPT60 score can be used as an accurate predictor of death within 60 min following withdrawal of life support. Our analysis also suggests that optimal cutoff scores for best accuracy vary between samples. The optimal cutoff scores of -3 and -9 for DPT30 and DPT60 for our study, respectively, differed considerably from the original Dallas study[1], underscoring the need for additional calibration and modifications of the tool so as to arrive at more widely applicable cutoff points.

In our study, there were only 3 attempts at DCD and only 2 successful donations, highlighting the fact that this form of organ donation is still the exception among pediatric patients. However, should each one of the 62 eligible patients have consented for DCD prior to withdrawal of support, 37 patients would have been eligible to donate a liver (death within 30 min) and 52 patients would have been eligible to donate kidneys (death within 60 min).

Our study is limited by factors inherent to its retrospective nature, specifically the accuracy of documentation of end-of-life events for these patients. However, clinical data and times of withdrawal of life support and death are extensively and redundantly documented at our institution, so the likelihood of this type of error is minimal. Nevertheless, a prospective study would be required to completely validate these data and test the real time prospective applicability of this tool. The sample size in our study was considerably smaller than in the original Dallas study (62 patients vs 518 patients, respectively). This relatively small sample could lead to sampling error and potentially impact the accuracy of the DPT score in an external cohort. However, if the DPT score is accurate it should be predictive in any cohort irrespective of the diagnostic profile and associated comorbidities of the external cohort.

In conclusion, A simple, convenient and accurate tool that predicts time to death after withdrawal of life support in children, such as the DPT would be an important adjunct to the decision-making process regarding DCD. In this external cohort, the DPT is clinically relevant in predicting time from withdrawal of life support to death. Our data show that the DPT is more useful in predicting death within 60 min than within 30 min of withdrawal of life support. The predictive accuracy of the DPT30 score is not as high and may require recalibration or incorporation of additional variables to become more clinically useful.

COMMENTS

Background

The number of patients awaiting organ transplantation far exceeds the number of organs available from living or brain dead donors in the United States. Donation after circulatory death (DCD), a form of donation where organs are recovered from a donor that dies following controlled withdrawal of life support, has been seen as an alternative to increase availability of organs and decrease waiting time. The ability to accurately predict the time interval between withdrawal of support and death is important in the DCD process because it can help inform the medical decision-makers (patient and family) and the medical team on the likelihood of death within the acceptable donation time window. This can help decrease the emotional stress associated with unsuccessful DCD on the donor family, potential organ recipient and medical teams. A pediatric tool that predicts the likelihood of death within the DCD time window has recently been develop but not yet validated externally. In this study, the authors apply this newly developed tool to a remote pediatric sample to evaluate its sensitivity, specificity and overall classification performance.

Research frontiers

Accurately predicting the time elapsed between withdrawal of support and death could have major implications in the process of DCD. This study is the first attempt to externally validate the dallas predictor tool (DPT); its results will help guide the application of this tool to remote samples.

Innovations and breakthroughs

In this study, the DPT accurately predicted death within 30 min of withdrawal of support in 66.1% of subjects, and death within 60 min of withdrawal of support in 75.8% of subjects. The authors have shown that the DPT accuracy is lower when applied to an external sample. The DPT may require recalibration or incorporation of additional variables to become more clinically useful, particularly for the 30 min time window.

Applications

This study suggests that the DPT can predict death within 60 min in over 75% of patients and can be used to inform the suitability of a potential pediatric donor being considered for DCD.

Terminology

DCD: Organ donation after circulatory death is a process by which organs are recovered from a donor that dies during controlled withdrawal of life support. DPT: The dallas predictor tool is method that predicts the likelihood of death within 30 or 60 min of withdrawal of life support in children.

Peer-review

The manuscript is well written and covers a gap in knowledge on this topic. In this manuscript, DPT is clinically relevant in predicting time from withdrawal of life support to death. Precisely, DPT is more useful in predicting death within 60 min of withdrawal of life support than within 30 min. Additional calibration and modifications of this important tool could help guide the intensive care team and families considering DCD.