Predictive value of the unplanned extubation risk assessment scale in hospitalized patients with tubes

Abstract

BACKGROUND

Critical patients often had various types of tubes, unplanned extubation of any kind of tube may cause serious injury to the patient, but previous reports mainly focused on endotracheal intubation. The limitations or incorrect use of the unplanned extubation risk assessment tool may lead to improper identification of patients at a high risk of unplanned extubation and cause delay or non-implementation of unplanned extubation prevention interventions. To effectively identify and manage the risk of unplanned extubation, a comprehensive and universal unplanned extubation risk assessment tool is needed.

AIM

To assess the predictive value of the Huaxi Unplanned Extubation Risk Assessment Scale in inpatients.

METHODS

This was a retrospective validation study. In this study, medical records were extracted between October 2020 and September 2021 from a tertiary comprehensive hospital in southwest China. For patients with tubes during hospitalization, the following information was extracted from the hospital information system: age, sex, admission mode, education, marital status, number of tubes, discharge mode, unplanned extubation occurrence, and the Huaxi Unplanned Extubation Risk Assessment Scale (HUERAS) score. Only inpatients were included, and those with indwelling needles were excluded. The best cut-off value and the area under the curve (AUC) of the Huaxi Unplanned Extubation Risk Assessment Scale were been identified.

RESULTS

A total of 76033 inpatients with indwelling tubes were included in this study, and 26 unplanned extubations occurred. The patients’ HUERAS scores were between 11 and 30, with an average score of 17.25 ± 3.73. The scores of patients with or without unplanned extubation were 22.85 ± 3.28 and 17.25 ± 3.73, respectively (P < 0.001). The results of the correlation analysis showed that the correlation coefficients between each characteristic and the total score ranged from 0.183 to 0.843. The best cut-off value was 21, and there were 14135 patients with a high risk of unplanned extubation, accounting for 18.59%. The Cronbach’s α, sensitivity, specificity, positive predictive value, and negative predictive value of the Huaxi Unplanned Extubation Risk Assessment Scale were 0.815, 84.62%, 81.43%, 0.16%, and 99.99%, respectively. The AUC of HUERAS was 0.851 (95%CI: 0.783-0.919, P < 0.001).

CONCLUSION

The HUERAS has good reliability and predictive validity. It can effectively identify inpatients at a high risk of unplanned extubation and help clinical nurses carry out risk screening and management.

Key Words: Inpatient; Unplanned extubation; Risk assessment; Prediction; Tube management

Core Tip: This was a retrospective validation study, in which 76033 inpatients with indwelling tubes were included. This study has validated the good predictive value of the Huaxi Unplanned Extubation Risk Assessment Scale. The scale is applicable to the unplanned extubation risk assessment of all types of tubes. The best cut-off value of the scale is 21, and the area under the curve is 0.851.

INTRODUCTION

Unplanned extubation is defined as the tube falling off by itself, premature removal of the tube by action of the patient, or premature removal due to improper operation by medical staff[1]. In recent years, the literature reported that the number of patients with unplanned extubations accounted for 3.6%-32.1% of hospitalized patients with tubes[2-5]. The occurrence density of unplanned extubation was 0.61-6.6 events/100 intubation days[3,4,6,7]. The occurrence of unplanned extubation could interrupt patients’ treatment plans, prolong hospital stays, increase mortality, increase patient pain, increase medical expenses, increase the workload of medical staff, and decrease the bed turnover rate[3,5,8,9]. For patients with unplanned extubation events, 53.5% had adverse consequences[3], the reintubation rate was 28.3%-39.9%[5,9], and the in-hospital mortality rate was 26.4%-39.5%[5,9], significantly higher than those patients without unplanned extubation events. Therefore, the medical industry attaches great importance to the prevention of unplanned extubation. The incidence of unplanned extubation (endotracheal tubes, nasogastric tubes, intravascular catheters, etc.) has become an important indicator for evaluating the quality of nursing[10,11]. So, an effective tool for risk assessment and management of unplanned extubation has become particularly important.

In recent years, a few researchers have developed several scales for the risk assessment of unplanned extubation[12-15]. Wang et al[12] used Delphi method and developed an unplanned extubation risk assessment tool for various types of tubes and patients over 14 years old. However, the details of unplanned extubation were not reported, and the reliability and validity of the scale lacked the support of clinical data. Vats et al[13] developed a scoring tool for unplanned extubation risk, and tried in pediatric patients with endotracheal tubes. While the study did not report the reliability and validity of the scoring tool. Two researches[14,15] designed an assessment tool respectively for unplanned endotracheal extubaiton of artificial airway patients and hospitalized patients with various types of tubes based on literature review and Delphi method. Although the Delphi panel gave good comments, the significance in finding high-risk patients with unplanned extubation lacked clinical application. Although several scales[12-15] for the risk assessment of unplanned extubation have been developed, the reliability and validity of the scales lack the support of clinical data, and their practicability and popularization need to be confirmed. In addition, the applicable population, age, and tubes of different scales are also inconsistent. Critical patients often have various types of tubes, such as endotracheal intubation, central venous catheter, gastric tube, and various drainage tubes, which require a comprehensive and universal risk assessment tool. Therefore, there is still a lack of a unified, efficient, and recognized evaluation tool. The limitations or incorrect use of the unplanned extubation risk assessment tool may lead to improper identification of patients at a high risk of unplanned extubation and cause delay or non-implementation of unplanned extubation prevention interventions. To effectively identify and manage the risk of unplanned extubation, combined with the literature of previous scale development and the literature reports on the risk factors for unplanned extubation, medical experts in our hospital developed a universal risk assessment tool for unplanned extubation.

Therefore, this retrospective review aimed to validate the predictive value of the Huaxi Unplanned Extubation Risk Assessment Scale (HUERAS) for unplanned extubation.

MATERIALS AND METHODS

This was a retrospective study of hospitalized patient records between October 2020 and September 2021 in a comprehensive tertiary hospital in Sichuan Province, Southwest China. The departments included 41 internal medicine wards, 24 surgery wards, and 8 intensive care units. Inclusion criteria were as follows: (1) Inpatients; and (2) patients with at least one invasive tube (excluding an indwelling needle) during hospitalization. Exclusion criteria were as follows: (1) Patients with refusal to participate in the study; and (2) only temporary tubes during operation.

The medical records extracted from the hospital information system consisted of two parts: the general patient characteristics and the unplanned extubation risk assessment score. The patient characteristics included age, sex, admission mode, length of hospital stay, education level, marital status, and whether unplanned extubation occurred during hospitalization. An unplanned extubation event was defined as the tube falling off by itself, premature removal of the tube by patient or medical staff’s improper operation[1]. The risk assessment of unplanned extubation was completed by nurses and recorded in the electronic medical record. Each nurse received training on the use of the unplanned extubation risk assessment scale. The risk assessment has been taken as the routine assessment in our hospital, and it is required to assess when inpatients have tubes or newly placed tubes during hospitalization.

The unplanned extubation risk scores were assessed by the HUERAS. The scale was formulated by the medical experts of the authors’ institution based on the analysis of a large number of unplanned extubation events in the previous years of the medical institution, relevant literature reports on the development of unplanned extubation risk assessment tools, combined with the research results of unplanned extubation risk factor assessment. The scale was developed after two rounds of Delphi expert consultation. The method of expert scoring was adapted for the assignment of each item, according to the importance and risk degree of the item. The results of relevant studies[16-19] on the risk factors related to unplanned extubation showed that the fixation mode of the tube and the activity state of patients were high-risk factors for unplanned extubation. Therefore, based on previous studies, this study focused on tube fixation and the evaluation of patients’ activity ability. The scale is suitable for hospitalized patients with various types of tubes. The scale consists of 10 characteristics: Age, state of consciousness, degree of understanding, emotional state, degree of cooperation, degree of tolerance, number of tubes, types of tubes, fixation mode of tubes, and activities. The total score of the scale is the sum of the scores of each characteristic. The total score is between 10-30. The higher the score, the higher the risk of unplanned extubation. The scoring method is shown in Table 1.

Table 1 Scoring method of the Huaxi Unplanned Extubation Risk Assessment Scale.

Characteristics	Options	Scoring
Age	14-65 years old/< 14 or ≥ 65 years old	1/2
State of consciousness	Medium or deep coma/Awake/Lethargy, light coma, or drowsiness/Blurred consciousness, irritability, or delirium	1/2/3/4
Degree of understanding	Understanding, newborn or deep sedation/Partial understanding/Incomprehension	1/2/3
Emotional state	Stable or deep sedation/Sometimes stable/Unstable	1/2/3
Degree of cooperation	Cooperative/Sometimes cooperative/Uncooperative	1/2/3
Tolerance degree of tubes	Tolerable/Pain or discomfort, but basically tolerable/Pain or discomfort leads to intolerance of the tube	1/2/3
Number of tubes	1/2-3/3 or more	1/2/3
Type of tubes	PICC; CVC, Jejunostomy, Splittable catheter, Pericardial drainage tube, Abdominal wound drainage tube, Bladder and kidney fistulas, Perirenal drainage tube; Tracheotomy tube, Closed thoracic drainage tube, Urinary catheter, Gastric tube, Nasointestinal canal, Double capsule three lumen tube, Radial artery puncture tube, Internal jugular vein puncture tube, Lumbar cistern drainage tube, Ventricular drainage tube and other head drainage tubes, Cervical plasma drainage tube, Breast plasma drainage tube; Oronasal endotracheal intubation	1/2/3/4
Fixation mode of tubes	Suture/Holder, water bag, airbag, or tie wrap/Adhesive tape	1/2/3
Activity ability	Autonomous activity/Using walking aids, unstable walking, or need help/Absolute bed rest	0/1/2

PICC: Peripherally inserted central catheter; CVC: Central venous catheter.

Statistical analyses were conducted using SPSS Statistics version 21.0. The counting data were described by the frequency and composition ratio, and the measurement data were described by the mean ± standard deviation. A t-test was used to compare the unplanned extubation risk scores. Pearson’s correlation analysis was used to analyze the correlation between each characteristic of the scale and the total score. Taking the occurrence of unplanned extubation events during hospitalization as the gold standard, the area under the curve (AUC) was used as the predictive value for the risk of unplanned extubation. The sensitivity, specificity, positive predictive value, negative predictive value, Youden index, and AUC were used to test the predictive validity of the scale.

RESULTS

Characteristics of participants

A total of 76033 inpatients with indwelling tubes were included in this study. The participants were mainly males (52.03%), aged from 1 to 106 years (average 51.12 ± 18.46). The length of hospital-stay ranged from 1 to 357 d (average 10.43 ± 11.55). The chief admission mode was outpatient admission (89.47%), the predominant marital status was married (82.73%), and in the majority of the patients, the number of tubes was one (54.85%) (Table 2).

Table 2 The characteristics of participants (n = 76033).

Character	Category	n	Percentage (%)
Sex	Male	39559	52.03
	Female	36474	47.97
Admission mode	Emergency admission	8009	10.53
	Outpatient admission	68024	89.47
Education	Master and above	1477	1.94
	University	10226	13.45
	College	9715	12.78
	Senior school, or Secondary specialized school	13240	17.41
	Junior high school	20063	26.39
	Primary school	14810	19.48
	Preschool, or illiteracy	6502	8.55
Marital status	Unmarried	8538	11.23
	Married	62902	82.73
	Divorced	1682	2.21
	Widowed	1894	2.49
	Others	1017	1.34
Number of tubes	1	41707	54.85
	2-3	22781	29.96
	> 3	11545	15.18
Discharge mode	Discharged according to doctor’s order	69284	91.12
	Voluntary discharge	2216	2.91
	Death	1041	1.37
	Transfer to another hospital	3326	4.37
	Others	166	0.22

A total of 26 unplanned extubation events occurred during hospitalization. Table 3 shows the basic information of unplanned extubation events.

Table 3 Basic information of unplanned extubation events (n = 26).

Characteristic	Category	n	Percentage (%)
Sex	Male	22	84.62
	Female	4	15.38
Number of indwelling tubes	1	5	19.23
	2-3	9	34.62
	> 3	12	46.15
Date type	Weekdays	24	92.31
	Weekend & holidays	2	7.69
Department type	Internal medicine	13	50.00
	Surgery	11	42.31
	ICU	2	7.69
Occurrence time	Day (8:00-18:00)	7	26.92
	Night (18:00-next day 8:00)	19	73.08
Type of tube extubation	Deep venous catheter (PICC & CVC)	8	30.77
	Urinary catheter	5	19.23
	Drainage tube	5	19.23
	Orogastric tube/Nasointestinal canal	4	15.38
	Endotracheal intubation	3	11.54
	Nasogastric tube & drainage tube	1	3.85
Circumstance of unplanned extubation events	In bed	20	76.92
	During transportation	3	11.54
	Moving out of bed	2	7.69
	Improper operation by medical staff	1	3.85

PICC: Peripherally inserted central catheter; CVC: Central venous catheter; ICU: Intensive care unit.

Reliability analysis of the HUERAS

Cronbach's α coefficient was used to evaluate the internal consistency reliability and was found to be 0.815 in this study. Correlation analysis was used to evaluate the internal correlation of the scale. The correlation coefficients between each characteristic and the total score ranged from 0.183 to 0.843 (Table 4).

Table 4 The correlation coefficient between each characteristic and the total score.

Characteristic	r	P value
Age	0.299	0.000
State of consciousness	0.632	0.000
Degree of understanding	0.814	0.000
Emotional state	0.780	0.000
Degree of cooperation	0.843	0.000
Tolerance degree of tubes	0.653	0.000
Number of tubes	0.724	0.000
Type of tubes	0.684	0.000
Fixation mode of tubes	0.435	0.000
Activity ability	0.183	0.000

Validity analysis of the HUERAS

The patients’ HUERAS scores were between 11 and 30, with an average score of 17.25 ± 3.73. The scores of patients with or without unplanned extubation were 22.85 ± 3.28 and 17.25 ± 3.73, respectively. The score of patients with unplanned extubation was higher than that of those without unplanned extubation (P < 0.001).

The AUC of HUERAS was 0.851, and the 95% confidence interval was 0.783-0.919, P < 0.001.

The sensitivity and specificity corresponding to different cutoff values of the HUERAS are shown in Table 5. The results showed that a score of 20.5 was the best cutoff value, the sensitivity of the scale was 84.6%, and the specificity was 81.4%.

Table 5 Sensitivity and specificity corresponding to each cutoff value of Huaxi Unplanned Extubation Risk Assessment Scale.

Cutoff	Sensitivity	1-specificity	Specificity	Youden indexa
10.0000	1.000	1.000	0	0
11.5000	1.000	1.000	0	0
12.5000	1.000	0.996	0.004	0.004
13.5000	1.000	0.932	0.068	0.068
14.5000	0.962	0.735	0.265	0.227
15.5000	0.962	0.601	0.399	0.361
16.5000	0.923	0.448	0.552	0.475
17.5000	0.885	0.335	0.665	0.55
18.5000	0.885	0.260	0.74	0.625
19.5000	0.846	0.219	0.781	0.627
20.5000	0.846	0.186	0.814	0.66
21.5000	0.769	0.157	0.843	0.612
22.5000	0.692	0.122	0.878	0.57
23.5000	0.500	0.093	0.907	0.407
24.5000	0.346	0.066	0.934	0.28
25.5000	0.115	0.045	0.955	0.07
26.5000	0.077	0.032	0.968	0.045
27.5000	0.038	0.018	0.982	0.02
28.5000	0.000	0.007	0.993	-0.007
29.5000	0.000	0.001	0.999	-0.001
31.0000	0.000	0.000	1	0

^aYouden index = sensitivity + specificity - 1.

Considering that the risk assessment score was an integer, the cutoff was determined to be 21 points. Thus, ≥ 21 points indicated the high-risk state of unplanned extubation. According to this standard, there were 14135 patients with a high risk of unplanned extubation, accounting for 18.59%. The sensitivity, specificity, positive predictive rate, and negative predictive rate of HUERAS were 84.62%, 81.43%, 0.16%, and 99.99%, respectively.

DISCUSSION

Universality of the risk assessment scale

Whether a scale is a specific scale or a universal scale has an important impact on the popularization of the use of the scale. In the past, scale development research for the risk assessment of unplanned extubation often developed and tested specific populations or specific catheters, which had a good effect on the risk assessment of unplanned extubation of specific populations or specific tubes.

Several studies[1,9,20] comprehensively used the CAM- intensive care unit (ICU)[21], the Richmond Agitation Sedation Scale[22], the Glasgow Coma Scale, the Bloomsbury sedation score, and other scales to comprehensively assess the risk of unplanned extubation. However, no special assessment tool has been developed for unplanned extubation risk assessment. Vats et al[13] developed a tool for unplanned extubation risk assessment of children with endotracheal intubation in the ICU and divided the risk of patients into low-, medium-, high- and very high-risk groups according to the score; however, the reliability and validity of the assessment tool were not described in their study. A tool had been developed to assess the risk of unplanned extubation for patients with endotracheal tubes in the ICU[14], but its scope of application was only for patients with endotracheal tubes. Furthermore, there was no actual data pertaining to risk assessment in patients; therefore, its actual predictive validity and application value were limited. Although Wang et al[12] developed a universal risk assessment tool for unplanned extubation in patients with different types of tubes and in different types of departments, patients < 14 years of age were not included, and the specific number of cases and relevant basic information of unplanned extubation were not reported in their research report; thus, their predictive validity and popularization were limited to a certain extent. Previous studies often focused on adult or pediatric patients in the ICU and the endotracheal catheters, while less attention was given to patients in other departments and other types of tubes. However, for hospitalized patients treated with tubes, the type of tubes and departments are different, and patients with tubes belong to various age groups. All these factors should be considered by medical staff.

Therefore, at the beginning of the design of this study, a universal unplanned extubation risk assessment scale suitable for all age groups and various types of tubes was developed. Thus, this study also included various types of wards, such as adult internal/surgical wards, pediatric surgery wards, and pediatric and adult ICUs. Furthermore, the tube types included central venous catheters, gastric tubes, urinary tubes, various drainage tubes, tracheal tubes, and other common clinical types of tubes to verify the universality of the scale. A total of 76033 hospitalized patients with tubes were included in this study, in whom 26 unplanned extubation events occurred. Among the patients with unplanned extubation, 46.15% had > 3 tubes; 92.31% unplanned extubations occurred on working days. Regarding the distribution of departments, medical, surgical, and ICU departments accounted for 50%, 42.31%, and 7.69% of unplanned extubations, respectively. Among the tube types, central venous catheter, urinary catheter, drainage tube, orogastric tube/nasogastric tube, and endotracheal tube accounted for 30.77%, 19.23%, 19.23%, 15.38%, and 11.54% of unplanned extubations, respectively. In addition, a patient had the gastric tube and drainage tube removed at the same time. Although previous studies have tended to focus more on patients with endotracheal intubation in ICU, these data suggest that a larger number of patients with other types of tubes in the general ward also deserve our research and attention.

In terms of the occurence time, 73.08% unplanned extubation occurred in the evening (18:00-the next day 8:00), which was related to the lower nurse-patient ratio in the evening[4,9] and patients’ confusion condition during sleep[23]. From the perspective of sex, males accounted for 84.62% of patients with unplanned extubation, which was consistent with previous studies[9,23] showing that male patients were more prone to unplanned extubation. Because the patients in this study were not limited to the ICU, their activity scenes were not limited to bed. In this study, the scenes of unplanned extubation were also analyzed. A total of 76.92% of cases occurred in bed, 11.54% occurred in the process of transportation, 7.69% occurred in out-of-bed activities, and 3.85% were caused due to improper operation by medical staff. Patients with unplanned extubation were relatively seriously ill and were in a state of sedation, limited bed rest, or physical restraint[23]. During transportation and out-of-bed activities, patients were also prone to unplanned extubation due to the large range of activities or pipeline traction.

Reliability and validity of HUERAS

Internal consistency is an important feature of the reliability of the scale. In this study, the Cronbach’s α was 0.815, which indicated that the scale had good reliability. Correlation analysis is a method to test the structural validity of the scale. The correlation coefficients between each characteristic and the total score in this study ranged from 0.183 to 0.843 (P < 0.001), which indicated that there was a significant correlation between each characteristic and the total score. The above results showed that the risk assessment tool of this study had good reliability and validity.

Predictive value of HUERAS

In the past, only few studies have reported the predictive value of their unplanned extubation risk assessment tool; thus, the practical use was difficult with low popularity. In this study, the HUERAS score in patients with unplanned extubation was 22.85 ± 3.28, which was higher than the score in the group without unplanned extubation (17.25 ± 3.73). It can be seen that the higher the score, the higher the risk of unplanned extubation. The AUC of HUERAS was 0.851. According to the classification standard of AUC, the scale had high accuracy in screening for the risk of unplanned extubation in hospitalized patients with tubes.

Cutoff of the HUERAS

In this study, the Youden Index was used to determine the cutoff of the unplanned extubation risk assessment tool. The statistical results showed that the best cutoff score on the scale was 20.5 points. Considering that the risk assessment score of unplanned extubation was an integer, the judgment standard of high risk of unplanned extubation was set at ≥ 21 points. Thus, the sensitivity and specificity of HUERAS were 84.62% and 81.43%, respectively, indicating that the assessment tool had a strong and balanced ability to identify high-risk groups of unplanned extubation. In this study, the positive predictive value was 0.16%, and the negative predictive value was 99.99%, indicating that in patients assessed as having a high risk of unplanned extubation, the proportion of patients with unplanned extubation was low. Considering that patients with unplanned extubation accounted for only 0.034% (26 cases) of the patients in this study, the low positive predictive value was in line with the actual situation. Among the low-risk patients with unplanned extubation, patients without unplanned extubation accounted for 99.99%, indicating that the exclusion rate of the scale for low-risk patients with unplanned extubation was very high and had good prediction ability for the low-risk population.

Limitations

Although this study was based on the risk assessment results of a large number of hospitalized patients with tubes, only 26 unplanned extubation events were actually reported, which was not really high enough to support all the research results based on statistical analyses in this study. Because the number of events were small compare to number of patients in the study, thus the fragility index was quite high. The possible causes, on the one hand, this study was concerned about the various types of tubes in patients, training nurses to conduct risk assessment could also improve nurses’ attention to the prevention of unplanned extubation. Nurses also better performed the preventive measures of unplanned extubation, such as secondary fixation, effective communication between nurse and patient, pain and sedation management. On the other hand, the effect of reporting bias cannot be ruled out. But this was a good beginning in research of this important topic. In the follow-up research, the team will continue to conduct in-depth study on this topic.

CONCLUSION

The HUERAS had good predictive validity and could effectively identify hospitalized patients with a high risk of unplanned extubation. This scale may help clinical nurses and nursing managers to accurately identify high-risk patients and take effective preventive measures in time to prevent the occurrence of unplanned extubation in hospitalized patients with tubes.

ARTICLE HIGHLIGHTS

Research background

Critical patients often had various types of tubes, unplanned extubation of any kind of tube may cause serious injury to the patient, but previous reports mainly focused on endotracheal intubation. The limitations or incorrect use of the unplanned extubation risk assessment tool may lead to improper identification of patients at a high risk of unplanned extubation and cause delay or non-implementation of unplanned extubation prevention interventions.

Research motivation

Previous studies about unplanned extubation risk assessment lacked the support of clinical data. The reliability and validity of the previous risk assessment scales and their practicability and popularization cannot be confirmed. To effectively identify and manage the risk of unplanned extubation, a comprehensive, universal, and effective unplanned extubation risk assessment tool is needed.

Research objectives

To assess the predictive value of the Huaxi Unplanned Extubation Risk Assessment Scale in inpatients.

Research methods

We performed a retrospective validation study. For patients with tubes during hospitalization, the patient characteristic, whether unplanned extubation occurred and the Huaxi Unplanned Extubation Risk Assessment Scale (HUERAS) score were extracted. The best cut-off value and the area under the curve (AUC) of the Huaxi Unplanned Extubation Risk Assessment Scale were been identified.

Research results

A total of 76033 inpatients with indwelling tubes were included in this study, and 26 unplanned extubations occurred. The best cut-off value was 21, and the Cronbach’s α, sensitivity, specificity, positive predictive value, and negative predictive value of the HUERAS were 0.815, 84.62%, 81.43%, 0.16%, and 99.99%, respectively. The AUC of HUERAS was 0.851 (95%CI: 0.783-0.919, P < 0.001). The prediction validity and generalization of the HUERAS need to be further confirmed by multi center research.

Research conclusions

The HUERAS has good reliability and predictive validity. It can effectively identify inpatients at a high risk of unplanned extubation and help clinical nurses carry out risk screening and management.

Research perspectives

Larger studies with multiple centers are needed to further confirm the prediction validity and generalization of the HUERAS.

ACKNOWLEDGEMENTS

The authors thank all the study participants who were enrolled in this study.