Evidence-Based Medicine
Copyright ©The Author(s) 2018. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Crit Care Med. Aug 4, 2018; 7(3): 39-45
Published online Aug 4, 2018. doi: 10.5492/wjccm.v7.i3.39
Validation of the VitalPAC Early Warning Score at the Intermediate Care Unit
Joost DJ Plate, Linda M Peelen, Luke PH Leenen, Falco Hietbrink
Joost DJ Plate, Luke PH Leenen, Falco Hietbrink, Division of Surgery, University Medical Centre Utrecht, Utrecht University, Utrecht 3584 CX, The Netherlands
Linda M Peelen, Julius Center for Health Sciences, University Medical Center Utrecht, Utrecht University, Utrecht 3584 CX, The Netherlands
Linda M Peelen, Departments of Anaesthesiology and Intensive Care Medicine, University Medical Center Utrecht, Utrecht University, Utrecht 3584 CX, The Netherlands
Author contributions: All authors contributed to this manuscript.
Conflict-of-interest statement: There are no conflicts of interest arising from this work.
Open-Access: This article is an open-access article, which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Correspondence to: Joost DJ Plate, MD, MSc, Research Scientist, Division of Surgery, University Medical Centre Utrecht, Utrecht University, Heidelberglaan 100, Utrecht 3584 CX, The Netherlands. j.d.j.plate@umcutrecht.nl
Telephone: +31-88-7555555 Fax: +31-88-7555438
Received: March 20, 2018
Peer-review started: March 20, 2018
First decision: April 23, 2018
Revised: June 19, 2018
Accepted: June 26, 2018
Article in press: June 27, 2018
Published online: August 4, 2018
Processing time: 139 Days and 12.7 Hours
ARTICLE HIGHLIGHTS
Research background

Early Warning Scoring (EWS) systems to recognize the clinically deteriorating patient are widely used in the clinical setting, including in Intermediate Care Units (IMCUs). However, they have been developed and validated for the general hospital ward population and hence their applicability within the IMCU population is unclear.

Research motivation

The application of prediction models (EWS) at a different setting than the setting at which they were developed (IMCU instead of hospital ward), could lead to an inefficient use of scarce resources and may compromise patient safety. To justly consider the (ongoing) use of the EWS at the IMCU, its discriminative performance and applicability need to be investigated.

Research objectives

This validation study aims to assess the performance and clinical relevance of the VitalPAC-EWS (ViEWS) at the IMCU. Further, it aims to improve the EWS for its use at the IMCU.

Research methods

Electronically collected data from 2014 to 2016 at the IMCU were used to obtain the area under the receiver operating curve (AUC) and the number needed to trigger (false alarm rate) at the current and the optimal threshold.

Research results

The AUC of the ViEWS was 0.72 (CI: 0.69-0.75). The number needed to trigger was 19 per one event. Although the discriminative performance is acceptable, the clinical relevance is limited as 19 false alarms are needed per one event. This carries the risk of alarm fatigue. Therefore, this study contributes to this research field that the use of the EWS at the stand-alone IMCU should be reconsidered. The main problem that remains to be solved are that an alternative system needs to be developed to timely detect clinical deterioration at the IMCU.

Research conclusions

The new findings of this study are that the use of the ViEWS at the IMCU should be reconsidered. It proposes that this is due to remarkable case-mix differences between the hospital ward and the IMCU. This study proposes to use new methods to detect clinical deterioration at the IMCU, using automated data collection and perhaps more sophisticated statistical methods. The implication for clinical practice is that the EWS in its current form at the IMCU should perhaps not be used.

Research perspectives

General experiences and lessons that can be learned from this study are that prediction models should not be used in different settings without prior validation. Further research should focus on alternative methods to detect the clinically deteriorating patient at the IMCU, through the modelling of repeated measurements in prediction models. Also, further research should focus on the use of the EWS in differently formatted IMCUs, such as the IMCU that is integrated into the ICU.