The artificial intelligence evidence-based medicine pyramid

Abstract

Several studies exist in the literature regarding the exploitation of artificial intelligence in intensive care. However, an important gap between clinical research and daily clinical practice still exists that can only be bridged by robust validation studies carried out by multidisciplinary teams.

Key Words: Artificial intelligence; Intensive care; Intensive care unit; Evidence-based medicine; Clinical research

Core Tip: Artificial intelligence (AI) use in intensive care is now a reality. However, there is still an important discrepancy between the results found in the scientific literature and the day-to-day clinical implementation of this technology. One reason for this is that the AI evidence pyramid in intensive care has only just begun to emerge. We need to focus on the next steps in AI pyramid evidence, amplifying the external validation of models and increasing the number of randomized clinical trials. Only robust validation studies carried out by multidisciplinary teams will help bridge this existing gap between clinical research and clinical practice.

TO THE EDITOR

We read with great interest the editorial by Luo et al[1] where the authors cogently present the main results regarding the use of artificial intelligence (AI) in the intensive care unit (ICU) for decision making and resource allocation. They simultaneously exposed the current limitations of the large-scale use of AI clinical tools in this setting. We share many of the reflections set out by Luo et al[1]. The presence of AI in medicine science and clinical practice has become a reality. Knowing how this new technology can assist the medical profession and how clinicians might take advantage of it are characteristics that are now required and are likely to be of assistance as far as personal career development is concerned[2]. However, the gap between the excellent results derived from biomedical research and the rare use in clinical practice is clear to everyone[3]. While this is probably the biggest deterrent to AI application on a daily professional basis, we must not stop considering it as a valuable ally. On the contrary, we need to ask clinical researchers to find answers for how these models can help intensivists carry out day-to-day activities.

Without external validation, the positive performance of these models in observational studies is no longer sufficient. This, however, should not lead to the erroneous conviction that AI implementation in the ICU should remain purely a scientific speculation, as its application outside the clinical reality regularly disproves this hypothesis. Intelligent vocal assistants and accurate search engines are just two examples of the efficient support offered to us by well-devised AI. The first results from clinical trials point in the same direction, with an example being the hypotension prediction index[4]. This is an algorithm implemented to predict hypotension, even before adverse events occur. Since its marketing, a number of clinical trials have tried to interpret its possible usefulness in clinical practice with most results showing a lower incidence of hypotensive events when compared with standard care[5-7].

We should bear in mind that anything stemming from evidence-based medicine (EBM) has a history based on the progressive collection of increasingly solid results, and the application of AI in the ICU follows the same path (Figure 1). We began with the intuition that AI might be useful in critical patients. Subsequently, stronger results, initially from retrospective followed by prospective observational studies, appeared. In the literature, a few clinical trials as well as sporadic systematic reviews and meta-analyses are available[8,9]. Presently, we are only halfway up the pyramid of the AI scientific evidence we initially imagined, and it is therefore logical that the use of AI tools is not widespread. This phenomenon is consistent with the concept of EBM. At this point, we need to focus on the second part of the pyramid, increasing the external validation of models and multiplying the number of randomized clinical trials.

Figure 1 A pyramid for artificial intelligence scientific evidence is proposed. Starting from the bottom and moving to the top, emerging results are becoming increasingly solid and strong. The two lowest rungs are the theory followed by the third, fourth and fifth steps that represent studies analyzing the use of artificial intelligence (AI) in clinical practice. From creation of the model with internal validation, we move towards external validation studies and the creation of usable real instruments (AI tools). The penultimate step [randomized controlled trials (RCTs)] and the tip of the pyramid (meta-analysis and systematic reviews) represent the strongest methodological analysis to reach conclusions on the real impact of this technology on healthcare systems. If we then imagine the support base of the pyramid we have the necessary tools for each step of clinical research in AI applied to the intensive care unit: Electronic health record, solid big data systems, internet of things technologies and models of eXplainable AI.

Furthermore, we must not underestimate the fact that this gap can only be bridged by the intervention of multidisciplinary teams. As with the creation of the AI surgical department in anesthesiology[10], similar systems need to be considered for the ICU. Engineers, data scientists and intensivists must create units capable of managing each phase of the AI application in the ICU, from the design and then to the creation and exploitation of AI clinical instruments. This cooperation should also take place in the post-marketing phase, with constant verification of the quality and safety of AI tools together with continuous systems updates. In conclusion, it is not surprising that AI is not yet widely used in daily ICU activities. We are still at the very beginning of the EBM pyramid, and the gap between bytes and the bedside will only be bridged by robust validation studies carried out by multidisciplinary teams.