Observational Study Open Access
Copyright ©The Author(s) 2021. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Clin Cases. Sep 6, 2021; 9(25): 7405-7416
Published online Sep 6, 2021. doi: 10.12998/wjcc.v9.i25.7405
Role of international normalized ratio in nonpulmonary sepsis screening: An observational study
Jing Zhang, Fa-Ming He, Department of Emergency and Intensive Care Medicine, The First Affiliated Hospital of Chongqing Medical University, Chongqing 400016, China
Hui-Min Du, Department of Oncology, The First Affiliated Hospital of Chongqing Medical University, Chongqing 400016, China
Ming-Xiang Cheng, Department of Hepatobiliary Surgery, The Second Affiliated Hospital of Chongqing Medical University, Chongqing 400016, China
Bai-Lin Niu, Department of Emergency and Critical Care Medicine, The First Affiliated Hospital of Chongqing Medical University, Chongqing 400016, China
ORCID number: Jing Zhang (0000-0002-9811-2419); Hui-Min Du (0000-0002-2561-0062); Ming-Xiang Cheng (0000-0002-7781-4075); Fa-Ming He (0000-0001-5314-3484); Bai-Lin Niu (0000-0001-9582-7884).
Author contributions: Niu BL contributed to the study conception and design; Zhang J, Du HM, Cheng MX and He FM contributed to data collection and analysis; Zhang J and Du HM wrote the first draft of the manuscript; Niu BL further polished the manuscript and authorized the publication of the manuscript; Zhang J, Du HM and Cheng MX contributed equally to this work.
Supported by the Joint Scientific Research Project of Chongqing Science and Technology Commission and Health Commission, No. 2019MSXM019 and No. 2021MSXM033; Natural Science Project of Science and Technology Department of Tibet Autonomous Region, No. XZ2019ZR-ZY55(Z); Chongqing Natural Science Foundation Project, No. cstc2020jcyj-msxmX0124; and COVID-19 Emergency Projects of Chongqing Medical University, No. X1-2611.
Institutional review board statement: The study was reviewed and approved by the First Affiliated Hospital of Chongqing Medical University Ethics Committee (Approval No. 2019-312).
Informed consent statement: Informed consent to participate in this study which have been obtained from the research subjects prior to study commencement, and the study participants gave consent to have their data published. All informed consents are in the form of hand-signed consent forms by participants.
Conflict-of-interest statement: The authors declare that they have no competing interests.
Data sharing statement: The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.
STROBE statement: The authors have read the STROBE Statement—checklist of items, and the manuscript was prepared and revised according to the STROBE Statement—checklist of items.
Open-Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/Licenses/by-nc/4.0/
Corresponding author: Bai-Lin Niu, MD, Associate Professor, Department of Emergency and Critical Care Medicine, The First Affiliated Hospital of Chongqing Medical University, No. 1 Youyi Road, Yuzhong District, Chongqing 400016, China. nberlin@126.com
Received: April 26, 2021
Peer-review started: April 26, 2021
First decision: May 23, 2021
Revised: June 6, 2021
Accepted: July 19, 2021
Article in press: July 19, 2021
Published online: September 6, 2021

Abstract
BACKGROUND

Currently, there is a lack of sepsis screening tools that can be widely used worldwide. Pulmonary sepsis can be of sufficient concern to physicians due to their noticeable symptoms, which usually rely less on screening tools.

AIM

To investigate the efficiency of the international normalized ratio (INR) for the early rapid recognition of adult nonpulmonary infectious sepsis.

METHODS

This is a prospective observational study. A total of 108 sepsis patients and 106 nonsepsis patients were enrolled according to relevant inclusion and exclusion criteria. Commonly used clinical indicators, such as white blood cell, neutrophil count, lymphocyte count, neutrophil-lymphocyte count ratio (NLCR), platelets (PLT), prothrombin time, INR, activated partial thromboplastin time, and quick Sequential “Sepsis-related” Organ Failure Assessment (qSOFA) scores were recorded within 24 h after admission. The diagnostic performances of these clinical indicators were analyzed and compared through multivariate logistic regression analysis, Spearman correlation, and receiver operating characteristic curve analysis.

RESULTS

The INR value of the sepsis group was significantly higher than that of the nonsepsis group. INR has superior diagnostic efficacy for sepsis, with an area under the curve value of 0.918, when those preexisting diseases which significantly affect coagulation function were excluded. The diagnostic efficacy of the INR was more significant than that of NLCR, PLT, and qSOFA (P < 0.05). Moreover, INR levels of 1.17, 1.20, and 1.22 could be used to categorize the relative risk of nonpulmonary infections sepsis into three categories: low, medium and high risk, respectively.

CONCLUSION

The INR is a promising and easily available biomarker for diagnosis, and it can be used as one of the indicators for early screening of adult nonpulmonary infectious sepsis. When its value is higher than the optimal cutoff value (1.22), high vigilance is required for adult nonpulmonary infectious sepsis.

Key Words: Sepsis, Coagulopathy, International normalized ratio, Screening tool, Quick Sequential “Sepsis-related” Organ Failure Assessment

Core Tip: The international normalized ratio (INR) has high specificity and sensitivity in the early identification of adult nonpulmonary source of sepsis. Sepsis is highly suspected when the INR value exceeds 1.22 in patients with non-pulmonary infection, especially for those patients without preexisting underlying disease or medication history that affects coagulation function. Due to its low cost, fast detection and easy interpretation, INR is suitable for the primary screening of sepsis for emergency patients, outpatient patients, particularly in low and middle-income countries.



INTRODUCTION

The incidence of sepsis has been increasing year by year, and it has become one of the leading causes of death in intensive care unit (ICU) patients[1]. Studies have confirmed that sepsis results in a "bimodal" distribution of death times, which are early death within a few days and late death within a few weeks or months[2,3]. Early death is mainly caused by an aggressive inflammatory response, while persistent immunosuppression leads to late death. Owing to the mechanism of late immunosuppression not being clear, immunomodulatory treatment is not yet very effective, and more than 40 large clinical studies of early anti-inflammatory therapy have also failed; early recognition might be an important way to improve the treatment of sepsis in the short term[4]. At present, Sepsis-3 is newly defined as sequential impairment of organ function due to suspected infection along with the Sequential “Sepsis-related” Organ Failure Assessment (SOFA) score increased 2 points or more[5], which emphasizes organ dysfunction caused by infection. Although Sepsis-3 increased the specificity of in-hospital diagnosis[6], it was not widely used in clinical practice because of its multiple complex indicators; it is difficult to carry out rapidly and widely in low and middle-income areas, especially in emergency and outpatient departments, and in nonintensive care units. The quick SOFA (qSOFA) score, as a fast, simple, and noninvasive screening tool, is widely used in clinical practice. However, Askim et al[7] found that the qSOFA score had a low sensitivity to screen sepsis, and Williams et al[8] found that the sensitivity was only 29.7% when the qSOFA score was ≥ 2 points for screening sepsis. Thus, the qSOFA score as a rapid screening tool for sepsis remains controversial.

In clinical practice, the clinical symptoms of patients with pulmonary infections are often conspicuous, which are likely to arouse the vigilance and attention of clinicians. Therefore, for outpatients with pulmonary infection, sepsis screening tools usually are less necessary. However, patients with nonpulmonary infections, such as abdominal and urinary tract infections, are more likely to deteriorate into sepsis or septic shock, with the incidence of both as high as 50%[9]. Therefore, these patients are more likely to need tools for rapid sepsis screening. Usually, a good screening tool should not only have sufficient sensitivity and specificity, but also be able to reflect the physiological characteristics of the disease.

Sepsis is a severe systemic inflammatory response that manifests with widespread inflammation, as well as endothelial and coagulation dysfunction[10]. A mass of studies has shown that sepsis patients are associated with different degrees of coagulopathy, which exists over the whole process in sepsis[11]. Stimulated by inflammatory factors, coagulopathy appears from the early pro-coagulant state of sub-clinical symptoms to the disseminated intravascular coagulation (DIC) at the terminal stage[12]. The platelet count, an indicator of coagulopathy status, also appear in the diagnostic criteria of Sepsis-3[13]. Thus, further exploration of the indicators of coagulation function status is expected to develop new screening tools for sepsis. Currently, there are many tools for coagulation function tests, such as conventional coagulation examinations [including parameters: Prothrombin time (PT), activated partial thromboplastin time (APTT), international normalized ratio (INR), etc.], thromboelastography (TEG)[12], rotational thromboelastometry (ROTEM)[14], etc. However, TEG and ROTEM are more commonly used in cardiovascular surgery to evaluate platelet function, coagulation and fibrinolytic systems, and blood transfusion in the early resuscitation stage of trauma patients[15,16]. In addition, TEG and ROTEM cannot be routinely performed in medical institutions at all levels. In contrast, the conventional coagulation examinations can be used as screening tools for the early and rapid identification of sepsis. In particular, the development of Point-of-Care INR Test further shorten its detection time[17]. Currently, blood cell analysis, as a routine test, has been widely performed in clinical practice. Thus, the purpose of this study was to explore the efficiency of routine indicators of blood cells and coagulation function in the early rapid recognition of adult nonpulmonary infectious sepsis.

MATERIALS AND METHODS
Study population

This was a prospective observational study with a small sample size conducted in the Department of the Emergency and Intensive Care Unit of The First Affiliated Hospital of Chongqing Medical University, which is a 3200-bed tertiary care teaching hospital with an annual load of approximately 154000 patients. Patients with suspected infection were enrolled in this study from August 2019 to July 2020. The study was approved by the Ethics Committee of the First Affiliated Hospital of Chongqing Medical University in compliance with the Declaration of Helsinki.

According to the Sepsis-3 criteria, the study subjects were divided into nonsepsis and sepsis groups. There were not consecutive patients enrolled. The inclusion criteria were the following: Age > 18-years-old and not limited by sex; and patients with confirmed nonpulmonary infections. The exclusion criteria were the following: Age < 18-years-old; patients with some preexisting diseases that may have notably affected their coagulation function, such as chronic liver diseases, hematologic system diseases, or patients who had previously undergone long-term treatment with immunosuppressants or anticoagulants; and those patients with incomplete data. The sex, age, and other basic data of all of the included patients were recorded. Acute Physiology and Chronic Health Evaluation II (APACHE II), SOFA and qSOFA scores were assessed on the day of admission within 24 h. These indicators, as well as white blood cell (WBC), neutrophil count (N#), lymphocyte count (L#), neutrophil–lymphocyte count ratio (NLCR), platelets (PLT), PT, INR and APTT, were recorded. Patients were assigned to the sepsis group when their SOFA score increased by 2 or more points, that is, when they met the diagnostic criteria for Sepsis-3.

Defining suspected infection and low-medium-high risk of sepsis

The first suspected infections were defined as a combination of antibiotics (either orally or intravenously) and body fluid cultures (pleural effusion, blood, abdominal effusion, urine, etc.). We need make sure that the combination of culture and antibiotics occurred within a specific time frame. If antibiotics were given first, culture samples need be obtained within 24 h. If culture samples were performed firstly, antibiotics must be given within 24 h. The time at which either of the above two events occurred was defined as the "onset" of infection. The sites of infection in deep tissue were determined by CT scan, while superficial lesions could be identified by physical examination. In order to obtain the predictive efficacy of different INR values on the low, medium and high risk of sepsis, we defined the positive predictive rate below 10%, around 50%, and above 85% as low, medium and high risk of sepsis, respectively.

Relevant biomarkers and determination

Peripheral venous blood samples were collected immediately after admission. Peripheral blood cell (including WBC, L#, N#, NLCR, PLT) were analyzed by the Sysmex XN-9000 (Kobe, Japan) through flow cytometry, and coagulation tests (including INR, PT, APTT) were measured with the Sysmex CS-5100 (Shanghai, China). All the tests were carried out in the Medical Laboratory of our hospital according to the standard procedures, the quality of which have been certified by the American Society of Pathologists.

Statistical analysis

The SPSS software, version 24.0 (IBM Corp. Armonk, NY, United States) and MedCalc software, version 19.0 (MedCalc Software, Belgium) were used for statistical analysis. All of the measured data in this study were subject to normal distributions, which were analyzed by the Kolmogorov-Smirnov test. Data were expressed as mean ± SD. For comparisons of two independent group of samples, Student’s t-test was employed. The chi-square test was used to compare the composition ratio between the two groups. Logistic regression analysis was performed to analyze the significance of INR in the early identifying of sepsis, through the univariate and multivariate analysis (forward, LR). The receiver operating characteristic (ROC) curves were plotted by the software. INR, PT, PLT, WBC, NLCR and qSOFA were compared for their efficacy in the early identification of sepsis, according to the area under their ROC curves (AUC). The cut-off points, sensitivities, specificities, positive predictive values, and negative predictive values of these indicators were, respectively, calculated to evaluate their diagnostic efficiency. Relationship of INR value with SOFA score and APACHE II score were analyzed through Spearman’s rank correlation.

RESULTS
Patient enrollment and comparison of baseline characteristics

301 patients were initially enrolled in this study, of whom 201 met all the inclusion criteria and were identified as study subjects (Figure 1). The baseline characteristics of the patients are shown in Table 1. There were no differences between the nonsepsis and sepsis groups in sex, age, or infection site (all P > 0.05), indicating that the baseline data were comparable.

Figure 1
Figure 1  Study population.
Table 1 Population and baseline characteristics.

Sepsis
Nonsepsis
P value
Sex, n (%) 0.083
Male 52 (48.1)59 (55.7)
Female 56 (51.9)47 (44.3)
Age, yr (mean ± SD)55 ± 952 ± 150.12
Body mass index (mean ± SD)24.1 ± 2.223.8 ± 2.50.25
Site of infection, n (%) 0.832
Abdominal50 (46.3)47 (44.3)
Urinary36 (33.3)35 (33.0)
Skin and others14 (13.0)18 (17.0)
Multiple sites8 (7.4)6 (5.7)

The comparison of WBC, NLCR, INR, PLT, PT, APTT and other indices and levels between the sepsis and nonsepsis groups is shown in Table 2, and it was found that all of them were significantly higher in the sepsis group than in the nonsepsis group (P < 0.05).

Table 2 Comparison of the international normalized ratio, white blood cell, neutrophil-lymphocyte count ratio, platelet, prothrombin time, activated partial thromboplastin time, Sequential “Sepsis-related” Organ Failure Assessment scores, quick Sequential “Sepsis-related” Organ Failure Assessment scores and Acute Physiology and Chronic Health Evaluation II scores between the sepsis and nonsepsis groups.

Sepsis
Nonsepsis
P value
WBC (× 109/L)12.90 ± 8.769.88 ± 5.090.022
NLCR28.45 ± 21.0510.44 ± 11.87< 0.001
INR1.49 ± 0.471.11 ± 0.15< 0.001
PLT (× 109/L)135.09 ± 92.35229.5 ± 72.83< 0.001
PT (s)17.42 ± 4.7613.86 ± 1.75< 0.001
APTT (s)46.06 ± 10.3036.88 ± 12.11< 0.001
SOFA6.93 ± 2.690.72 ± 0.45< 0.001
qSOFA1.17 ± 0.660.79 ± 0.750.003
APACHE Ⅱ18.81 ± 6.909.29 ± 4.14< 0.001
Analysis of factors influencing the identification of sepsis

Univariate and multivariate analyses for the diagnosis of sepsis showed that WBC, NLCR, the INR, PLT, PT, APTT, and qSOFA were of significance in the diagnosis of sepsis (P < 0.05) (Table 3). Logistic regression multivariate analysis was performed on WBC, NLCR, the INR, PLT, PT , APTT, and qSOFA, among which WBC, NLCR, the INR, PLT, and PT were statistically significant (OR = 0.875, 95%CI: 0.772-0.992, P = 0.037; OR = 1.145, 95%CI: 1.069-1.226, P < 0.001; OR = 27.106, 95%CI: 5.038-145.825, P < 0.001; OR=0.981, 95%CI: 0.97-0.991, P < 0.001; OR = 1.475, 95%CI: 1.032-2.106, P = 0.033, respectively); Thus, WBC, NLCR, the INR, PLT, and PT were associated with the diagnosis of sepsis, while APTT and qSOFA were not.

Table 3 Analysis of influencing factors for the diagnosis of sepsis.
Univariate

Multivariate


OR (95%CI)
P value
OR (95%CI)
P value
WBC (× 109/L)1.066 (1.007-1.128)0.0290.875 (0.772-0.992)0.037
NLCR1.092 (1.051-1.135)< 0.0011.145 (1.069-1.226)< 0.001
INR35.214 (12.887-96.223)< 0.00127.106 (5.038-145.825)< 0.001
PLT (× 109/L)0.985 (0.980-0.991)< 0.0010.981 (0.97-0.991)< 0.001
PT (s)1.868 (1.458-2.393)< 0.0011.475 (1.032-2.106)0.033
APTT (s)1.091 (1.045-1.139)< 0.0010.190
qSOFA2.122 (1.265-3.559)0.0040.907
Comparative analysis of ROC curves

ROC curve analysis displayed that the INR had the largest AUC value for the diagnosis of sepsis: 0.918 (95%CI: 0.857-0.959) (Figure 2 and Table 4). The AUC values for other biomarkers were, respectively, as follows: PT 0.868 (95%CI: 0.796-0.921); PLT 0.841 (95%CI: 0.766-0.9); NLCR 0.83 (95%CI: 0.754-0.891); qSOFA 0.638 (95%CI: 0.548-0.721); and WBC 0.599 (95%CI: 0.508-0.684). The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for WBC, NLCR, the INR, PT, PLT, and qSOFA are depicted in Table 4. The sensitivity and specificity were 90.0% (95%CI: 0.805-0.959) and 84.48% (95%CI: 0.726-0.927), and the PPV and NPV were 87.5% (95%CI: 0.793-0.928) and 87.5% (95%CI: 0.775-0.934), respectively, when the INR cutoff value was 1.22.

Figure 2
Figure 2 Receiver operating characteristic curve of white blood cell, neutrophil-lymphocyte count ratio, platelet, prothrombin time, international normalized ratio and quick Sequential “Sepsis-related” Organ Failure Assessment score within 24 h after admission between sepsis and nonsepsis. The results showed that the area under the curve of international normalized ratio (INR), prothrombin time (PT), neutrophil-lymphocyte count ratio (NLCR), platelet (PLT), quick Sequential “Sepsis-related” Organ Failure Assessment (qSOFA), and white blood cell count (WBC) in the diagnosis of sepsis were 0.918, 0.868, 0.83, 0.841, 0.638 and 0.599, respectively, and they were statistically significant (P < 0.05). The study indicates that the INR could be a helpful diagnostic biomarker in sepsis.
Table 4 Performance characteristics of the single biomarker for diagnosing sepsis.

AUC (95%CI)
Cut-off value
Sensitivity (%)
Specificity (%)
PPV (%)
NPV (%)
WBC (× 109/L)0.599 (0.508-0.684)11.2948.5772.416853.8
NLCR0.83 (0.754-0.891)11.8681.4377.5981.477.6
INR (× 109/L)0.918 (0.857-0.959)1.229084.4887.587.5
PLT0.841 (0.766-0.9)1808082.7684.877.4
PT (s)0.868 (0.796-0.921)15.38087.9388.978.5
qSOFA ≥ 20.638 (0.548-0.721)28.5779.3162.547.9
The correlation of INR with SOFA and APACHE II score, and low-medium-high risk of sepsis according to INR

Since SOFA score and APACHE II score are not Gaussian distribution, Spearman’s rank correlation method was used. It can be found that there is a strong correlation between INR value and SOFA score (r = 0.660, 95%CI: 0.574-0.731, P < 0.001), while there is a weak correlation between INR value and Apache II score (r = 0.457, 95%CI: 0.341-0.560, P < 0.001). When INR value was less than or equal to 1.17, 1.20 and greater than or equal to 1.22, respectively, with PPV of 8.24%, 55.6% and 87.5%, there were low, medium, and high risk of sepsis (Figure 3).

Figure 3
Figure 3 Correlation analysis of international normalized ratio value with Sequential “Sepsis-related” Organ Failure Assessment score and Acute Physiology and Chronic Health Evaluation II score, and analysis of international normalized ratio value for predicting the risk of nonpulmonary sepsis. A: Spearman rank correlation analysis between international normalized ratio (INR) value and Sequential “Sepsis-related” Organ Failure Assessment (SOFA) score; B: Spearman rank correlation analysis between INR value and Acute Physiology and Chronic Health Evaluation II (APACHE II) score; C: The positive prediction rate and negative prediction rate of sepsis under different INR values. 95%CI: 95% confidence interval.
DISCUSSION

Early diagnosis and timely treatment of sepsis can improve the prognosis and reduce the mortality of sepsis. Although Sepsis-3 revealed organ dysfunction caused by infection, it is difficult to diagnose early for patients who have severe infections not reaching significant organ failure[13]. In clinical practice, blood culture is still used to diagnose sepsis. Due to culture time, will take a longer time and has a less positive rate, it usually leads to delayed and missed diagnoses of sepsis. Exploring new and effective markers for the screening of sepsis is necessary. Zboromyrska et al[18] studied multiplex real-time polymerase chain reaction, the MagicplexTM Sepsis test, the specificity of which was up to 95%, while the sensitivity was only 29%. Dong et al[19] found that members of the miR-148 family (miR-148A/B and miR-152) were candidate biomarkers for sepsis by studying the Gene Expression Omnibus dataset GSE12624, but further study is required to evaluate the diagnostic performance. Guo et al[20] found that the AUC value of miR-495 in the diagnosis of sepsis was 0.915 when the cutoff value was 0.655, the sensitivity was 89.5%, and the specificity was 83.0%. Although all these new markers have their own advantages, the disadvantage is that their detection takes a long time, costs a lot, and has limited efficiency, which is difficult to carry out in economically underdeveloped areas. As we learn more about the pathophysiology of sepsis, we may be able to go back and develop new screening indicators for tests routinely performed in the laboratory.

Coagulopathy is one of the characteristic pathophysiological changes of sepsis, which exists across the whole process in sepsis and is a critical factor for the occurrence, development, and prognosis of sepsis[21]. With the persistence of infection or sepsis, the consumption of coagulation factors and decrease of platelets will occur, as well as the bleeding and dysfunction of the skin, mucous membranes and other organs, constituting DIC[22]. Therefore, the indicators related to coagulation function have great potential to be used as screening indicators for sepsis. Zhang et al[23] found that the AUC of PT for the diagnosis of sepsis was 0.806 when the cutoff value was 20, and the sensitivity and the specificity were 83.54% and 65.22%, respectively. In this study, we investigated common indicators of coagulation function and some markers of inflammation, such as WBC, NLCR, PLT, INR, PT, APTT, and qSOFA scores, in the identification efficiency of adult nonpulmonary infectious sepsis. As for procalcitonin, there have been quite a lot of relevant studies reported[24-29]. In addition, procalcitonin is not a routine test item that can be carried out in all economically challenged areas. Here, we mainly analyzed the most commonly used test indicators. Our results showed that the INR had the highest efficiency in the diagnosis of adult nonpulmonary infectious sepsis, compared with WBC, NLCR, PLT, PT, APTT and qSOFA. Lyons et al[30] found that coagulopathy was closely related to the severity and mortality of sepsis, and with the extension of the INR, the more severe that the sepsis was, the higher that the mortality was. Recent studies of coronavirus disease 2019 patients with secondary bacterial infection have also reported that prolonged INR time is a monitoring indicator in severe patients, and the longer the INR time is, the worse the prognosis of patients[31-33]. This study also found that there was a strong correlation between INR value and SOFA score (r = 0.660). The SOFA score was correlated with the prognosis of sepsis, which also suggested that INR had a good predictive value of nonpulmonary infectious sepsis from another perspective. In particular, when INR was less than or equal to 1.17, between 1.17 and 1.20, or greater than or equal to 1.22, the high positive predictor rates of sepsis could be broken into low, medium and high levels. All the above related studies have indicated that INR could be used as one of the early screening indicators for sepsis; however, there are many underlying diseases that can affect INR. Although relevant conclusions in this study were drawn under corresponding exclusion criteria, it is true that in clinical practice, there are still a small number of potentially infected patients with those underlying diseases that cannot be detected an initial presentation. For this condition, when their INR reach the high-risk value of sepsis, e.g., more than 1.22, sepsis should be highly suspected because they are at higher risk of adverse outcomes when they are missed.

Many studies have found qSOFA to be of low sensitivity as a sepsis screening tool[7]. Our study also confirmed that the sensitivity of qSOFA was only 28.57%, the specificity was 79.31%, and the AUC was 0.638 (95%CI: 0.548-0.721). However, because qSOFA is a non-invasive and fast screening method, setting it as a preliminary screening is still feasible. Because infection and stress can cause neutropenia and lymphopenia, neutrophil lymphocyte count ratio (NLCR) has attracted more and more attention of clinical researchers[34]. Fuss et al[35] found that the NLCR level of sepsis in burn patients was significantly higher than in common burn patients. Relevant studies have also found that NLCR could be a simple and feasible indicator for the prediction of sepsis after percutaneous nephrolithotomy[36]. Our study showed that the diagnostic value of NLCR in diagnosing adult nonpulmonary infectious sepsis was lower than that of the INR.

Platelets play an important role in sepsis because they are at the intersection of the immune system, coagulation cascade, and endothelial cells[37,38]. Patients who had sustained thrombocytopenia or a drop in PLT of > 30% during their ICU stay had higher mortality[39]. Therefore, thrombocytopenia was considered to be an independent risk factor for death in patients with sepsis or septic shock[40]. However, as one of the diagnostic criteria for Sepsis-3, Sepas et al[41] found, when the platelet count was less than 229 × 109/L, the sensitivity and specificity of the diagnosis of acute appendicitis were only 24% and 75%, respectively. Similarly, our study also showed that when the platelet count was less than 180 × 109/L, the sensitivity and specificity of the diagnosis of sepsis were 80.0% and 82.76%, respectively, which were still lower than those of the INR. Furthermore, the base range of platelets was larger, and the sensitivity was lower, which could easily lead to delayed diagnosis of sepsis with delayed treatment and increased mortality of sepsis patients. Therefore, whether INR could replace platelet count as one of the indicators of SOFA score in sepsis criteria in the future still needs further studies with large sample size and multiple centers.

CONCLUSION

The INR has high specificity and sensitivity in the early identification of adult nonpulmonary infectious sepsis. When the INR value exceeds 1.22 in patients with non-pulmonary infection, sepsis should be highly suspected, especially for those patients without preexisting underlying disease or medication history that affects coagulation function. Due to its low cost, rapid detection and easy interpretation of the results, it is particularly suitable for the preliminary screening of sepsis in emergency patients, outpatient patients and patients in economically backward areas.

ARTICLE HIGHLIGHTS
Research background

Currently, there is a lack of sepsis screening tools that can be widely used worldwide. Pulmonary sepsis can be of sufficient concern to physicians due to their noticeable symptoms, which usually rely less on screening tools.

Research motivation

To investigate the efficiency of the international normalized ratio (INR) for the early rapid recognition of adult nonpulmonary infectious sepsis.

Research objectives

A total of 108 sepsis patients and 106 nonsepsis patients were enrolled according to relevant inclusion and exclusion criteria.

Research methods

Commonly used clinical indicators, such as white blood cell, neutrophil count, lymphocyte count, neutrophil-lymphocyte count ratio (NLCR), platelets (PLT), prothrombin time, INR, activated partial thromboplastin time and quick Sequential “Sepsis-related” Organ Failure Assessment (qSOFA) scores, were recorded within 24 h after admission. The diagnostic performances of them were analyzed and compared through multivariate logistic regression analysis, Spearman correlation, and receiver operating characteristic curve analysis.

Research results

The level of the INR was significantly prolonged in the sepsis group. The INR had high diagnostic performance for sepsis, with an area under the curve value of 0.918 (95%CI: 0.857-0.959), when the preexisting diseases that significantly affect coagulation function were excluded. The diagnostic efficacy of the INR was more significant than that of NLCR, PLT and qSOFA (P < 0.05). Moreover, INR levels of 1.17, 1.20, and 1.22 could be used to delineate patients as low, medium or high risk for nonpulmonary infectious sepsis, respectively.

Research conclusions

The INR is a promising and easily available biomarker for diagnosis, and it can be used as one of the indicators for early screening of adult nonpulmonary infectious sepsis. When its value is higher than the optimal cutoff (1.22) value, high vigilance is required for adult nonpulmonary infectious sepsis.

Research perspectives

Due to its low cost, fast detection and easy interpretation, INR is suitable for the primary screening of sepsis for emergency patients and outpatients, particularly in low and middle-income countries. Sepsis is highly suspected when the INR value exceeds 1.22 in patients with non-pulmonary infection, especially for those patients without preexisting underlying disease or medication history that affects coagulation function.

ACKNOWLEDGEMENTS

We would like to thank Prof. Peng B, Department of Statistics, Chongqing Medical University, for his advice on the statistics in this study.

Footnotes

Manuscript source: Unsolicited manuscript

Corresponding Author's Membership in Professional Societies: Chongqing Medical Association Critical Care Medicine Branch, No. S20200630644148; and Chinese Anti-Cancer Association, No. M160F01924M.

Specialty type: Medicine, general and internal

Country/Territory of origin: China

Peer-review report’s scientific quality classification

Grade A (Excellent): A

Grade B (Very good): 0

Grade C (Good): C

Grade D (Fair): 0

Grade E (Poor): 0

P-Reviewer: Covino M, Rodrigues AT S-Editor: Wu YXJ L-Editor: Filipodia P-Editor: Ma YJ

References
1.  Horak J, Martinkova V, Radej J, Matejovič M. Back to basics: recognition of sepsis with new definition. J Clin Med. 2019;8.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 7]  [Cited by in F6Publishing: 7]  [Article Influence: 1.4]  [Reference Citation Analysis (0)]
2.  Hotchkiss RS, Monneret G, Payen D. Sepsis-induced immunosuppression: from cellular dysfunctions to immunotherapy. Nat Rev Immunol. 2013;13:862-874.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1350]  [Cited by in F6Publishing: 1567]  [Article Influence: 142.5]  [Reference Citation Analysis (0)]
3.  Delano MJ, Ward PA. Sepsis-induced immune dysfunction: can immune therapies reduce mortality? J Clin Invest. 2016;126:23-31.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 320]  [Cited by in F6Publishing: 388]  [Article Influence: 48.5]  [Reference Citation Analysis (0)]
4.  Reinhart K, Daniels R, Kissoon N, Machado FR, Schachter RD, Finfer S. Recognizing sepsis as a global health priority - a WHO resolution. N Engl J Med. 2017;377:414-417.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 598]  [Cited by in F6Publishing: 661]  [Article Influence: 94.4]  [Reference Citation Analysis (0)]
5.  Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, Bellomo R, Bernard GR, Chiche JD, Coopersmith CM, Hotchkiss RS, Levy MM, Marshall JC, Martin GS, Opal SM, Rubenfeld GD, van der Poll T, Vincent JL, Angus DC. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315:801-810.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 13771]  [Cited by in F6Publishing: 13580]  [Article Influence: 1697.5]  [Reference Citation Analysis (1)]
6.  Costa RT, Nassar AP Jr, Caruso P. Accuracy of SOFA, qSOFA, and SIRS scores for mortality in cancer patients admitted to an intensive care unit with suspected infection. J Crit Care. 2018;45:52-57.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 27]  [Cited by in F6Publishing: 27]  [Article Influence: 4.5]  [Reference Citation Analysis (0)]
7.  Askim Å, Moser F, Gustad LT, Stene H, Gundersen M, Åsvold BO, Dale J, Bjørnsen LP, Damås JK, Solligård E. Poor performance of quick-SOFA (qSOFA) score in predicting severe sepsis and mortality - a prospective study of patients admitted with infection to the emergency department. Scand J Trauma Resusc Emerg Med. 2017;25:56.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 133]  [Cited by in F6Publishing: 155]  [Article Influence: 22.1]  [Reference Citation Analysis (0)]
8.  Williams JM, Greenslade JH, McKenzie JV, Chu K, Brown AFT, Lipman J. Systemic inflammatory response syndrome, quick sequential organ function assessment, and organ dysfunction: insights from a prospective database of ed patients with infection. Chest. 2017;151:586-596.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 187]  [Cited by in F6Publishing: 160]  [Article Influence: 20.0]  [Reference Citation Analysis (0)]
9.  Szabo BG, Kiss R, Lenart KS, Marosi B, Vad E, Lakatos B, Ostorhazi E. Clinical and microbiological characteristics and outcomes of community-acquired sepsis among adults: a single center, 1-year retrospective observational cohort study from Hungary. BMC Infect Dis. 2019;19:584.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 5]  [Cited by in F6Publishing: 5]  [Article Influence: 1.0]  [Reference Citation Analysis (0)]
10.  Patel P, Walborn A, Rondina M, Fareed J, Hoppensteadt D. Markers of Inflammation and Infection in Sepsis and Disseminated Intravascular Coagulation. Clin Appl Thromb Hemost. 2019;25:1076029619843338.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 43]  [Cited by in F6Publishing: 40]  [Article Influence: 8.0]  [Reference Citation Analysis (0)]
11.  Levi M, Schultz MJ. What do sepsis-induced coagulation test result abnormalities mean to intensivists? Intensive Care Med. 2017;43:581-583.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 9]  [Cited by in F6Publishing: 12]  [Article Influence: 1.7]  [Reference Citation Analysis (0)]
12.  Lipinska-Gediga M. Coagulopathy in sepsis - a new look at an old problem. Anaesthesiol Intensive Ther. 2016;48:352-359.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 12]  [Cited by in F6Publishing: 13]  [Article Influence: 1.6]  [Reference Citation Analysis (0)]
13.  Sartelli M, Kluger Y, Ansaloni L, Hardcastle TC, Rello J, Watkins RR, Bassetti M, Giamarellou E, Coccolini F, Abu-Zidan FM, Adesunkanmi AK, Augustin G, Baiocchi GL, Bala M, Baraket O, Beltran MA, Jusoh AC, Demetrashvili Z, De Simone B, de Souza HP, Cui Y, Davies RJ, Dhingra S, Diaz JJ, Di Saverio S, Dogjani A, Elmangory MM, Enani MA, Ferrada P, Fraga GP, Frattima S, Ghnnam W, Gomes CA, Kanj SS, Karamarkovic A, Kenig J, Khamis F, Khokha V, Koike K, Kok KYY, Isik A, Labricciosa FM, Latifi R, Lee JG, Litvin A, Machain GM, Manzano-Nunez R, Major P, Marwah S, McFarlane M, Memish ZA, Mesina C, Moore EE, Moore FA, Naidoo N, Negoi I, Ofori-Asenso R, Olaoye I, Ordoñez CA, Ouadii M, Paolillo C, Picetti E, Pintar T, Ponce-de-Leon A, Pupelis G, Reis T, Sakakushev B, Kafil HS, Sato N, Shah JN, Siribumrungwong B, Talving P, Tranà C, Ulrych J, Yuan KC, Catena F. Raising concerns about the Sepsis-3 definitions. World J Emerg Surg. 2018;13:6.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 64]  [Cited by in F6Publishing: 63]  [Article Influence: 10.5]  [Reference Citation Analysis (0)]
14.  Davies GR, Lawrence M, Pillai S, Mills GM, Aubrey R, Thomas D, Williams R, Morris K, Evans PA. The effect of sepsis and septic shock on the viscoelastic properties of clot quality and mass using rotational thromboelastometry: A prospective observational study. J Crit Care. 2018;44:7-11.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 21]  [Cited by in F6Publishing: 20]  [Article Influence: 2.9]  [Reference Citation Analysis (0)]
15.  Korpallová B, Samoš M, Bolek T, Škorňová I, Kovář F, Kubisz P, Staško J, Mokáň M. Role of thromboelastography and rotational thromboelastometry in the management of cardiovascular diseases. Clin Appl Thromb Hemost. 2018;24:1199-1207.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 22]  [Cited by in F6Publishing: 24]  [Article Influence: 4.0]  [Reference Citation Analysis (0)]
16.  Stettler GR, Sumislawski JJ, Moore EE, Nunns GR, Kornblith LZ, Conroy AS, Callcut RA, Silliman CC, Banerjee A, Cohen MJ, Sauaia A. Citrated kaolin thrombelastography (TEG) thresholds for goal-directed therapy in injured patients receiving massive transfusion. J Trauma Acute Care Surg. 2018;85:734-740.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 19]  [Cited by in F6Publishing: 21]  [Article Influence: 4.2]  [Reference Citation Analysis (0)]
17.  Fantz CR. Confirming Point-of-care INR test results. JAMA. 2020;323:1190-1191.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1]  [Cited by in F6Publishing: 3]  [Article Influence: 0.8]  [Reference Citation Analysis (0)]
18.  Zboromyrska Y, Cilloniz C, Cobos-Trigueros N, Almela M, Hurtado JC, Vergara A, Mata C, Soriano A, Mensa J, Marco F, Vila J. Evaluation of the Magicplex Sepsis Real-Time Test for the Rapid Diagnosis of Bloodstream Infections in Adults. Front Cell Infect Microbiol. 2019;9:56.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 22]  [Cited by in F6Publishing: 20]  [Article Influence: 4.0]  [Reference Citation Analysis (0)]
19.  Dong L, Li H, Zhang S, Yang G. miR148 family members are putative biomarkers for sepsis. Mol Med Rep. 2019;19:5133-5141.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 7]  [Cited by in F6Publishing: 7]  [Article Influence: 1.4]  [Reference Citation Analysis (0)]
20.  Guo H, Tang L, Xu J, Lin C, Ling X, Lu C, Liu Z. MicroRNA-495 serves as a diagnostic biomarker in patients with sepsis and regulates sepsis-induced inflammation and cardiac dysfunction. Eur J Med Res. 2019;24:37.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 23]  [Cited by in F6Publishing: 23]  [Article Influence: 4.6]  [Reference Citation Analysis (0)]
21.  Gando S, Levi M, Toh CH. Disseminated intravascular coagulation. Nat Rev Dis Primers. 2016;2:16037.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 284]  [Cited by in F6Publishing: 303]  [Article Influence: 37.9]  [Reference Citation Analysis (0)]
22.  Levi M, van der Poll T. Coagulation and sepsis. Thromb Res. 2017;149:38-44.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 380]  [Cited by in F6Publishing: 550]  [Article Influence: 68.8]  [Reference Citation Analysis (0)]
23.  Zhang Y, Khalid S, Jiang L. Diagnostic and predictive performance of biomarkers in patients with sepsis in an intensive care unit. J Int Med Res. 2019;47:44-58.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 18]  [Cited by in F6Publishing: 22]  [Article Influence: 3.7]  [Reference Citation Analysis (0)]
24.  Mierzchała-Pasierb M, Lipińska-Gediga M. Sepsis diagnosis and monitoring - procalcitonin as standard, but what next? Anaesthesiol Intensive Ther. 2019;51:299-305.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 13]  [Cited by in F6Publishing: 14]  [Article Influence: 3.5]  [Reference Citation Analysis (0)]
25.  Leng Y, Chen C, Zhang Y, Luo C, Liu B. Ability of serum procalcitonin to distinguish focus of infection and pathogen types in patients with bloodstream infection. Ann Transl Med. 2019;7:135.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 12]  [Cited by in F6Publishing: 13]  [Article Influence: 2.6]  [Reference Citation Analysis (0)]
26.  Song J, Park DW, Moon S, Cho HJ, Park JH, Seok H, Choi WS. Diagnostic and prognostic value of interleukin-6, pentraxin 3, and procalcitonin levels among sepsis and septic shock patients: a prospective controlled study according to the Sepsis-3 definitions. BMC Infect Dis. 2019;19:968.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 81]  [Cited by in F6Publishing: 130]  [Article Influence: 26.0]  [Reference Citation Analysis (0)]
27.  Ruan L, Chen GY, Liu Z, Zhao Y, Xu GY, Li SF, Li CN, Chen LS, Tao Z. The combination of procalcitonin and C-reactive protein or presepsin alone improves the accuracy of diagnosis of neonatal sepsis: a meta-analysis and systematic review. Crit Care. 2018;22:316.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 65]  [Cited by in F6Publishing: 98]  [Article Influence: 16.3]  [Reference Citation Analysis (0)]
28.  Jekarl DW, Lee S, Kim M, Kim Y, Woo SH, Lee WJ. Procalcitonin as a prognostic marker for sepsis based on SEPSIS-3. J Clin Lab Anal. 2019;33:e22996.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 25]  [Cited by in F6Publishing: 28]  [Article Influence: 5.6]  [Reference Citation Analysis (0)]
29.  Wacker C, Prkno A, Brunkhorst FM, Schlattmann P. Procalcitonin as a diagnostic marker for sepsis: a systematic review and meta-analysis. Lancet Infect Dis. 2013;13:426-435.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 638]  [Cited by in F6Publishing: 693]  [Article Influence: 63.0]  [Reference Citation Analysis (0)]
30.  Lyons PG, Micek ST, Hampton N, Kollef MH. Sepsis-Associated Coagulopathy Severity Predicts Hospital Mortality. Crit Care Med. 2018;46:736-742.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 54]  [Cited by in F6Publishing: 80]  [Article Influence: 16.0]  [Reference Citation Analysis (0)]
31.  Zhang Y, Zheng L, Liu L, Zhao M, Xiao J, Zhao Q. Liver impairment in COVID-19 patients: A retrospective analysis of 115 cases from a single centre in Wuhan city, China. Liver Int. 2020;40:2095-2103.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 293]  [Cited by in F6Publishing: 317]  [Article Influence: 79.3]  [Reference Citation Analysis (0)]
32.  Jin X, Duan Y, Bao T, Gu J, Chen Y, Li Y, Mao S, Xie W. The values of coagulation function in COVID-19 patients. PLoS One. 2020;15:e0241329.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 38]  [Cited by in F6Publishing: 46]  [Article Influence: 11.5]  [Reference Citation Analysis (0)]
33.  Marietta M, Vandelli P, Mighali P, Vicini R, Coluccio V, D'Amico R;  COVID-19 HD Study Group. Randomised controlled trial comparing efficacy and safety of high vs low Low-Molecular Weight Heparin dosages in hospitalized patients with severe COVID-19 pneumonia and coagulopathy not requiring invasive mechanical ventilation (COVID-19 HD): a structured summary of a study protocol. Trials. 2020;21:574.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 29]  [Cited by in F6Publishing: 31]  [Article Influence: 7.8]  [Reference Citation Analysis (0)]
34.  Westerdijk K, Simons KS, Zegers M, Wever PC, Pickkers P, de Jager CPC. The value of the neutrophil-lymphocyte count ratio in the diagnosis of sepsis in patients admitted to the Intensive Care Unit: A retrospective cohort study. PLoS One. 2019;14:e0212861.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 19]  [Cited by in F6Publishing: 23]  [Article Influence: 4.6]  [Reference Citation Analysis (0)]
35.  Fuss J, Voloboyeva A, Poliovyj V. Prognostic value of using neutrophil-lymphocyte ratio in patients with burn injury for the diagnosis of sepsis and bacteraemia. Pol Przegl Chir. 2018;90:13-16.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 4]  [Cited by in F6Publishing: 6]  [Article Influence: 1.2]  [Reference Citation Analysis (0)]
36.  Sen V, Bozkurt IH, Aydogdu O, Yonguc T, Yarimoglu S, Sen P, Koras O, Degirmenci T. Significance of preoperative neutrophil-lymphocyte count ratio on predicting postoperative sepsis after percutaneous nephrolithotomy. Kaohsiung J Med Sci. 2016;32:507-513.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 24]  [Cited by in F6Publishing: 26]  [Article Influence: 3.3]  [Reference Citation Analysis (0)]
37.  Middleton E, Rondina MT. Platelets in infectious disease. Hematology Am Soc Hematol Educ Program. 2016;2016:256-261.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 17]  [Cited by in F6Publishing: 16]  [Article Influence: 2.0]  [Reference Citation Analysis (0)]
38.  Greco E, Lupia E, Bosco O, Vizio B, Montrucchio G. Platelets and Multi-Organ Failure in Sepsis. Int J Mol Sci. 2017;18.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 64]  [Cited by in F6Publishing: 98]  [Article Influence: 14.0]  [Reference Citation Analysis (0)]
39.  Zhou Z, Feng T, Xie Y, Huang P, Xie H, Tian R, Qian B, Wang R. The effect of recombinant human thrombopoietin (rhTPO) on sepsis patients with acute severe thrombocytopenia: a study protocol for a multicentre randomised controlled trial (RESCUE trial). BMC Infect Dis. 2019;19:780.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 13]  [Cited by in F6Publishing: 13]  [Article Influence: 2.6]  [Reference Citation Analysis (0)]
40.  Vardon Bounes F, Mémier V, Marcaud M, Jacquemin A, Hamzeh-Cognasse H, Garcia C, Series J, Sié P, Minville V, Gratacap MP, Payrastre B. Platelet activation and prothrombotic properties in a mouse model of peritoneal sepsis. Sci Rep. 2018;8:13536.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 33]  [Cited by in F6Publishing: 24]  [Article Influence: 4.0]  [Reference Citation Analysis (0)]
41.  Sepas HN, Negahi A, Mousavie SH, Nasiri M. Evaluation of the Potential Association of Platelet Levels, Mean Platelet Volume and Platelet Distribution Width with Acute Appendicitis. Open Access Maced J Med Sci. 2019;7:2271-2276.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 5]  [Cited by in F6Publishing: 5]  [Article Influence: 1.0]  [Reference Citation Analysis (0)]