Field evaluation of COVID-19 rapid antigen test: Are rapid antigen tests less reliable among the elderly?

Abstract

BACKGROUND

The global outbreak of coronavirus disease 2019 (COVID-19) leads to the development of accessible and cost-effective rapid antigen-detection tests (RATs), as quick and accurate diagnosis is crucial to curb the pandemic.

AIM

To evaluate the Humasis COVID-19 Ag Test (Humasis Co., Ltd., Gyeonggi-do, Republic of Korea) in the diagnosis of severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2).

METHODS

This retrospective study was carried out at the Croatian Institute of Public Health and included patients with clinical symptoms of COVID-19 lasting no longer than 5 d prior to testing, whose nasopharyngeal swabs were primarily tested with RAT. Negative RAT samples underwent confirmatory real-time reverse transcription-polymerase chain reaction (RT-PCR). Diagnostic efficacy was determined compared to RT-PCR. The patients were divided into three age groups (< 18, 19-65, > 65 years). Statistical analysis was performed with the significance level set at P < 0.05.

RESULTS

In total, 2490 symptomatic patients were tested; 953 samples were positive on RAT, and 1537 were negative. All negative RAT samples were subjected to RT-PCR; 266 samples were positive and marked as false-negative results on RAT. The calculated negative predictive value as a measure of RAT efficacy was 82.69%. The χ² test and Kruskal-Wallis test showed a significant difference in the proportion of false negatives (P < 0.001) and RT-PCR cycle (Ct) values for false-negative RATs (P = 0.012) among the age groups. The young age group was significantly less likely to be false negative, whereas the false negatives from the elderly group experienced significantly lower Ct values than the other two age groups.

CONCLUSION

Evaluated RAT demonstrated satisfactory performance with more reliable results in younger patients. Humasis COVID-19 Ag RAT is potentially a valuable tool in areas where access to molecular methods is limited; however, RT-PCR remains a gold standard for SARS-CoV-2 detection.

Key Words: Coronavirus disease 2019; Severe acute respiratory syndrome coronavirus-2; Rapid antigen test; Real-time reverse transcription-polymerase chain reaction; Croatia

Core Tip: The global outbreak of coronavirus disease 2019 led to the development of severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) rapid antigen-detection tests (RATs), as a fast and accurate diagnosis is crucial to curb the pandemic. Evaluated RAT demonstrated satisfactory performance with more reliable results in younger patients. The young age group was significantly less likely to be false negative, whereas the false negatives from the elderly group showed significantly lower reverse transcription-polymerase chain reaction (RT-PCR) cycle values. Therefore, RT-PCR remains a gold standard for SARS-CoV-2 diagnosis.

INTRODUCTION

Coronavirus disease 2019 (COVID-19) caused by severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) originated in Wuhan, Hubei province, China, in December 2019 and was declared a pandemic by the World Health Organization (WHO) on March 11, 2020. One of the strategic objectives in response to the pandemic was the early identification of infected individuals[1]. According to the WHO interim guidance published on March 2, 2020, nucleic acid amplification tests such as quantitative reverse transcription-polymerase chain reaction (RT-PCR) are considered a gold standard for diagnostic detection of SARS-CoV-2[2]. RT-PCR takes several hours to detect the SARS-CoV-2 RNA. Furthermore, it requires trained staff and is a high-priced molecular diagnostic tool. Bearing all this in mind, RT-PCR as a diagnostic tool has been challenging worldwide. In addition, highly sensitive and specific tests are consequential in identifying infected individuals and subsequently implementing control measures to limit the outbreak.

In contrast to RT-PCR, antigen-detection rapid diagnostic tests (RATs) are less expensive, easier to use, and provide results more rapidly. These methods are designed to directly detect SARS-CoV-2 proteins produced during the viral replication in respiratory secretions. RAT testing should be conducted by trained personnel within the first 5-7 d following the onset of symptoms[3]. Since the molecular capacity is difficult to scale-up and there was a need to expand the COVID-19 testing capacity due to the increasing numbers of cases at the end of 2020, Croatia implemented the RAT as a part of the national strategy for handling the SARS-CoV-2 pandemic[4].

This study evaluated the performance of the first RAT used at the Croatian Institute of Public Health (CIPH) during the second pandemic wave.

MATERIALS AND METHODS

The evaluation period included samples from November 11, 2020 to January 23, 2021. According to the national strategy protocol, all symptomatic patients who self-reported at least one characteristic COVID-19 symptom no longer than 5 d prior to testing were subjected to RAT. Self-reported symptoms were concisely revised by a medical doctor, after which a nasopharyngeal swab was taken and placed into the viral transport medium. Then the swabs were transported to the laboratory and tested within 60 min from sampling using the Humasis COVID-19 Ag Test Kit (Humasis Co., Ltd., Gyeonggi-do, Republic of Korea), an in vitro diagnostic test based on an immunochromatographic assay designed for the qualitative detection of SARS-CoV-2 nucleocapsid and receptor binding domain antigens according to the manufacturer’s instructions. According to the internal protocol, positive RAT (containing both test and control lines) was considered SARS-CoV-2-positive and was not further tested. Testing was repeated if the rapid antigen test was invalid (containing no control line). Negative tests (containing only the control line) were subjected to confirmatory RT-PCR testing. RT-PCR testing was performed from the same nasopharyngeal swab. Viral RNA was extracted using the automated extraction system Nextractor-NX-48 Genolution. Real-time RT-PCR was performed on the Applied Biosystem Instrument QuantStudio5 using the RT-PCR assay (GeneFinderTM COVID-19 Plus RealAmp Kit; Osang Healthcare Co Ltd, Republic of Korea) according to the manufacturer’s instructions. Basic demographic analysis and descriptive statistic tests were performed. C², Mann-Whitney U, and Kruskal-Wallis tests were performed using an online tool (https://www.socscistatistics.com/tests) on the data concerning RT-PCR results and average RT-PCR cycle for the three age groups (< 18, 19-65, > 65 years). Weekly positive, negative, and false-negative RAT were correlated with average Ct values for false-negative antigen tests and 7 d average COVID-19 incidence in Croatia. P < 0.05 was considered statistically significant.

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of the Croatian Institute of Public Health (Protocol No. 030-02/20-05/1, Approved on May 7, 2020).

RESULTS

During the evaluation period, 2490 RATs were performed on nasopharyngeal swabs obtained from patients with a positive history of SARS-CoV-2 infection in the last 5 d prior to sampling. The largest number of patients tested was 545 in the week from November 11, 2020 to November 14, 2020. We observed a steady decline in symptomatic patients during the study period (Figure 1).

Figure 1 Weekly number of symptomatic patients tested by rapid antigen-detection tests.

Of the 2490 performed antigen detection tests on symptomatic patients, 953 patients tested positive, yielding an overall positive RAT prevalence of 37.90%, while 1537 patients tested negative and were subjected to confirmatory RT-PCR test according to WHO and national CIPH guidelines. The dynamics of the weekly ratio of positive results in the observed period are shown in Figure 2. The highest prevalence of 44.83% was seen in the 1^st wk of 2021, while the lowest prevalence of 22.62% was observed the week prior. Among the 1537 samples that underwent both RAT and RT-PCR tests, RT-PCR confirmed 1271 patients as true negatives, while 266 patients were RT-PCR positive. Those patients were classified as false negatives on the antigen detection test, and the calculated negative predictive value of the RAT was 82.69%. The highest ratio of false-negative results (28.57%, from 196 negative RATs) was observed in the 49^th wk of 2020, whereas the lowest ratio (2.86%) was observed in the 2^nd wk of 2021 (Figure 3). False-negative patients were summarized according to the RT-PCR cycle in which they were identified as positive, as shown in Figure 4. The Shapiro-Wilk test of normality revealed a not-normal, asymmetrical distribution with a long left tail. Sixty-five percent of them had a lower viral load (Ct ≥ 25). The lowest Ct value observed was 18, but for only 5.62% of false-negative patients, the Ct value was ≤ 20.

Figure 2 Ratio of positive rapid antigen-detection tests results by week. RAT: Rapid antigen.

Figure 3 Ratio of false-negative rapid antigen-detection tests results by week.

Figure 4 Number of rapid antigen-detection tests false-negative patients testing positive per each reverse transcription-polymerase chain reaction cycle. Ct: Reverse transcription-polymerase chain reaction cycle.

Spearman’s correlation coefficient was calculated to determine the strength of correlation between the total number of tested patients and false negatives and the possible correlation between false negatives and Ct values. The results showed a significant correlation between the total number of patients tested and false negatives (P < 0.05). There was also a significant correlation between the number of negatives and false negatives (P < 0.05), which showed that an increase in the sample size of tested patients and an increase in the number of negative results would increase the number of false-negative results. Meanwhile, Spearman's correlation coefficient did not show any significant correlation between Ct values and the prevalence of false-negative RAT results (P = 0.52) (Figure 5). This showed that during this time, an increase in the number of cycles in the confirmatory RT-PCR tests did not decrease the occurrence or prevalence of positive RT-PCR results.

Figure 5 Correlation between reverse transcription-polymerase chain reaction cycle values and prevalence of false-negative results.

Interestingly, Spearman’s coefficient of correlation between the weekly average of newly diagnosed in Croatia and the number or percentage of false-negative RATs also did not reach a significance level (P = 0.2) (Table 1). Graphical representation of the correlation between the total number of tested patients and the number of false-negative results, together with the corresponding calculated predictive model, showed that approximately 10% of false-negative results could be expected each time (Figure 6). We also analyzed if age causes any variation in the occurrence of negative and false-negative results and if there is any difference in mean Ct value with different age groups. Patients were subdivided into three age groups: < 18, 19-65, > 65. Statistical analysis with the χ² test showed a significant difference in the occurrence of false negatives between the youngest group (< 18 years of age) and the adult (19-65 years of age) or elderly group (> 65 years of age) (P < 0.0001), as well as the difference between the adults and elderly (P = 0.033). The overall difference between the three groups was significant (χ² test; P < 0.01).

Figure 6 Predictive model of correlation between the total number of patients and the number of false negatives.

Table 1 Percentage of false-negative rapid antigen-detection tests and reverse transcription-polymerase chain reaction cycle values by age groups.

Age group	N tested	Female, n (%)	Mean age in yr	RAT false negatives, n (%)	Ct mean	Ct median
0-18	292	139 (47.60%)	11.71	21 (7.19%)	26.58	27
19-65	1194	698 (58.46%)	38.62	230 (19.26%)	25.89	26
65+	51	24 (47.06%)	72.51	16 (31.37%)	23.86	24

Ct: Reverse transcription-polymerase chain reaction cycle; RAT: Rapid antigen-detection test.

Kruskal-Wallis test showed a significant statistical difference in mean Ct value among all three age groups (P = 0.012). Subsequently, Mann-Whitney U-tests revealed that the overall differences arose from the differences between the oldest group compared to any of the younger ones, whereas the difference between the 0-18 and 19-65 age groups did not reach a significance level.

DISCUSSION

Published studies have reported a wide range of data concerning RAT accuracy and consequently developed various opinions on their use in COVID-19 diagnostics. However, these data evaluated numerous different RAT; therefore, the results are not entirely comparable. Two studies evaluating Abbot's Panbio COVID-19 test reported negative predictive values of 99.5%[5] and 94.6%[6]. The evaluation of COVID-19 Ag Respi-Strip (Coris BioConcept) gave the results for negative predictive values (85.25%), similar to our results (82.69%)[7]. There are only a few studies that evaluated Humasis Covid-19 RAT. Klajmon et al[8] tested 189 samples using RT-PCR and RAT simultaneously. The calculated negative predictive value (97.22%) was higher than our study but more similar to Abbot's PanBio RAT (94.6% and 99.5%, respectively). Magyar et al[9] evaluated the performance of ten commercially available SARS-CoV-2 RAT, including the Humasis COVID-19 Ag test, compared to RT-PCR. The mean Ct value (24.1 ± 5.53) was similar to our findings (23.86-26.58) and overall negative predictive value (79.0%).

During the study period, declining numbers of tested and confirmed COVID-19 cases were observed, which can be attributed to the epidemiological situation in Croatia characterized with declining incidence and prevalence of COVID-19[4,10]. The main strength of this in-field evaluation is the sample size, whereas the limitations include various swab-takers and six RAT operators/interpreters; however, they were always competent and well-educated personnel (e.g., laboratory personnel or medical doctors). Furthermore, the lower negative predictive value could partially result from the declining incidence and prevalence of COVID-19 in Croatia during the studied period. The retrospective nature of this study can be seen as a disadvantage in terms of reproducibility and scientific accuracy. However, on the other hand, it depicts a realistic, day-to-day approach to COVID-19 diagnostics. National and European Centre for Disease Prevention guidelines were, however, strictly followed. According to WHO recommendations, RAT with a minimum of 80% sensitivity and 97% specificity was used in patients with symptoms consistent with COVID-19[11]. Since symptomatic persons may carry a higher risk of transmitting the infection, this study investigated only symptomatic people. Therefore, RAT-positive samples were not further tested; thus, the sensitivity, specificity, and positive predictive value of the rapid antigen test included in the study could not be determined. The negative predictive value of 83% was lower than those stated by the manufacturer (2 false-negative tests of 22, giving a negative predictive value of 91%). However, the patients self-reported the symptoms and their durations, and although reviewed by a doctor before sampling, this could cause a decrease in the negative predictive value.

To the best of our knowledge, this study is the first to address the age groups differences, finding out significant differences concerning the false-negative RAT rates and the Ct values of these specimens. The youngest age group was significantly less likely to be false negative, whereas the oldest age group's Ct values were significantly lower than those of the remaining two groups. Although due to the difference in the number of tested patients between the age groups, further research is needed. Based on our results, we recommend that the elderly patients could preferably test by RT-PCR test due to their increased vulnerability towards COVID-19 infection and possibly lower performance of rapid antigen tests in that age group.

The national recommendation that negative RAT results in symptomatic patients suggestive of COVID-19 should be further tested by RT-PCR seems like a good solution because negative RAT results do not rule out the infection, especially in the high prevalence settings during study period[4]. Identifying and confirming COVID-19 in symptomatic persons is important to provide appropriate care and implement public health measures without delay[11]. Therefore, despite the limitations of RAT, including inferior results to RT-PCR[8, 12], their previously described use is effective and can be a support for RT-PCR testing of SARS-CoV-2. Nevertheless, the RAT cost is significantly lower than RT-PCR[13].

CONCLUSION

In conclusion, RAT is an acceptable tool in COVID-19 diagnostics that saves time, human and financial resources and has reasonably accurate results if the viral load is high (Ct < 25), but caution is needed. In order to prevent viral spread, in the settings when it is feasible, all RAT negative samples should be subjected to RT-PCR analysis.

ARTICLE HIGHLIGHTS

Research background

One of the strategic objectives in response to the coronavirus disease 2019 (COVID-19) pandemic was the early identification of infected individuals. Compared to the reverse transcription-polymerase chain reaction (RT-PCR), antigen-detection rapid diagnostic tests (RATs) are less expensive, easier to use, and provide results faster.

Research motivation

The sensitivity and specificity of different severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) RAT vary.

Research objectives

To evaluate the value of Humasis COVID-19 Ag Test in the diagnosis of SARS-CoV-2 infection.

Research methods

A commercial immunochromatographic assay for the qualitative detection of SARS-CoV-2 nucleocapsid and receptor binding domain antigens (Humasis COVID-19 Ag Test Kit; Humasis Co., Ltd., Gyeonggi-do, Republic of Korea) was used to detect SARS-CoV-2 antigen in the nasopharyngeal swab samples. Negative samples were subjected to confirmatory RT-PCR testing.

Research results

A total of 2490 RAT were performed on nasopharyngeal swabs obtained from patients with a history of SARS-CoV-2 infection in the last 5 d prior to sampling. The overall positive RAT prevalence was 37.90% (953 positive samples) while the calculated negative RAT predictive value was 82.69%. A significant difference in the prevalence of false negatives was observed (7.19%, 19.26% and 31.37%, respectively) between the youngest group (< 18 years of age) and the adult (19-65 years of age) or elderly group (> 65 years of age) (P < 0.0001). In addition, a significant difference in mean Ct value between all three age groups was found (26.58, 25.89, and 23.86, respectively; P = 0.012).

Research conclusions

RAT is an acceptable tool in COVID-19 diagnostics in samples with a high viral load (Ct < 25), but caution is needed. The results seem to be more reliable in younger patients.

Research perspectives

Further studies including different RAT are needed to confirm this observation.