Introduction

In late December of 2019, a cluster of patients was diagnosed with a strange viral pneumonia in Wuhan City, Hubei Province, China, which later was confirmed to be caused by the novel coronavirus (the disease named COVID-19)1. Up to now, millions of cases have been identified, causing thousands of deaths at an alarming pace worldwide. Officially, the World Health Organization has declared the pandemic of COVID-192 and due to the non-existence of effective antiviral drug or vaccine, both detecting patient at an early stage and immediate patient isolation play a mandatory role in the fighting against COVID-193.

The chest computed tomography (CT) scan plays a central role on the disease staging and checking the treatment efficacy, while the reverse transcription-polymerase chain reaction (RT-PCR) remains the mainstay of COVID-19 diagnosis4,5, though limited to identify the virus, which poses important restrictions6.

Recent studies claim that initial chest CT may enable the detection of the disease with higher sensitivity in comparison to RT-PCR7. This systematic review and meta-analysis were performed to determine the diagnostic accuracy of the initial chest CT scan compared to RT-PCR in COVID-19 patients.

Materials and methods

All stages of this study followed the PRISMA guidelines and all relevant English, Chinese, and other language case-series, cross-sectional, and cohort studies were selected and checked for scientific validity.

Inclusion criteria: observational epidemiological study design, clear report of the number of positive cases by PCR and chest CT, and the ability to calculate accuracy indicators.

Exclusion criteria: case reports or not meeting one or more inclusion criteria.

Search strategy

All relevant literature from three main databases: MEDLINE (PubMed), Scopus, and EMBASE were explored from January 1st, 2019, to the 21th May 2020, using the keywords “COVID19 virus”, “2019 novel coronavirus”, “Wuhan coronavirus”, “2019-nCoV”, “X-Ray Computed Tomography”, “Polymerase Chain Reaction”, “Reverse Transcriptase PCR”, and “PCR Reverse Transcriptase” (Supplementary file 1). The references of the selected articles were also reviewed.

The variables extracted included the first author name, publication year, country and city of the study, subjects average age, gender, study design, total sample size, true positives, true negatives, false positives, and false negatives.

Data extraction and statistical analysis

Two researchers (SZA and SSTZ) screened articles separately by checking titles and abstracts. Disagreements were solved by a third one (FKH). Included articles had data of confirmed COVID-19 patients by chest CT scan and quantitative real-time polymerase chain reaction (RT-PCR) and were accessed in full text. The quality assessment was performed by the Newcastle–Ottawa Scale (NOS) assessment tool. The papers that receive scores more than 6 were reflected as the “high quality” and underwent additional meta-analysis steps.

The outcomes of interest, including the CT-scan to identify COVID-19 were submitted to summary receiver operating characteristic (SROC) curve by the random effect model for sensitivity and specificity.

Sensitivity, indicating the capacity of index test to identify patients, considered by “Sensitivity = TP/(TP + FN)”. Specificity as the examination to remove disease-free, calculated by “Specificity = TN/(FP + TN)”. The Metaprop command to calculate sensitivity and specificity excluded studies that have reported 100% sensitivity or specificity.

Positive Predictive Value (PPV) is the probability of disease if the test is positive calculated by "Positive predictive Value = TP/(FP + TP)". Negative Predictive Value (NPV) is the probability of disease-free if the test is negative calculated by "negative predictive value = TN/(FN + TN)".

The Cochran's Q-test of heterogeneity at 5% was used to evaluate statistical heterogeneity based on the Higgins classification in which an I2 > 75% means significant heterogeneity.

Deeks’ funnel plot was used to evaluate publication bias by the “Metafunnel”. Briefly, to create the funnel plot, the odds ratio were first calculated using the equations of (TP/FN)/(FP/TN) and after estimating the odds ratio logarithm, the standard error of odds ratio was calculated. Extracted data were collected in Excel 2007 (Microsoft Corporation, Redmond, CA) and analysis was done by using STATA v.14.0SE (College Station, TX, USA) and RevMan 5.

Results

Among 1022 identified articles, 115 were considered eligible and after the NOS screening, 60 articles, including 5744 subjects were included, all published in the first quarter of 2020 (Fig. 1).

Figure 1
figure 1

The number of articles during several steps based on the PRISMA flow diagram (2009). (http://www.prisma-statement.org/PRISMAStatement/CitingAndUsingPRISMA.aspx).

Full size image

A summary of the information of included articles is shown in Table 1. The forest plot, False Positives (FP), False Negatives (FN), True Positives (TP), True Negatives (TN), Sensitivity, Specificity, and 95% Confidence Intervals (CI) of each study are shown in Fig. 2 .

Table 1 Data of 60 included studies in the data extraction step.
Full size table
Figure 2
figure 2

Sensitivity and specificity of 60 included studies.

Full size image

Sensitivity ranged from 25 to 100% and the specificity was estimated to vary from 19 to 70%.

After excluding studies reporting 100% sensitivity or specificity, the sensitivity was ranging from 25 to 97% and specificity from 25 to 70% (Figs. 3, 4). Based on 37 studies, the sensitivity of CT compared to PCR was 87% (95% CI 85–90%) and based on seven studies the specificity of CT was 46% (95% CI 29–63%).

Figure 3
figure 3

Summary of sensitivity and 95% CI, generated by the STATA.

Full size image
Figure 4
figure 4

Summary of specificity and 95% CI, generated by the STATA.

Full size image

The positive predictive value of CT was 69% (95% CI 56–72%) and the negative predictive value was 89% (95% CI 82–96%) and the variation of the estimated numbers were 33% to 97% and 33% to 96% for PPV and NPV, respectively (Figs. 5, 6).

Figure 5
figure 5

Summary of positive predictive value and 95% CI of 18 studies, generated by the STATA.

Full size image
Figure 6
figure 6

Summary of negative predictive value and 95% CI of six studies, generated by the STATA.

Full size image

The symmetry between the two sides of the funnel plot regression line indicates that the included publications are not biased. However, due to a large number of zeros in the FP and TN cells, it was possible to calculate the odds ratio for six studies only, and the interpretation of this plot in our study should be done with caution (Fig. 7).

Figure 7
figure 7

The Deeks’ funnel plot curve for assessment of publication bias.

Full size image

Table 2 includes 35 studies with the first RT-PCR test of the suspected patients available (subsequent RT-PCR data were included if available). Moreover, the swabs should have been taken from sputum, nasopharyngeal, oropharyngeal, nose, or throat and if a combination was used, nasopharyngeal or throat swab was considered as the primary.

Table 2 The number of positive test results in RT-PCR testing and the number of confirmed ones.
Full size table

The COVID-19 diagnosis was confirmed by positive result of the first, second, third, and fourth RT-PCR tests and also information of patients who had negative results until the fourth test or no more than one test conducted, but considered as confirmed or most likely ill later according to more RT-PCR tests or examining other swabs, clinical manifestations, typical chest CT scan's features or developmental changes in the series of CT scans or a mixture of prior methods.

In the articles with more than 10 total confirmed patients (first 10 articles included), the RT-PCR test could diagnose 58.9% of the COVID-19 infected patients in the first test, and about 41.1% of infected patients could not be recognized in the first place by RT-PCR test. Among these 10 articles, 5 included the information of second tests (number 1, 2, 3, 7 and 9). In these five articles, the mean percentage of secondary diagnosed patients divided by total confirmed patients is 18.6%. Out of 4 articles (number 1, 2, 3 and 7) with the exact data of thirdly and fourthly conducted tests, the mean percentage of positivity are 6.3% and 0.7%, respectively. Moreover, the percentage of patients who were not diagnosed after 4 times of repeating the test is 8.6% (in the previous 4 articles). The numbers and sequences of primers and probes could be influential on PCR sensitivity and specificity which were surveyed in Table 3.

Table 3 From the information illustrated in Table 2, those which their primer and probe's data were available.
Full size table

As we can see in Table 3, in the case report by Wendong Hao44, using one pair of primer and probe did not indicate a positive result at first but in the fourth repeated test. In another case report by Feng Ly65, the oropharyngeal swab by detection of N gene showed a positive result in the third, fourth, and fifth time, whereas ORF1ab detection showed a positive result in the fifth examination. On the other hand, Xavier Marchand-Senécal et al.67 and Yosuke Hirotsu et al.61 reported 2 cases that were diagnosed initially with one pair of primer and probe of PCR test.

By using 2 pairs of primer and probe, the mean of initially detecting patients divided by total confirmed patients is 51.3% in the 4 studies above. Also, Qing Chen52 findings with the utilization of 3 pairs of primer and probe caused 100% of initially discovering COVID-19 patients52. Moreover, using 7 sets of primers and probes also resulted positively for the first test61. Based on limited data available, it seems that the greater the number of primer and probe, more likely to initially detect patients, although more specific information is needed from future studies.

Discussion

Considering the outcomes of RT-PCR as a reference, in our meta-analysis, the sensitivity and specificity of initial chest CT scan for detecting patients, who were highly suspicious for COVID-19, were 87% and 43% respectively. The PPV and NPV of CT scans were 67% and 84% respectively.

It means that 67% of individuals with positive chest CT scans had positive RT-PCR and 84% of individuals with negative chest CT scans had negative RT-PCR. So, a chest CT scan may have beneficial diagnostic features as adjuvant diagnostic tool compared to RT-PCR36,68.

Tao Ai and colleagues studying 1014 patients, 888 (88%) with a positive chest CT scan and 601 with a positive RT-PCR for COVID-19, described 97%, 25%, 65%, and 83% of sensitivity, specificity, PPV, and NPV for the CT scan, respectively. The relatively high sensitivity and low specificity in this study might be related to the low odds ratio of positive RT-PCR, considered as the reference test69, as suggested by the World Health Organization (WHO)70.

Some patients have typical chest CT scan findings and symptoms for COVID-19 but their initial RT-PCR results were negative agreeing with previous research reports19,36,44. Fang et al. described 15 out of 51 patients who have an initial negative RT-PCR while their chest CT scan was positive36,44, so it is very important to pay attention to chest CT scan, epidemiologic features, and clinical symptoms. Furthermore, a combination of humoral (IgG-IgM antibody) and cellular immunity, in addition to RT-PCR could refine the detection of COVID-1923,49.

The results of Chan and colleagues indicated that among 273 specimens (15 COVID-19 positives), the RdRp-P2 test showed 77 positive specimens and the RdRp/Hel test showed 42 positives. Moreover, RdRp/Hel analysis did not cross-reacted with any human coronaviruses or other respiratory pathogens while RdRp-P2 analysis reacted to SARS-CoV either71. Another study expressed that the sensitivity of N gene assay in finding the positive samples is 10 times higher than the ORF-1b gene assay72.

In February, 280 suspected patients with clinical manifestations of COVID-19 were tested in the Marseille hospital. None of the patients were positive for SARS-COV-273. Guo-Qing Qian et al. reported that all the patients, except three of them, were confirmed with the second RT-PCR test18. In the study by Tao Ai et al., it is highlighted that out of 1014 patients, 308 patients with negative PCR results, were strongly perceived as infected by clinical manifestations and CT scans. The percentage of a positive test for the first, second, third, and fourth tests were 37.7%, 22.6%, 5%, and 0.9%, respectively. Unfortunately, about 33.9% of the patients could not be diagnosed even with the fourth test. CT scans were positive in 580 of 601 (97%) COVID-19 confirmed patients by the RT-PCR test and in 308 of 413(74.6%) patients with negative RT-PCR assay74. In a retrospective analysis, among 51 patients, 98% (50/51) had abnormal CT, besides 70.6% (36/51) had positive PCR assay initially and about 30% of the tests became positive after second, third, and fourth scanning44.

A study of five children suggested that four children had positive PCR outcomes within the first assay, but one with COVID-19 suggestive CT findings turned positive after six times of examining35. It is also possible that negative RT-PCR with three times of repeating, turns positive on the fourth test, while CT demonstrated typical features such as GGO44. Chest CT presented rapidly developing multiple patchy consolidations and GGOs in both lungs of a case reported by J Wei, while in the later stage, there was the development of fibrosis. So with high-resolution CT, it will be easier to find GGOs in the early stage75.

What stands out from Table 2 is that 58.9% of the infected COVID-19 patients could be recognized in the first test and about 18.6%, 6.3% and 0.7% could be diagnosed in the further second, third and fourth tests, respectively. Besides, about 9% of the infected patients have not been detected, even after the fourth test. According to the results and due to the PCR's cost and time consumption, it seems that repeating the test up to 3 times is reasonable in patients with initially negative results (with 24 h to 3 days' time interval based on literature). Also, CT scan findings and clinical manifestation should be encountered in all patients, especially in suspected ones with multiple negative PCRs.

About 80% of COVID-19 patients have mild disease and just about 15% of them will reach severe stages. Positive RT-PCR results usually have a high positive predictive value, but negative RT-PCR should be repeated three times to increase the negative predictive value up to 98% (57% at first test, 34% at second one, and 7% at third time).

If the patient's death is due to COVID-19, but their PCR is negative, even if their chest CT is positive, their cause of death would not report COVID-19. Some patients with negative PCR result die, but based on our results, 87% of them are Covid-19 positive and their disease should be confirmed by repeating PCR for up to three times.

On the other hand, the exact place of chest CT is for staging the COVID-19 disease as mild, moderate, and severe, instead of being a screening tool. Some antibody and serology testing can support the RT-PCR test. A study in two patients with COVID-19 pneumonia by Lin and colleagues indicated the presence of ground-glass lesions and patchy consolidations in repeated chest CT76. Also, the lesions were classically accompanied by bronchial bundles or subpleural lesions. In patients who have a fever but not having the previous contact with the epidemic area, the appropriate finding of the COVID-19 RNA is compulsory to guarantee the high efficacy of treatment76.

We acknowledge that our study had some limitations: (1) the specificity of CT scan was not as reliable as the sensitivity, due to the majority of studies' nature, which were case-series and the number of true negative patients in those studies were zero. (2) It has been postulated in Bernheim et al.'s study that the chance of detecting lung involvement in chest CT scan will be increased if the duration between symptom onset and initial chest CT scan rises and this duration was different among 60 studies.

In conclusion, the results of the present systematic review and meta analysis shed new light on the comparison between chest-CT scan and rRT-PCR validity in terms of diagnosis in patients with COVID-19. Due to lower diagnostic sensitivity of chest-CT scan in comparison to rRT-PCR, performing rRT-PCR is mandatory for any individuals with suspicious symptoms. Nevertheless, the initial negative rRT-PCR result is not fully able to roll out COVID-19 in all cases and because of that, repeating the test for three times is vital to roll out COVID-19.