New Meta-analysis Reveals Significant Weaknesses in Evidence Base for Point-of-Care SARS-CoV-2 Serology Tests

Meta-analysis shows that the evidence supporting point-of-care SARS-CoV-2 serology testing is non-existent.

A meta-analysis recently published in the BMJ indicates that the evidence for SARS-CoV-2 serology testing has significant weaknesses and that the evidence for point-of-care serology testing is particularly weak.

This study reviewed available literature for articles that included a combination of the terms COVID-19 (or SARS-CoV-2) and serological tests. The search included databases and preprint servers from between Jan. 1, 2020, and April 30, 2020. Forty studies were eligible within the test criteria. A majority of these studies (70%) were from China, and the rest were from Italy, the United States, Denmark, Spain, Sweden, Japan, the United Kingdom, and Germany.

Researchers looked for studies that measured the sensitivity and specificity of SARS-CoV-2 antibody tests compared to a control test to examine overall test performance. The researchers found that half the studies were not peer reviewed, and that most studies had a high or unclear risk of bias. Only two studies included data for point-of-care testing, both by using lateral flow immunoassays (LFIAs).

Researchers found that the pooled sensitivity results ranged from 66% to 97.8% and that pooled specificities ranged from 96.6% to 99.7%. The authors noted that pooled sensitivities were consistently lower for LFIA tests: “The strongest difference [was] seen for LFIAs, where the sensitivity of commercial kits was 65.0% (49.0% to 78.2%) and that of non-commercial tests was 88.2% (83.6% to 91.3%).

“The poor performance of existing serological tests for covid-19 raises questions about the utility of using such methods for medical decision making, particularly given time and effort required to do these tests and the challenging workloads many clinics are facing,” wrote the authors. “Our findings should also give pause to governments that are contemplating the use of serological tests—in particular, point-of-care tests—to issue immunity ‘certificates’ or ‘passports’.”

The authors provided an example based on their finding: “If an LFIA is applied to a population with a true SARS-CoV-2 prevalence of 10%, for every 1000 people tested, 31 who never had covid-19 will be incorrectly told they are immune, and 34 people who had covid-19 will be incorrectly told that they were never infected.” These staggering statistics lead the authors to conclude: “The evidence does not support the continued use of existing point-of-care serological tests for COVID-19.”

SARS-CoV-2 serology testing has become more prevalent as the global response to the COVID-19 pandemic continues to develop. This research, however, casts doubt on the evidence that SARS-CoV-2 serology testing yields useful data.

“In summary, we have found major weaknesses in the evidence base for serological tests for covid-19,” the authors of the BMJ article reported. “While the scientific community should be lauded for the pace at which novel serological tests have been developed, this review underscores the need for high quality clinical studies to evaluate these tools.”

Meta-Analysis Confirms Prior COVID-19 Serology Testing Concerns

Prior to the BMJ article, the American Medical Association (AMA) warned about the limitations of serology tests, including that they too often return false-positive results and that very little is known about immunity with the presence of COVID-19 antibodies, according to Jesse Ehrenfeld, MD, MPH, immediate Past Chair of the AMA Board of Trustees and Senior Associate Dean at that the Medical College of Wisconsin. Ehrenfeld spoke to legislators in early June about the risks of serology testing and was quoted on the AMA website.

With the performance of SARS-CoV-2 diagnostic testing questionable, the US Food and Drug Administration (FDA) announced its participation in the COVID-19 Diagnostics Evidence Accelerator. This is a project of the Reagan-Udall Foundation for the FDA and Friends of Cancer Research, and it aims to expand and intensify health data aggregation and analytics in the context of both diagnostic and clinical COVID-19 data.

FDA Hopes to Reassure with Diagnostics Evidence Accelerator Effort

According to the FDA, key research questions that will be examined through the Diagnostics Evidence Accelerator include evaluating the real-world performance of SARS-CoV-2 diagnostic tests and antibody tests, how they are best used to tailor testing strategies to specific populations, and what test results and data may reveal about the prevalence of disease, chains of transmission, and individual and population-level immunity.

“We recognize that questions related to the accuracy of diagnostic tests require different data elements that are not as widely available as those in the therapeutic space. They also require a different approach to data integration and analysis,” stated Jeffrey Shuren, MD, JD, Director of FDA’s Center for Devices and Radiological Health, as part of the FDA’s Accelerator announcement.

Through Interoperability and Portability of Laboratory Data

The Accelerator project will leverage FDA’s SHIELD initiative, a multi-stakeholder collaboration to improve the quality, interoperability and portability of laboratory data within and between institutions so that diagnostic information can be drawn from different sources or shared between institutions, according to a more recent update from Shuren. “SHIELD harmonizes COVID-19 test data referenced in the HHS COVID-19 laboratory data reporting requirements, which can be used to evaluate the real-world performance of SARS-CoV-2 diagnostic tests and antibody tests,” stated Shuren.

Clinical laboratory COVID-19 test data reporting plays a crucial role in understanding the performance of diagnostic test performance.

Careful selection of antibody tests and validation of these tests is necessary to ensure that clinical laboratories are providing patients and public health entities with useful and accurate data. Clinical laboratory leaders should be aware of the weakness in the literature supporting SARS-CoV-2 serology testing, especially in point-of-care testing.