A three-pronged approach includes a manufacturer claim of good performance, an independent government evaluation, and another study that indicates good performance, from a clinical laboratory in the field.
Clinical laboratories across the country face unprecedented challenges in responding to the increased demand for SARS-CoV-2 serology testing. This week, two leaders in the laboratory industry provided insights into critical factors clinical laboratory and pathology directors should consider in achieving high quality and accuracy of their SARS-CoV-2 serology tests.
James O. Westgard, PhD, and Sten A. Westgard are directors at Westgard QC, a firm that has specialized in clinical laboratory quality methods for nearly 30 years. On June 11, 2020, the Westgards presented Part 2 of an exclusive webinar program hosted by the COVID-19 STAT Intelligence Briefings Service and Dark Daily. This program focused on strategies for evaluating COVID-19 serology testing and selecting reliable serology tests.
Diving deeper into how clinical laboratory directors can ensure high-quality, accurate SARS-CoV-2 serology tests, Dr. Westgard presented the first half of the June 11 webinar. Helpful to understand, he said, is the smart testing algorithm developed by Michael T. Osterholm, PhD, MPH, of the University of Minnesota and Director of the Center for Infectious Disease Research and Policy (CIDRAP). “It starts with the right infrastructure, then it talks about population, then right test, followed by the right interpretation, and finally the right action,” explained Dr. Westgard.
Testing systems and test types were explained as well. “As a laboratory you need to be thinking about having a system of tests and methods.” Neutralization assays, lateral flow assays, and immunoassays were covered as the three main methods.
Additionally, Dr. Westgard said that while it is imperative to evaluate quantitative test performance, performance evaluation needs are growing. “One of the things that we have to start thinking about is qualitative versus quantitative testing,” he said. “We are used to quantitative testing, but qualitative testing has some differences.” Evaluating tests with a binary response requires a different approach and consideration that Dr. Westgard explored in depth during the 90-minute program, which is available on demand here.
Achieving High Confidence Levels in the Quality and Accuracy of Serological Tests: First Segment Points of Interest
- Neutralization assays, while not performed by most clinical laboratories, are an essential component for understanding the intended use of tests and the relevance of serology test results.
- In-depth analysis of the specific calculations and data is needed to thoroughly evaluate qualitative tests: Qualitative tests provide a hit rate that is related to standard deviation (SD) in quantitative testing.
- Precision can be evaluated from the probability distribution of a normal curve, when mean and SD cannot be calculated.
Sten Westgard, who has written and edited several books on clinical laboratory quality, presented the second half of the webinar. New data is becoming available on the sensitivity and specificity of certain tests, by independent sources, instead of from the manufacturers, he said.
“Now we have some data that is coming from labs that have been using these methods in the field. We’re not going to get a perfect method, where every study says that everything is wonderful,” Westgard explained. “It’s a little too early to expect that level of performance, but we do want to find a method where, not only does the manufacturer claim there’s good performance, but we also have an independent government evaluation and maybe another study from a lab in the field that indicates good performance.”
Making matters complicated is that attitudes are mixed toward serology testing, both in the US and around the world.
“I know for some labs, they’re still on the fence about using serology,” Westgard said. “Maybe more in the US; we’re being a little more reluctant, but a lot of places in the rest of the world, they’re full speed ahead with testing serology. Even if it’s only for understanding prevalence, we still need to know that. We need to understand how many people in our community have it and what those implications will eventually mean for immunity, for going back to work, etcetera.”
Achieving High Confidence Levels in Serology Test Quality and Accuracy: Second Segment Points of Interest
- Westgard QC has an online calculator to help clinical laboratories understand the accuracy of their test and to calculate clinical specificity and clinical sensitivity.
- While counterintuitive, lower prevalence makes more accurate testing necessary. This will be more of a consideration as case numbers decrease.
- Smaller sample sizes in validation and verification provide less precise data with a large confidence window. The smallest optimal number of samples for verification is 30 samples.
- Repeat testing for serology is becoming more important and more heavily emphasized. The FDA has a two-test comparator calculator, emphasizing the FDA’s stance on how important this is. Westgard QC also has a two-test comparator tool.
- While sensitivity is often considered important in clinical testing, specificity is more important in serology testing.
- More routine considerations, such as quality control and proficiency testing, are becoming prevalent as the SARS-CoV-2 testing begins to normalize.
To get the most from this educational webinar, please access the streaming program here, or by copying the following URL (https://covid19briefings.com/2020/05/24/achieving-high-confidence-levels-in-the-quality-and-accuracy-of-your-clinical-labs-chosen-covid-19-serology-tests/) into your browser.
—By Caleb Williams, Editor, COVID-19 STAT
