Westgard QC Webinar Part 2 Points of Interest: Achieving High Confidence Levels in the Quality and Accuracy of COVID-19 Serology Tests

Educational webinar hosted by COVID-19 STAT and Dark Daily on quality and accuracy of serological tests
Share on facebook
Facebook
Share on twitter
Twitter
Share on linkedin
LinkedIn
Share on pinterest
Pinterest
Share on whatsapp
WhatsApp
Share on email
Email
Share on facebook
Share on twitter
Share on linkedin
Share on google
Share on whatsapp
Share on email

A three-pronged approach includes a manufacturer claim of good performance, an independent government evaluation, and another study that indicates good performance, from a clinical laboratory in the field.

Clinical laboratories across the country face unprecedented challenges in responding to the increased demand for SARS-CoV-2 serology testing. This week, two leaders in the laboratory industry provided insights into critical factors clinical laboratory and pathology directors should consider in achieving high quality and accuracy of their SARS-CoV-2 serology tests.

James O. Westgard, PhD, and Sten A. Westgard are directors at Westgard QC, a firm that has specialized in clinical laboratory quality methods for nearly 30 years. On June 11, 2020, the Westgards presented Part 2 of an exclusive webinar program hosted by the COVID-19 STAT Intelligence Briefings Service and Dark Daily. This program focused on strategies for evaluating COVID-19 serology testing and selecting reliable serology tests. 

Diving deeper into how clinical laboratory directors can ensure high-quality, accurate SARS-CoV-2 serology tests, Dr. Westgard presented the first half of the June 11 webinar. Helpful to understand, he said, is the smart testing algorithm developed by Michael T. Osterholm, PhD, MPH, of the University of Minnesota and Director of the Center for Infectious Disease Research and Policy (CIDRAP). “It starts with the right infrastructure, then it talks about population, then right test, followed by the right interpretation, and finally the right action,” explained Dr. Westgard.

Dr. James Westgard featured in May 21, 2020, webinar on quality in selecting COVID-19 serological testing methods
James O. Westgard, PhD

Testing systems and test types were explained as well. “As a laboratory you need to be thinking about having a system of tests and methods.” Neutralization assays, lateral flow assays, and immunoassays were covered as the three main methods.

Additionally, Dr. Westgard said that while it is imperative to evaluate quantitative test performance, performance evaluation needs are growing. “One of the things that we have to start thinking about is qualitative versus quantitative testing,” he said. “We are used to quantitative testing, but qualitative testing has some differences.” Evaluating tests with a binary response requires a different approach and consideration that Dr. Westgard explored in depth during the 90-minute program, which is available on demand here.

Achieving High Confidence Levels in the Quality and Accuracy of Serological Tests: First Segment Points of Interest

  • Neutralization assays, while not performed by most clinical laboratories, are an essential component for understanding the intended use of tests and the relevance of serology test results.
  • In-depth analysis of the specific calculations and data is needed to thoroughly evaluate qualitative tests: Qualitative tests provide a hit rate that is related to standard deviation (SD) in quantitative testing.
  • Precision can be evaluated from the probability distribution of a normal curve, when mean and SD cannot be calculated.

Sten Westgard, who has written and edited several books on clinical laboratory quality, presented the second half of the webinar. New data is becoming available on the sensitivity and specificity of certain tests, by independent sources, instead of from the manufacturers, he said. 

Sten Westgard of Westgard QC explains the how-to of COVID-19 serology test quality
Sten Westgard

“Now we have some data that is coming from labs that have been using these methods in the field. We’re not going to get a perfect method, where every study says that everything is wonderful,” Westgard explained. “It’s a little too early to expect that level of performance, but we do want to find a method where, not only does the manufacturer claim there’s good performance, but we also have an independent government evaluation and maybe another study from a lab in the field that indicates good performance.”

Making matters complicated is that attitudes are mixed toward serology testing, both in the US and around the world. 

“I know for some labs, they’re still on the fence about using serology,” Westgard said. “Maybe more in the US; we’re being a little more reluctant, but a lot of places in the rest of the world, they’re full speed ahead with testing serology. Even if it’s only for understanding prevalence, we still need to know that. We need to understand how many people in our community have it and what those implications will eventually mean for immunity, for going back to work, etcetera.”

Achieving High Confidence Levels in Serology Test Quality and Accuracy: Second Segment Points of Interest

  • Westgard QC has an online calculator to help clinical laboratories understand the accuracy of their test and to calculate clinical specificity and clinical sensitivity.
  • While counterintuitive, lower prevalence makes more accurate testing necessary. This will be more of a consideration as case numbers decrease.
  • Smaller sample sizes in validation and verification provide less precise data with a large confidence window. The smallest optimal number of samples for verification is 30 samples.
  • Repeat testing for serology is becoming more important and more heavily emphasized. The FDA has a two-test comparator calculator, emphasizing the FDA’s stance on how important this is. Westgard QC also has a two-test comparator tool.
  • While sensitivity is often considered important in clinical testing, specificity is more important in serology testing.
  • More routine considerations, such as quality control and proficiency testing, are becoming prevalent as the SARS-CoV-2 testing begins to normalize.

To get the most from this educational webinar, please access the streaming program here, or by copying the following URL (https://covid19briefings.com/2020/05/24/achieving-high-confidence-levels-in-the-quality-and-accuracy-of-your-clinical-labs-chosen-covid-19-serology-tests/) into your browser.

Caleb Williams

—By Caleb Williams, Editor, COVID-19 STAT

Related Resources:

Achieving High Confidence Levels in the Quality and Accuracy of Your Clinical Lab’s Chosen COVID-19 Serology Tests, featuring James Westgard, PhD

The CIDRAP Viewpoint

Serology “Shortlist” for EUA SARS-COV-2 Testing

Westgard QC

COVID-19 TESTING DATA

Helping Your Lab Stay Afloat During Challenging Times

Dear Colleague,

Robert L. MichelIn these challenging times for our colleagues in clinical laboratories, pathology practices and hospital administration, The Dark Intelligence Group (TDIG) — through generous grants from our partners — is bringing you the COVID-19 STAT Intelligence Briefings.

We’ve added analysts, writers and skilled editors to the COVID-19 STAT team to uncover, analyze, and immediately report to you what’s happening with the SARS-CoV-2 pandemic, packaged with the insights you need to deliver quality testing for patient care and keep your lab’s financial ship afloat. As part of our mission, we are also bringing you real-time data from our partners around testing that you will not find anywhere else.

Please let me know how we can better improve this service to fit your needs. If you would like to offer a grant to keep our mission going with this site, please contact me at rmichel@darkreport.com.

Sincerely,

Robert L. Michel
President, Founder
The Dark Intelligence Group