Case Count Confusion: The Pitfalls of Public Health’s Oft-Used Metric

Kinsa was founded on a simple premise: if we know when and where illness will surge, we can take action to reduce transmission and ultimately save lives.

What could we do if we had days or even weeks advance notice of a surge in illness? We could send in test kits and virologists to investigate a potential new pathogen, position PPE and other resources, prepare hospitals for surges, inform the public on how to best reduce the risk to themselves and their families…these are just a few examples. One thing is clear: time is of the essence.

So, how much advance notice, or “lead time” is possible? If you rely on the healthcare system, as most public health departments currently do, the answer is none. That’s because this system looks at positive test results and doctor’s office visits. By the time they’re counted, illness has already spread in the community. Lagging indicators like these don’t help us get ahead of a pandemic.

That’s where Kinsa comes in. This system, which communicates with people as soon as they experience symptoms, well before they get sick enough to visit a doctor or get tested — that is, if a test even exists and is available (more on that in a minute), results in significant lead time, or early knowledge of where and when illness is spreading.

This is critical in two key situations.

1. The first is detecting the emergence of a new pathogen. When COVID first arrived on our shores in March 2020, there was no widely available test. Kinsa, however, began identifying pockets of unusual illness levels around the country. As it turns out, Kinsa’s illness signal led the first wave of hospitalizations and deaths — a critical window to prepare and prevent transmission — by 20 days.
2. The second situation reflects where we are today: an ongoing, evolving pandemic. You may have a lingering sense that because at-home testing is so prevalent, and since at-home tests typically aren’t reported — that cases are being undercounted because many people are forgoing the PCR testing that is counted by the health care system. However, Kinsa’s system helps paint a true picture by capturing illness incidences that will never be reflected in public health data. In essence, Kinsa provides lead time — that is, a window of awareness of rising illness before it is captured in public health data systems. The chart below shows how Kinsa picked up on the peak of the Omicron wave, about 11 days before it was reflected in case counts.

Kinsa’s advantage is strongest exactly when we need it most: when there is less access to testing (less funding for ubiquitous PCR testing), decreased lab turnaround time (longer delays in getting results), and more at-home testing (fewer reported results).

This has been born out throughout the pandemic:

*Mean age and turn around time statistics are approximations.

• Original Variant: testing was sparse in March 2020, so many of the first cases of COVID went undetected. By identifying atypical illness levels in the population, Kinsa picked up an increase of illness 20 days prior to the increase in covid laboratory positives.
• Alpha (B.1.1.7): Kinsa’s national lead time during Alpha was 8 days. This lead time was in part due to test turnaround time averaging 2–4 days.
• Delta (B.1.617.2): Test turnaround time was the shortest it was the entire pandemic (<24 hrs), and as a result, Kinsa’s system did not have lead time during this wave.
• Omicron (BA.1): Test turnaround time was the longest it had been since the beginning of the pandemic, and as a result Kinsa’s national lead time during Omicron was 11 days.

It’s likely that the Omicron dynamic will continue during future waves of covid. Our reliance on at-home testing and longer turnaround time for lab testing will mean that public health data won’t paint a complete picture. Kinsa will continue to capture illness levels in real time and share this information at www.healthweather.us, giving people a valuable resource to understand their risk and take steps to protect themselves as this pandemic rages on.

For more great content on lead time from our epidemiologists and data scientists, see here.