‭Using Opt-in Non-Probability Surveys for Over-Time State Level Estimates: The Role‬‭ of Guardrails and Data Quality‬

Socially impactful phenomena often occur across time and space – examples include pandemics, climate change, mass protests, and political campaigns. Survey data can play an important role in understanding these situations. Yet, there is a tension such that obtaining longitudinal, spatially expansive probability samples can be logistically and financially untenable. We offer a strategy that relies on (accessible) opt-in non-probability samples to obtain temporal and spatial granularity. It requires the use of multiple guardrail benchmarks (e.g., probability samples, administrative data) to validate a contemporaneous estimate, and acute attention to sampling representativeness and response validity. We demonstrate the utility of this approach with data on vaccination and infection rates during the COVID-19 pandemic in the U.S. We show that the opt-in non-probability data not only offer accurate estimates but also outperform problematic administrative benchmarks during certain time periods. We encourage further discussion about how to establish infrastructure to address emergent topics where probability samples are not readily available.