The Polling Crisis
“Don’t sweat the polls,” The Atlantic reassured us this October. In 2016, forecaster Sam Wang vowed to eat a bug if Donald Trump won — and ended up having to. The memory was still fresh, but in the intervening years pollsters had worked hard to regain our trust. As The Atlantic (and just about every other respectable publication) explained, the 2016 error was due to late deciders and the failure to weight by education. This year, most voters had made up their mind long before the election and pollsters had corrected the weighting mistake. Everything would be fine.
When the 2020 results came in, the reassurances vanished like a puff of smoke. Not only was this year’s statewide error as large as in 2016, but each major national poll was off by at least 3.5 points, while the Senate polls were even worse than the general election ones.
America is in the middle of a polling crisis — or is it? FiveThirtyEight’s Nate Silver told us there’s no need to panic. After all, the magnitude of this year’s polling error was about average by historical standards (that is, from 1972 onwards). Indeed, the 2020 error is nowhere near as bad as 1980’s, when state and national polls were off by nearly 9 points, completely missing Ronald Reagan’s landslide victory over Jimmy Carter.
Silver is right that the magnitude of this year’s polling error is historically unremarkable, but history holds other lessons too.
Historically, most polling failures have been due to unique factors, which never recurred in other election cycles. When FiveThirtyEight compared the biases in polls between 2000 and 2012, it found that “the accuracy of a state’s polls one election does not appear to have any bearing on their accuracy the next.” In 2020, that is no longer the case. What is new about the 2020 state-poll error is precisely…that it isn’t new. The general pattern of the 2020 error matches the 2016 pattern: by and large, the polls overestimated Democratic support in the same states and to similar degrees as in 2016. More precisely, there is a small but statistically significant correlation between the 2016 and 2020 state poll errors. (The correlation is larger for the Midwestern states which decided the 2016 election.)¹
