The 2016 U.S. presidential election is over. And if you'd been following the polls, the results might have been surprising. What went wrong with those polls? It has to do with statistical and systematic uncertainties.
Unless you’ve been living in a deep-sea submarine, trekking through the rainforest, or perhaps traveling through a magical space and time portal you stumbled upon in your grandmother’s closet, you know that the 2016 presidential election in the United States came to an eventful conclusion this week.
And if you had been paying attention to the election—in particular the polls—during the weeks and months leading up to election day, you might very well have found the result to be a wee bit surprising. Because the polls consistently told a different story than the one that played out.
Or did they?
Were the polls really wrong? Or were they just really uncertain? Did the polls fail to correctly predict the future? Or was it a mistake for people to think that predicting the future is something that polls can do? The answers to all of these questions is “Yes.”
The reason for all the misconceptions about the polls tied up in these questions is a fundamental misconception about statistics. Namely, what do statistics actually tell us? How certain should we be about them? And how exactly can they lead us astray?
What Do Polls Actually Predict?
During the last few election cycles, an industry has developed around compiling polling data into models which are used to predict the likelihood of various outcomes. But what exactly do these statistical models tell us? In particular, I’d like to talk about what it meant that Nate Silver’s fivethirtyeight.com gave Hillary Clinton a 71.4% chance of winning and Donald Trump (the eventual winner) “just” a 28.6% chance of winning.
A lot of Clinton supporters that I know were relieved to see her chance of winning climbing so high. When I asked them why they felt so good about it, they replied that a number over 70% made it feel like the odds were in her favor—that it was essentially a “sure thing.” But if you stop and think about it, you’ll realize that a 70% chance of winning isn’t actually all that great. And a 30% chance of winning isn’t actually all that bad (as we found out).