ERRORS IN POLITICAL POLLING

Polling is a complicated business. The key thing to understand is that the margin of error does not always describe the true error inherent in the poll, so polls that boast a small error can end up being completely wrong.

The concept of polling rests on the assumption that the opinions of the people sampled in the poll accurately represent the distribution of opinions across the entire population, which can never be completely true. The “margin of error” describes the uncertainty that comes from having such a small sample size relative to the size of the population. In general, the more people are surveyed, the smaller the margin of error. But this doesn’t take into account another key source of error called “biased sampling”. The fact that a poll samples a lot of people does not mean that it does so in the truly random fashion that would be needed to extrapolate results to the larger population. Unfortunately, many polls fall victim to a number of biases that significantly skew their results despite their small margin of error.

Another possible source of polling error is known as volunteer bias, wherein the people who volunteer their opinions to a poll do not represent the distribution of the entire population.

The time at which polls are conducted also affects their margin of error. The accuracy of polls drastically increases the election gets closer.

To successfully sift through the massive amounts of data, we must keep in mind the error inherent in all polls due to sample size and methodology. Polling data is rarely perfect and often inconclusive or misleading, so its pays to pay attention to the details.

Problems

The first problem in the U.S. presidential election this year is that some Trump voters might be lying to the pollsters. May be they don't want to admit that they support Trump. The pollsters call this “social desirability bias in speaking with interviewers on the phone. But on November 8, in the privacy of the voting booth, they will cast their secret ballot for the Republican.

The second problem is that the pollsters’ standard criteria for “likely voters” may not work this year. If you are in the polling business, it’s not hard to call people and ask whom they plan to vote for. The hard part is deciding whether to count them as “likely voters”—because more than 40 percent of Americans eligible to vote have not cast a ballot in the last two presidential elections. So all pollsters rank the people they poll on the likelihood of their voting. The traditional likely voter methodology is pretty straightforward: They ask if the voter is registered, if he intends to vote, if he knows where his polling place is—and whether he’s voted in the past. The most important criterion for a “likely” voter is whether they voted in the last election. If they didn’t, they are typically judged “not likely” to vote, and they are not counted in the poll results. That’s the science of opinion polling, based on historical experience.

But the scientists are not unanimous about this methodology. “A voter can tell he’s registered,” tell he’s certain to vote, tell he’s very engaged by the election, tell he knows where his polling place is, etc., and still be excluded from the results if he didn’t vote in the past. If a voter intends to vote, he should be counted in the poll results.

Pundits are worrying that pollsters are using the wrong definition of “likely voter” this time. In fact, that’s what Trump is counting on. His campaign is betting on people who have not voted recently especially white working-class men alienated by the whole system. They may get themselves to their polling place this year, for the first time in a long time, to cast a vote for Trump.

Pollsters do measure “intensity” of political preferences. Gallup, for example, asks whether support for a candidate is felt “strongly” or not. The assumption of course is that those who hold their views "strongly' are more likely to show up on election day than those who say they are simply expressing a preference. A July poll found the level of “strong” support was about equal for both Clinton and Trump—and strikingly low. The equal proportions suggest intensity is not going to skew the poll results this year.

The final problem is one everyone knows: the uniqueness of Trump himself. All of political science is based on history, on the idea that patterns in the past will continue in the future. But Trump is so different from every other candidate in the recent past that he could break out of the historic patterns of voting. That’s pretty much what happened in the primaries, when so many experts said with great conviction that Trump couldn’t win. Their reasoning was strong: He had no ground game, no field operation working to get his supporters to the polls on election day; he had no TV ads, which candidates all consider essential; he wasn’t raising money, or spending it. He had no real campaign organization and no experience in politics. In the past, candidates like that never won. But, of course, the Republican primaries were different this time.