Statistical Sins: Bad Polling, Cherry-Picking, and Trust

Opinion polling has been used in politics for decades. In fact, the first known example dates from 1824, when Andrew Jackson went up against John Quincy Adams for the Presidency. The poll ended up correctly predicting that Jackson would win, and as a result, opinion polling became more popular. Today, it is a huge industry that many have gotten into - and that's not necessarily a good thing.

Two weeks ago, FiveThirtyEight’s Harry Enten discussed a highly publicized, though likely fake, poll that showed musician Kid Rock as a strong contender for US Senator of Michigan. (By the way, the link to the original poll results is no longer available.)

Fake polls are certainly problematic when they're used to frame and support arguments. As Enten points out, poll results can influence donors and voters, making polling results a sort of self-fulfilling prophecy. And as internet surveys become easier to conduct, the number of poorly conducted polls is likely to increase:

There is no power on earth that can or will keep politicians from cherry-picking polling data to get press, raise donations, or simply boost internal campaign morale. The bad news Enten shares is that the number of “pollsters” generating questionable data is rising, and could rise even more in the very near future.

Some political observers will prove to be suckers for phony or slanted polls, while others will throw the baby out with the bathwater and refuse to look at polling data altogether, relying instead on disingenuous “insider” assessments of elections or, worse yet, making anecdotes a substitute for analysis (anyone can sound superficially informed by talking to preselected groups of voters and gleaning predictable insights).

That quote above comes from a New York Magazine article discussing Enten's work. But the conclusion they draw from the rise of bad polling is very likely: people will respond to poll results that turn out to be inaccurate by simply disbelieving any poll results. In fact, we've probably already begun to see it in part due to the fact that, though polls had Hillary Clinton winning the Presidency, Trump won instead.

Yes, it's happened before:

And yes, we've always had a problem with politicians cherry-picking poll results to only pay attention to and present polls that are favorable to them. And, oh by the way, yes, we have a numerical and scientific literacy problem in this country, that goes back quite a ways - certainly before Trump won the Presidency.

The problem is that now we have a person in the White House who not only cherry-picks poll results - which has to be challenging considering how poorly he does in most polls - but has actually said he only believes results that are favorable to him. Verbalizing that issue combined with these examples of bad polls popping up more often is likely going to give people without the knowledge of what makes a poll good or bad license to do the same as Trump. At the very least, bad polling is going to hurt the public's trust in polling in general, even outside of politics.

It appears some are recognizing that Trump's attitude toward polls reflects his lack of competence as opposed to problems with the nature of polling. But among his base - that is, the people who go along with him the majority of the time - we're likely to continue to see lack of trust in polling, cherry-picking of results, and a blurred line between opinion and fact.