Today I'd like to talk about something that's near and dear to my heart: polling. No other subject in politics is more widely discussed with less understanding than the numbers polling firms spit out like clockwork. Cable news is especially guilty of this: Their breathless reports that "Obama's up 6 in New Mexico! McCain's up 2 in Florida!" absurdly oversimplify the actual science of measuring public opinion. The truth is, accurately reading polls is a blend of art and science that requires a bit of knowledge, a bit of history and a dash of humility. So if you want to really understand the state of any given race, read on.
The biggest mistake most people make when they try to interpret polling data is that they disregard the margin of error. The MoE is what makes every poll, to one extent or another, "fuzzy." It's what prevents us from ever knowing the precise, actual percentages of the public's preferences. Most reputable polls carry an MoE of plus or minus three points at a 95 percent confidence level. So let's take the latest FOX News poll, which puts Obama at 49 percent and McCain at 40 percent nationally. With an MoE of three percent at a 95 percent confidence level, we can say that we know for a fact that there is a 95 percent chance that Barack Obama's support is as high as 52 percent or as low as 46 percent, and that McCain's support is as high as 43 percent or as low as 37 percent. What FOX pegged as a 9 point difference could be as small as 3 points: 46-43 percent.
If you're thinking that all this is sounding like the odds of the success of Sex Panther (60 percent of the time, it works every time), you're not alone. Although a single poll like the FOX one is good at getting "ballpark" figures, when we start getting into closely matched numbers where two points is a huge difference, we need more precision. And that leads us to the second biggest mistake people make when trying to interpret a poll: They look at it in isolation.
Instead, the best way of ascertaining public support levels is to not give credence to any one poll (especially since people tend to cherry pick the ones that show their candidate ahead), but instead to combine all polls you trust into one rolling average. This greatly diminishes the impact of an "outlier" and minimizes the margin of error considerably. Fortunately for us, there are a number of sites out there that combine all the different results of various pollsters to try to form one coherent number. Check out these polling aggregators, which I'll list in order of worst to best:
RealClearPolitics.com: These guys were doing it first. They combine tons of national and state polls to create an average that's far more reliable than even the best single poll. Some controversy has erupted over whether or not they cherry pick certain polls to make things look better for Republicans, though, so you may want to try:
Pollster.com: Pollster.com doesn't have the issues RCP does because of a very simple philosophy. They take every poll they can get their hands on and throw it into the mix. Very effective, very well run and complete with great graphs visually depicting public opinion trends.
Fivethirtyeight.com: This is the gold standard, in my opinion. The guys running this site know statistics and sabermetrics inside and out, and they've turned their exhaustive knowledge of running numbers to politics. I firmly believe that the closest we will ever get to knowing the true level of support for any issue or candidate can be found at 538.
There's also a whole host of other, lesser things to keep in mind. Examples include knowing which pollsters have "house effects" (a lean to Democrats or Republicans) and even what's being polled. But if you only remember what I just told you, then congratulations — you probably know more than Bill O'Reilly or Wolf Blitzer.
--
Michael Sherry is a senior majoring in political science. He can be reached at Michael.Sherry@tufts.edu.
