Gallup has had it’s share of critics among Democratic Party supporters in recent years, including contributors to TDS, who felt that bias was built into its methodology. For better or worse, however, Gallup, likely enjoys the highest name recognition among all pollsters. No doubt, then, that many will interpret Gallup’s abandonment of horse race polling during the primary season as indicative of a decline in public regard for polling in general.
You won’t have much trouble finding well-stated criticism of Gallup’s horse race polls lately, especially since they have had a couple bad years (2010 and 2012). At FiveThirtyEight.com, however, Harry Enten explains “Why we’re worse off without Gallup“:
Gallup uses rigorous polling methodologies. It employs live interviewers; it calls a lot of cell phones; it calls back people who are harder to reach. More than that, it took the criticism it received after the 2012 election seriously, even bringing in outside help to figure out what went wrong. Gallup rates as solidly average in FiveThirtyEight’s pollster ratings in large part because of those techniques. It’s had two bad elections recently, but it’s never a good idea to judge a pollster on just a couple of election cycles; Gallup has also had good years.
Polling consumers are far better off in a world of Gallup’s than in a world of Zogby Internet polls and fly-by-night surveys from pollsters we’ve never heard of. There is plenty of shadiness in the polling community, and Gallup seemed to be opening its doors.
Gallup says it will still conduct issue polling, but here’s the problem: Elections are one of the few ways to judge a pollster’s accuracy. And that accuracy is important: We use polls for all kinds of things beyond elections. How do Americans feel about the economy? Do elected leaders have the trust of the public? Is there support for striking a deal with Iran? By forgoing horse-race polls, Gallup has taken away a tool to judge its results publicly.
Enten acknowledges that there will still be plenty of pollsters doing horse race surveys. As Steven Shepard notes at Politico, Editor-in-chief Frank Newport’s Gallup announcement did have a smidgeon of wiggle room in it:
Asked whether Gallup plans to skip horse-race polling for the entire 2016 primary process, Newport said, “That’s certainly what we’ve decided to date.”…And Newport also wouldn’t commit to horse-race surveys for the general election…”We have not made final decisions on what we are going to do in 2016 yet,” Newport said.
Think of Gallup’s retreat from the horse race circus as an experiment. They can always reverse their decision if it begins to look unprofitable to them. For now, I give Gallup’s big quit a thumbs up. More emphasis on thoughtful issue polling is certainly needed, and the value of early horse race polls in the primary season can be fairly described as dubious. Moreover, if Gallup can purge bias from their methods going forward, they could have a more positive impact on data analysis of U.S. elections.