I read an awful lot of stuff about polls, and know just enough about polling to know (most of the time) when I’m being spun. Having seen one clear example, I decided to slice and dice it at New York.
Polling averages like those published by RealClearPolitics and the highly masticated poll-based analysisat FiveThirtyEight are a good corrective to the tendency to see only the results that confirm the reader’s biases, hopes, and dreams. But the announcement of polls is often accompanied by the blare of partisan trumpets, and the results laundered by a partisan spin cycle. This is very evident with respect to a piece at CNBC today. Here’s the lede:
“With economic optimism soaring in the country, will Democrats be able to sweep to power in either house of Congress or will buoyant sentiment help Republicans keep hold of their Congressional majorities?
“The latest CNBC All-America Economic Survey offers mixed signals, but leans against a wave Democratic election like that those that swept Republicans to power in 2010 and 2014.
“The poll of 800 Americans across the country, with a margin of error of 3.5 percent, found a six-point Democratic lead on the question of who voters will choose in the November congressional elections. The 42 percent to 36 percent margin is not far from what pollsters would expect given the greater percentage of Democratic registered voters.
“‘A six point differential is not something that’s going to cause a big electoral wave,’ said Micah Roberts, the Republican pollster on the CNBC poll, a partner Public Opinion Strategies. ‘Economic confidence that people have among a lot of groups is providing a buffer’ for Republicans.”
1) So the findings “lean against a wave election like those that swept Republicans to power in 2010 and 2014.” It’s hard to understand exactly what this means. Republicans picked up 63 net House seats in 2010; nobody’s predicting Democrats will do that well, and they need just 23 seats to win control of the House. Republicans netted 13 House seats in 2014. That isn’t “like” 2010. If this is supposed to be a reference to the Republican conquest of the Senate in 2014, we’re really mixing apples and oranges since a national poll of partisan preferences has little or nothing to do with a Senate landscape that exists in one-third of the states.
2) This is a poll of 800 adults — not registered voters, much less likely voters. That’s a very imprecise sample. And the Margin of Error of 3.5 percent could be pretty significant when it comes to a generic ballot difference — the key statistic in the poll — of 6 percent.
3) The suggestion that the Democrats’ margin in party preferences (the so-called generic congressional ballot) is meaningless because it’s “not far from what pollsters would expect given the greater percentage of Democratic registered voters” is very misleading. The generic ballot includes Democrats, Republicans, and independents; if the plurality of Democratic registration determined it, Democrats would always have an advantage, which they don’t.
4) All the economic data in this poll is interesting, but isn’t terribly predictive when it comes to midterm elections, which are pretty highly correlated to overall presidential approval ratings (which have been underwater almost the entirety of the Trump presidency) and to the generic congressional ballot. Economic perceptions can help explain why voters feel the way they do, but if “economic optimism is soaring,” that will show up in the more predictive poll findings.
5) At varying points CNBC suggests the poll shows there probably won’t be a “wave Democratic election” or a “big electoral wave” or a “massive wave election.” Nowhere is there any definition of the phantom phenomenon the data are supposed to rebut. Presumably a Democratic takeover of the House would be considered a “wave,” if not a “massive wave,” whatever that means. CNN’s Harry Enten thinks a Democratic generic ballot advantage of six to eight points will be sufficient to accomplish that; Emory University’s Alan Abramowitz thinks four points could be enough. Others think it will require a bigger margin. But this particular poll doesn’t provide any decisive guidance on the subject.
At the moment, the RealClearPolitics polling average on the generic ballotquestion gives Democrats a 7.2 percent advantage. FiveThirtyEight’s gives Democrats an 8.6 percent lead. Both these averages include the CNBC poll, though not very high in their listings, because it was conducted from October 4–7, which not exactly super-fresh in the world of public opinion.
I went through the analysis above not because this particular poll used a questionable methodology (it didn’t) or is somehow useless (it’s not). But it’s important to know when the presenter of polling data is selling you a bill of goods for her or his own reasons. There will be a lot more of that as November 6 approaches.