If you follow public opinion research on health care reform, you probably know that recent polls have generally shown a modest but definite trend towards support for reform efforts. The latest Gallup poll, for example, shows a plurality of respondents favoring reform legislation for the first time in a while. A new AP/GfK survey shows the support/oppose ratio on health reform has changed from 34/49 in September to 40/40 now.
There’s one quite jarring exception to this trend: Fox News, which released a poll this week showing that only 33% of Americans support health reform legislation, while 53% oppose it.
Since this is Fox we are talking about, could the results simply be a matter of systemic bias? You might think so, but as Nate Silver points out in a careful deconstruction of the poll at fivethirtyeight.com, the same survey puts the president’s job approval rate at 58%, a relatively high number.
How can you square the high approval rate with the exceptionally low assessments of the president’s top domestic priority? Look at where the questions appear in the poll, says Silver. The job approval question is first, while the health reform questions comes after a long series of heavily loaded questions about the president that are pretty close to Republican talking points. This is a texbook case of what is called “question order bias,” whereby poll questions that seem unobjectionable in isolation elicit very different responses when placed after “pushy” questions that predispose the respondent in one direction or another. And this is an old habit with Fox, helping to explain why its “horse-race” numbers during the last campaign–based on questions asked at the beginning of their surveys–were pretty much like everyone else’s, but its “issue” numbers tend to skew very, very conservative. As Silver concludes:

[T]hese question order effects can arise even when pollsters have the best of intentions, and even when they are asking unbiased questions. If, for instance, back during the Presidential campaign, you had asked a series of perfectly neutrally-worded questions on the economy before asking about the horse race, they could easily have tipped the numbers slightly in Obama’s direction, since the economy was perceived to be the Democrats’ strength….
But when you ask a series of biased questions before taking the voters’ temperature on health care or the horse race, you have much less excuse. Going forward, Fox News should put its health care questions closer to the top of their survey or break them out into a separate poll; take their numbers with a grain of salt until they do.

Make that a shaker of salt, Nate.