For serious political junkies, nothing’s more frustrating than reading about some striking poll results, and then discovering that the reliability of the poll is in question because the polling firm (or the campaign or media enterprise sponsoring the survey) won’t tell you much of anything about its methodology.
To deal with this persistent problem, Pollster.com, Mark Blumenthal’s indispensible site, has started a “Disclosure Project” aimed at eliciting the kinds of information necessary to separate the wheat from the chaff, or at least to compare divergent results:
Starting today we will begin to formally request answers to a limited but fundamental set of methodological questions for every public poll asking about the primary election released in, for now, a limited set of states: Iowa, New Hampshire, South Carolina or for the nation as a whole. We are starting today with requests emailed to the Iowa pollsters and will work our way through the other early states and national polls over the next few weeks, expanding to other states as our time and resources allow.
The questions focus on “screening” for likely participation in primaries or caucuses; sample size and composition; and polling techniques. And as Blumenthal pointedly mentions, pollsters are actually required by the code of ethics of their profession to make such information available on request.
This project isn’t just of concern to us junkies. Like it or not, polls affect media coverage, donations, volunteer activity, campaign strategies, and sometimes, even election results. (I can remember a gubernatorial election in my home state of Georgia many years ago when a candidate kept releasing “internal poll” results showing a late surge towards a runoff position, creating considerable media coverage and momentum. It was generally believed by political insiders that the campaign was literally just making the numbers up.)
The least we can expect is that pollsters and their paymasters let the rest of us in on their methods if they expect us to take the results seriously.
Yes, this is a vital component of the current discourse. Everyone knows TV polls, website polls and the like are “unscientific”. People have a tendency to trust polls published in newspapers, however, and need to be forewarned. Pollster.com will tend to intimidate those who would otherwise like to skew their sampling or otherwise fudge their results. Mark Blumenthal is yet another diligent patriot at work.
Naturally, I hope you will discover Nancy Tobi and others who want to make sure the one poll that counts, the one on election day, is counted by hand. That’s right, by hand. There is a small but exponentially growing movement to throw all DREs and optical scanners out and to revert to hand counting. It only adds a few short hours to the tabulation and is self auditing, producing unquestionably correct results. It costs less than the machines and is done in the individual precincts before the ballot boxes can be moved or stuffed.
It will be the end of weird phenomena like we saw in 2004, with Bush gaining 64% more of the urban vote than he had won in 2000. Somehow.