For serious political junkies, nothing’s more frustrating than reading about some striking poll results, and then discovering that the reliability of the poll is in question because the polling firm (or the campaign or media enterprise sponsoring the survey) won’t tell you much of anything about its methodology.
To deal with this persistent problem, Pollster.com, Mark Blumenthal’s indispensible site, has started a “Disclosure Project” aimed at eliciting the kinds of information necessary to separate the wheat from the chaff, or at least to compare divergent results:
Starting today we will begin to formally request answers to a limited but fundamental set of methodological questions for every public poll asking about the primary election released in, for now, a limited set of states: Iowa, New Hampshire, South Carolina or for the nation as a whole. We are starting today with requests emailed to the Iowa pollsters and will work our way through the other early states and national polls over the next few weeks, expanding to other states as our time and resources allow.
The questions focus on “screening” for likely participation in primaries or caucuses; sample size and composition; and polling techniques. And as Blumenthal pointedly mentions, pollsters are actually required by the code of ethics of their profession to make such information available on request.
This project isn’t just of concern to us junkies. Like it or not, polls affect media coverage, donations, volunteer activity, campaign strategies, and sometimes, even election results. (I can remember a gubernatorial election in my home state of Georgia many years ago when a candidate kept releasing “internal poll” results showing a late surge towards a runoff position, creating considerable media coverage and momentum. It was generally believed by political insiders that the campaign was literally just making the numbers up.)
The least we can expect is that pollsters and their paymasters let the rest of us in on their methods if they expect us to take the results seriously.