By Alan Abramowitz
In today’s New York Times op-ed, “Polls Apart“, Pew Research Center pollster, Andrew Kohut, observes:
The round of national surveys taken after the third presidential debate indicates that the polls are not going to give us a clear picture of who will win the election until the final days of the campaign, if then….It’s worth recalling that four years ago voters were similarly divided between the candidates. But the opinions about Al Gore and Mr. Bush did not bounce around, because there just wasn’t all that much enthusiasm for either man.
Andrew Kohut is correct that the polls probably will not give us a lear idea of who is going to win this year’s presidential election. However, the problem is not that voters’ preferences are more volatile than normal this year, as Kohut claims. The problem is that poll results are volatile due to the effects of sampling error, differences in weighting procedures, and different methods of identifying likely voters. As a result, even polls conducted at exactly the same time can produce divergent results. Nor is this situation unique to the 2004 election. The same volatility was evident prior to the 2000 election when, for example, the Gallup tracking poll released on October 26 showed George Bush leading Al Gore by 13 points while the Zogby tracking poll released the same day showed Gore leading Bush by 2 points.
8 comments on “A Comment on Andrew Kohut’s New York Times Op-Ed”
kate on
PEW and Andy Kohut had the race right on… with this pre-election prediction release…
check it out peoplepress.org
Not so much horse and buggy huh?
Mady..about Zogby, Kerry had a lead last week. Bush had a lead at the end of last week/ the beginning of this one. Now its tied. Indicates that the race is pretty stable right now in a tie.
ABC WaPo is easy. Look at their registered voter data. A huge change 3 days ago from a pretty even split to about a 6 point spread that is fairly stable over 3 days. Indicates to me a pretty big Bush outlier came on 3 days ago, and since then we’ve had 48-47 or so splits. When that big day rolls off, it’ll move closer to tied again. It’s a 4 day moving average, so I expect one more bad spread for us and then, bam, back to normal.
Mady –
One clear difference between the Washington Post/ABC poll and all other polls whose methods I’ve seen described in sufficient detail is that the Post/ABC poll does not weight respondents for Hispanic origin. Hispanics are, as far as I can tell, lumped in with whites.
I would expect that Hispanics with high school education or less would have an extremely low rate of participation in a poll, because of long working hours and language barrier. (Even if you speak a non-native language reasonably well, do you speak it well enough to feel like doing a 20-minutes telephone interview?) The non-participating low-education Hispanics are replaced, due to the weighting process, with low-education whites who (at least in the income-biased sample produced by polling methods) tend slightly toward Bush.
(Note that the ABC weighting method — I haven’t seen the details of the Post’s — is different from most other polls. I called it “cell-weighting” in an earlier post. The more usual weighting method, which I called “parameter weighting,” will also underrepresent low-education Hispanics in the final results, but they will be replaced by a mixture of low-education whites and blacks and more educated Hispanics. This, I would guess, will also bias results against Kerry, but to a substantially lesser degree.)
I think this will be remembered as the year when polls were shown to be living in the horse and buggy world, while the US had moved on to autos.
The polls are measuring a universe that cannot be measured by phone polls pressed into 2-3 day chunks of time in samples that total 1000 and don’t look remotely like the electorate.
Two items:
1. I was going to ask the same question as a previous post: why is the WP/ABC poll so different? Even on registereds, it showed a 7-point Bush lead today, I think…
2. Any idea how many new folks might come out in FLA on the minimum wage referendum? That’s a huge potential swing for Kerry, it seems to me…
Keep up the good work — love this site!!
eg
Yes.
What’s different is the intensity around the election – it’s so high. We Dems want the polls to reassure us, to give us a moment of hope, a moment of reprieve from this crushing Republican control of the Executive, the Legislative and seemingly the Judiciary.
We fear we will lose in Florida again. We fear that the Supreme Court will decide it all again. We fear that we’ll lose because we didn’t get that one more vote out in Minnesota or Ohio or New Mexico. The closeness of the 2000 election, the division in the country, the obsessive news coverage of the coverage of the coverage of the election and the pundits talking about the polls and the insane number of little instant polls and the blogs and the emails and the talk radio and and and . . . It’s hard to keep positive and to think all of our little-door-knocks and small contributions will add up to getting rid of a horrible, horrible president.
I mean – if Kerry wins, it will be the first time since Paul Wellstone won in Minnesota that I can say it was the work of all of us ordinary people doing what we can in the crevices of our work-a-day lives. That would be a pretty big wish come true –
It’s not that no one cared about Gore or Bush as candidates, no one really cared about the election – until, of course, that final day in Florida when it all blew up and then there was Kyoto and ABM and September 11 and Iraq and we all realized how wrong we had been not to work as hard as we could in Fall 2000.
Kerry in a landslide –
Two things:
First of all, I find it alarming that Kerry has not led at all in any recent Zogby polls. Yes, they do waver, but never to Kerry getting a majority percentage.
Also, could anyone explain the methodology of the Washington Post/ABC tracking poll, and why, in a time when very few polls show results that favor Kerry, but most show a quite close race, the Post has been showing Bush consistently at 50% or over. Today it shows, I think, 51 Bush, 45 Kerry.
Thanks,
Mady
Polling is definitely going through growing pains. You cite a number of reasons why polls vary (clearly, on election day most or all will have been wrong–if by wrong we mean to understand predicting the outcome).
I think the polls themselves may be part of the problem. We have respondent fatigue–not on an individual level, but as a nation. Look at those so-called “undecideds.” I find it odd that their numbers have not declined significantly since June. My guess is that they’re not measuring conviction so much as fatigue. It seems like something between 5-10% of every poll are undecided. Wouldn’t we expect to see that decline as the electorate gets to know the candidates? But it hasn’t. My guess is that this group has just had it with pollsters. They’re decided, but they ain’t tellin’ us.
Also, it’s odd that Andy would make the argument about voter volatility when he himself has measured how strongly convinced so much of the electorate is.
This year’s big media narrative has been the confirmation saga of Neera Tanden, Biden’s nominee for director of the Office of Management and Budget. At New York I wrote about how over-heated the talk surrounding Tanden has become.
Okay, folks, this is getting ridiculous. When a vote in the Senate Homeland Security and Governmental Affairs Committee on the nomination of Neera Tanden was postponed earlier this week, you would have thought it presented an existential threat to the Biden presidency. “Scrutiny over Tanden’s selection has continued to build as the story over her uneven reception on Capitol Hill stretched through the week,” said one Washington Post story. Politico Playbook suggested that if Tanden didn’t recover, the brouhaha “has the potential to be what Biden might call a BFD.” There’sbeen all sorts of unintentionally funny speculation about whether the White House is playing some sort of “three-dimensional chess” in its handling of the confirmation, disguising a nefarious plan B or C.
Perhaps it reflects the law of supply and demand, which requires the inflation of any bit of trouble for Biden into a crisis. After all, his Cabinet nominees have been approved by the Senate with a minimum of 56 votes; the second-lowest level of support was 64 votes. One nominee who was the subject of all sorts of initial shrieking, Tom Vilsack, was confirmed with 92 Senate votes. Meanwhile, Congress is on track to approve the largest package of legislation moved by any president since at least the Reagan budget of 1981, with a lot of the work on it being conducted quietly in both chambers. Maybe if the bill hits some sort of roadblock, or if Republican fury at HHS nominee Xavier Becerra (whose confirmation has predictably become the big fundraising and mobilization vehicle for the GOP’s very loud anti-abortion constituency) reaches a certain decibel level, Tanden can get out of the spotlight for a bit.
But what’s really unfair — and beyond that, surreal — is the extent to which this confirmation is being treated as more important than all the others combined, or indeed, as a make-or-break moment for a presidency that has barely begun. It’s not. If Tanden cannot get confirmed, the Biden administration won’t miss a beat, and I am reasonably sure she will still have a distinguished future in public affairs (though perhaps one without much of a social-media presence). And if she is confirmed, we’ll all forget about the brouhaha and begin focusing on how she does the job, which she is, by all accounts, qualified to perform.
PEW and Andy Kohut had the race right on… with this pre-election prediction release…
check it out peoplepress.org
Not so much horse and buggy huh?
Mady..about Zogby, Kerry had a lead last week. Bush had a lead at the end of last week/ the beginning of this one. Now its tied. Indicates that the race is pretty stable right now in a tie.
ABC WaPo is easy. Look at their registered voter data. A huge change 3 days ago from a pretty even split to about a 6 point spread that is fairly stable over 3 days. Indicates to me a pretty big Bush outlier came on 3 days ago, and since then we’ve had 48-47 or so splits. When that big day rolls off, it’ll move closer to tied again. It’s a 4 day moving average, so I expect one more bad spread for us and then, bam, back to normal.
Mady –
One clear difference between the Washington Post/ABC poll and all other polls whose methods I’ve seen described in sufficient detail is that the Post/ABC poll does not weight respondents for Hispanic origin. Hispanics are, as far as I can tell, lumped in with whites.
I would expect that Hispanics with high school education or less would have an extremely low rate of participation in a poll, because of long working hours and language barrier. (Even if you speak a non-native language reasonably well, do you speak it well enough to feel like doing a 20-minutes telephone interview?) The non-participating low-education Hispanics are replaced, due to the weighting process, with low-education whites who (at least in the income-biased sample produced by polling methods) tend slightly toward Bush.
(Note that the ABC weighting method — I haven’t seen the details of the Post’s — is different from most other polls. I called it “cell-weighting” in an earlier post. The more usual weighting method, which I called “parameter weighting,” will also underrepresent low-education Hispanics in the final results, but they will be replaced by a mixture of low-education whites and blacks and more educated Hispanics. This, I would guess, will also bias results against Kerry, but to a substantially lesser degree.)
I think this will be remembered as the year when polls were shown to be living in the horse and buggy world, while the US had moved on to autos.
The polls are measuring a universe that cannot be measured by phone polls pressed into 2-3 day chunks of time in samples that total 1000 and don’t look remotely like the electorate.
Two items:
1. I was going to ask the same question as a previous post: why is the WP/ABC poll so different? Even on registereds, it showed a 7-point Bush lead today, I think…
2. Any idea how many new folks might come out in FLA on the minimum wage referendum? That’s a huge potential swing for Kerry, it seems to me…
Keep up the good work — love this site!!
eg
Yes.
What’s different is the intensity around the election – it’s so high. We Dems want the polls to reassure us, to give us a moment of hope, a moment of reprieve from this crushing Republican control of the Executive, the Legislative and seemingly the Judiciary.
We fear we will lose in Florida again. We fear that the Supreme Court will decide it all again. We fear that we’ll lose because we didn’t get that one more vote out in Minnesota or Ohio or New Mexico. The closeness of the 2000 election, the division in the country, the obsessive news coverage of the coverage of the coverage of the election and the pundits talking about the polls and the insane number of little instant polls and the blogs and the emails and the talk radio and and and . . . It’s hard to keep positive and to think all of our little-door-knocks and small contributions will add up to getting rid of a horrible, horrible president.
I mean – if Kerry wins, it will be the first time since Paul Wellstone won in Minnesota that I can say it was the work of all of us ordinary people doing what we can in the crevices of our work-a-day lives. That would be a pretty big wish come true –
It’s not that no one cared about Gore or Bush as candidates, no one really cared about the election – until, of course, that final day in Florida when it all blew up and then there was Kyoto and ABM and September 11 and Iraq and we all realized how wrong we had been not to work as hard as we could in Fall 2000.
Kerry in a landslide –
Two things:
First of all, I find it alarming that Kerry has not led at all in any recent Zogby polls. Yes, they do waver, but never to Kerry getting a majority percentage.
Also, could anyone explain the methodology of the Washington Post/ABC tracking poll, and why, in a time when very few polls show results that favor Kerry, but most show a quite close race, the Post has been showing Bush consistently at 50% or over. Today it shows, I think, 51 Bush, 45 Kerry.
Thanks,
Mady
Polling is definitely going through growing pains. You cite a number of reasons why polls vary (clearly, on election day most or all will have been wrong–if by wrong we mean to understand predicting the outcome).
I think the polls themselves may be part of the problem. We have respondent fatigue–not on an individual level, but as a nation. Look at those so-called “undecideds.” I find it odd that their numbers have not declined significantly since June. My guess is that they’re not measuring conviction so much as fatigue. It seems like something between 5-10% of every poll are undecided. Wouldn’t we expect to see that decline as the electorate gets to know the candidates? But it hasn’t. My guess is that this group has just had it with pollsters. They’re decided, but they ain’t tellin’ us.
Also, it’s odd that Andy would make the argument about voter volatility when he himself has measured how strongly convinced so much of the electorate is.