Having experienced some vertigo in sorting through polling data this year, I looked into some of the reasons for all the disparate findings, and wrote about it at New York:

There’s been a lot of talk about polling accuracy this election year, as there has been in the last five election cycles. Four of those election years (2012, 2014, 2016, and 2020) produced results significantly different from the expectations created by the best-known and (previously) most reliable outfits conducting national and state-level public-opinion research. In 2012, Democrats overachieved their standing in the polls, as did Republicans in 2016 and 2020. In 2018, the polls pretty much nailed the results nationally, though there were some misses in Senate races won by Republicans.

There are several reasons fears about polling accuracy are strong right now. First, 2022 is a midterm election where very small changes in the results could yield big consequences, thanks to the dead-even Senate and the tiny Democratic margin of control in the House (there are also many potential 2024 presidential battlegrounds where partisan control of the election machinery is up for grabs this year).

Second, there is a bit of residual trauma in the political commentariat about pro-Democratic polling errors before the astonishing victory of Donald Trump in 2016 and before his near-reelection (echoed by strong House gains by Republicans) in 2020. Pro-Republican errors in 2012, and the mostly accurate 2018 surveys, have been all but forgotten. Pro-Republican polling errors in 2022 special elections have all but been ignored or minimized.

Third, there are some pretty significant differences in what the pollsters are showing nationally and in individual contests this year. Consider the most-cited (and typically most reliable) indicator of the House national popular vote, the polling question known as the generic congressional ballot. The polling averages (per RealClearPolitics) on this indicator have been nearly even since the beginning of August. But one pollster, Trafalgar Group, has been showing Republicans with a five-to-eight point advantage in monthly soundings since July. (Another pollster Republicans love to love, Rasmussen, has consistently shown the GOP leading in the general ballot as well, though not by as large a margin.)

Similarly, Trafalgar has Republican Senate candidate Tiffany Smiley statistically tied with Washington’s Democratic incumbent Patty Murray, while all but one of the other polls of this race have Murray up by double digits. The Cook Political Report rates the contest as “Solid Democratic,” which means not remotely competitive. Then there’s the Pennsylvania governor’s race, which is turning into a Democratic rout, thanks to the incompetent campaigning and extremist antics of Republican nominee Doug Mastriano. Six of the last seven public polls have shown Democrat Josh Shapiro up by double digits. The exception? Trafalgar Group, which showed a statistical tie in mid-September.

There are some races where Tragalgar isn’t so much an outlier as one end of a pretty broad spectrum of findings. In the Ohio Senate race, for example, the RCP averages have Republican J.D. Vance leading Democratic Tim Ryan by 1.2 percent. Trafalgar Group shows Vance up by 5 percent, while Siena has Ryan up by 3 percent.

Now as it happens, Trafalgar got a lot of positive attention after the 2016 presidential election for accurately showing Donald Trump ahead in Florida, Michigan, and Pennsylvania, when nearly every other national polling outfit had Hillary Clinton winning all three states. And the same pollster wound up with a relatively low average error in 2020, particularly as compared to some of the big established firms like Monmouth, Quinnipiac, and SSRS (though Trafalgar Group founder and former Republican operative Robert Cahaly incorrectly predicted a Trump reelection, and erroneously showed him ahead in Arizona, Michigan, and Pennsylvania). While Cahaly stubbornly keeps his full methodology private (he uses live calling, robocalling, and online sampling), he is famous for claiming he adjusts his findings to reflect alleged “social desirability bias,” which mostly means putting a thumb on the scales of red voters who allegedly assume pollsters want them to support blue candidates. So Trafalgar assumes a general pro-Democratic polling bias that he aims to correct. You can see how that might or might not work out well.

Another common source of polling differences involves the basic sample. Often Republicans look better in polls of “likely voters” rather than “registered voters” or “all adults,” particularly in a midterm election with Democrats controlling the White House, a scenario that usually (but not universally) gives the GOP a turnout advantage. But at this stage of the election cycle, virtually all pollsters have already “switched over” to likely voter models, eliminating one artificial reason for differences in findings.

The thing about a recent record for polling accuracy is that it earns pollsters more business, so Trafalgar Group (and to some extent Emerson College, which did pretty well in 2020) is expanding its footprint this year, and its arguably affecting the polling averages more than in the past. But in some of the more heavily polled contests, averages probably still smooth out the differences between pollsters and their methodologies. In the red-hot Georgia U.S. Senate contest between Democratic incumbent Raphael Warnock and Republican challenger Herschel Walker, there have been eight published polls in the RealClearPolitics database since the beginning of August. Four (including Trafalgar Group, Emerson, Insider Advantage, and the University of Georgia) show Walker ahead, and four (Quinnipiac, Marist, YouGov, and Fox News) show Warnock ahead. The average puts Warnock ahead by 0.7 percent; in other words, the race is tied. For variety’s sake, you can consult the polling averages at FiveThirtyEight, which weights poll findings according to pollster accuracy and partisan bias. It’s still a tie race, with the projected vote share being 49.8 percent for Warnock and 48.6 for Walker. Indeed, if you are placing a bet on the contest the best wager is that neither candidate will win a majority and Georgia will again hold one of its notorious general election runoffs.

We won’t know until after the elections how to assess pollsters, or how to retroactively adjudge the impact on expectations of the very real differences in their findings. But at this point we can say that if Trafalgar Group’s polling is correct, there is a broader range of competitive statewide elections in play (if Patty Murray is truly in trouble, which Democrats are really safe?), and Kevin McCarthy can go ahead and put in an order for a Speaker’s gavel. But like partisan activists, a lot of people in the political prediction business will be white-knuckling it and composing their spins on and after November 8.