|
In response to
"So yea, these polling numbers seem way too good to be accurate -- (link)"
by
Jovian
|
Yes, the Polling Warning Signs Are Flashing Again
Democrats are polling well in exactly the places where surveys missed most in 2020.
|
By Nate Cohn
Sept. 12, 2022
Ahead of the last presidential election, we created a website tracking the latest polls — internally, we called it a “polling diary.” Despite a tough polling cycle, one feature proved to be particularly helpful: a table showing what would happen if the 2020 polls were as “wrong” as they were in 2016, when pollsters systematically underestimated Donald J. Trump’s strength against Hillary Clinton.
The table proved eerily prescient. Here’s what it looked like on Election Day in 2020, plus a new column with the final result. As you can see, the final results were a lot like the poll estimates “with 2016-like poll error.”
We created this poll error table for a reason: Early in the 2020 cycle, we noticed that Joe Biden seemed to be outperforming Mrs. Clinton in the same places where the polls overestimated her four years earlier. That pattern didn’t necessarily mean the polls would be wrong — it could have just reflected Mr. Biden’s promised strength among white working-class voters, for instance — but it was a warning sign.
That warning sign is flashing again: Democratic Senate candidates are outrunning expectations in the same places where the polls overestimated Mr. Biden in 2020 and Mrs. Clinton in 2016.
Wisconsin is a good example. On paper, the Republican senator Ron Johnson ought to be favored to win re-election. The FiveThirtyEight fundamentals index, for instance, makes him a two-point favorite. Instead, the polls have exceeded the wildest expectations of Democrats. The state’s gold-standard Marquette Law School survey even showed the Democrat Mandela Barnes leading Mr. Johnson by seven percentage points.
But in this case, good for Wisconsin Democrats might be too good to be true. The state was ground zero for survey error in 2020, when pre-election polls proved to be too good to be true for Mr. Biden. In the end, the polls overestimated Mr. Biden by about eight percentage points. Eerily enough, Mr. Barnes is faring better than expected by a similar margin.
The Wisconsin data is just one example of a broader pattern across the battlegrounds: The more the polls overestimated Mr. Biden last time, the better Democrats seem to be doing relative to expectations. And conversely, Democrats are posting less impressive numbers in some of the states where the polls were fairly accurate two years ago, like Georgia.
If you put this relationship on a chart — Beware, my editors have permitted me to include somewhat challenging scatterplots on occasion — you see a consistent link between Democratic strength today and polling error two years ago.
It raises the possibility that the apparent Democratic strength in Wisconsin and elsewhere is a mirage — an artifact of persistent and unaddressed biases in survey research.
If the polls are wrong yet again, it will not be hard to explain. Most pollsters haven’t made significant methodological changes since the last election. The major polling community post-mortem declared that it was “impossible” to definitively ascertain what went wrong in the 2020 election.
The pattern of Democratic strength isn’t the only sign that the polls might still be off in similar ways. Since the Supreme Court’s Dobbs decision on abortion, some pollsters have said they’re seeing the familiar signs of nonresponse bias — when people who don’t respond to a poll are meaningfully different from those who participate — creeping back into their surveys.
Brian Stryker, a partner at Impact Research (Mr. Biden is a client), told me that his polling firm was getting “a ton of Democratic responses” in recent surveys, especially in “the familiar places” where the polls have erred in recent cycles.
None of this means the polls are destined to be as wrong as they were in 2020. Some of the polling challenges in 2020 might have since subsided, such as the greater likelihood that liberals were at home (and thus more likely to take polls) during the pandemic. And historically, it has been hard to anticipate polling error simply by looking at the error from the previous cycle. For example, the polls in 2018 weren’t so bad.
Some pollsters are making efforts to deal with the challenge. Mr. Stryker said his firm was “restricting the number of Democratic primary voters, early voters and other super-engaged Democrats” in their surveys. The New York Times/Siena College polls take similar steps.
But the pattern is worth taking seriously after what happened two years ago.
With that in mind, here’s an update of the table from 2020, with the polling averages in nine states the Cook Political Report describes as competitive. If the polls are just as wrong as they were in 2020, the race for the Senate looks very different.
The apparent Democratic edge in Senate races in Wisconsin, North Carolina and Ohio would evaporate. To take the chamber, Republicans would need any two of Georgia, Arizona, Nevada or Pennsylvania. With Democrats today well ahead in Pennsylvania and Arizona, the fight for control of the chamber would come down to very close races in Nevada and Georgia.
Regardless of who was favored, the race for Senate control would be extremely competitive. Republican control of the House would seem to be a foregone conclusion.
Nate Cohn is The Times’s chief political analyst. He covers elections, polling and demographics for The Upshot. Before joining The Times in 2013, he was a staff writer for The New Republic. @Nate_Cohn
|
Responses:
|
