November 6 vs November 7: An Enlightening Tale of Two Polls

A big poll was conducted among U.S. voters on November 7 in Ohio, Virginia, Kentucky and elsewhere. Millions of people participated. The result: Democratic candidates and issues did much better than expected.

That should have come as no surprise to those following other off-year opinion surveys in 2023, that is when voters actually went to the polls to express their opinions rather than responding to questions in telephone or Internet polls.

An analysis from the respected analytical group FiveThirtyEight found that in 38 special elections held through August, Democrats outperformed the partisan lean—or the relative liberal or conservative history—of the areas where the races were held by an average of 10%, doing better in parts of the country that typically support the party while cutting down on GOP margins in red cities and counties, too.

Ironically, a spate of media polls published in the days before the November 7 elections seemed to spell doom for the Democrats. Most notably, on November 6, the New York Times, ran, as its major front page story: “Voters in 5 Battlegrounds Favor Trump Over Biden.” With its prominent placement, NYT editors were judging it the most important story in the world that day.

Being the Times, the story had enormous political and journalistic weight. Wire services carried the story around the world. Cable TV burned hot with red and blue numbers, akin to election night results. Political pundits of all persuasions found outlets for their “analysis” of the poll’s meaning. Biden supporters went into deep woe-is-us mode.

By treating its poll as the most important story in the world that day and not placing it in its proper context, the Times did far more to shape opinion than reflect it. Their poll is, what all polls are, a snapshot of the time it is taken, deserving, at best, inside the paper placement as one of many factors related to next year’s elections. And what a lot of factors there are still to come: two wars and their outcomes, Trump’s trials, an unpredictable economy—just to name a few highlights.

As a reminder of how useless year-old polls are in forecasting election outcomes, just look back at these jewels:

September, 2015, CNN: Hillary Clinton leads Donald Trump in its poll by 10 points!!

July, 2015: NBC/WSJ: Jeb Bush leads the Republican primary with 22%!!

December, 2007: Gallup: Clinton 18 points ahead of Obama!!

---

---

The Cost of Pre-Election Polling

The media has an insatiable appetite for polling. Already in 2023 media polling has cost a number of Republican presidential candidates the opportunity for voter exposure because they hadn’t yet risen high enough in polls to qualify for GOP TV debates. Less noticed, media polls also have influenced which Republicans raised money and recruited influential campaign supporters. In a real sense, early media polls become players in the campaign, not observers of it.

What’s worse, the polls themselves are weak reeds to lean on these days. In an earlier poll for the New York Times, its survey guru Nate Cohn admitted that it took two hours of dialing just to complete one interview. Polls no longer are as random a sample of the voter universe as they once were. For many reasons—lack of time, fear of spamming, political call overload—fewer people are willing to answer calls or emails from strangers. The cohort of responders keeps shrinking.

The NYT used Siena College, a respected polling operation, to conduct its poll, interviewing an impressive 3,602 people. That number of samples has a tiny margin of error, less than 2%. But the surveys covered six states, presumably 600 samples per state, with a margin of error in each state of about 4.5%. A big difference in error size and one noted, but not explained in the Times article.

Not so incidentally, in describing its methodology the Times says that the 3,602 respondents were registered voters. In 2020’s general election, about 66% of those eligible to vote actually did. With lagging enthusiasm for a Biden-Trump rematch, what percentage of the 3,602 people in Siena’s poll will vote? Who among the 3,602 were the likely voters? How would polling only likely voters have changed the results? Many otherwise good polling firms miss the result because they guess wrong on who would vote and the size of the turnout. A year distant, it’s next to impossible.

Equally difficult this early is creating a sample that’s a microcosm of the actual voting universe. To design its poll, Siena had to make educated guesses on how many samples to get from young people, and those of all ages, and other demographics, such as race, education and income levels.

Take the youth vote, for example. In 2020, about 31% of those aged 18-29 voted. That was up from 23% in 2016 and 14% in 2008. What will the youth turnout be in 2024? Higher because of fear of Trump? Lower because of lack of enthusiasm for Biden? Next year, through primaries and other factors, polling firms will get data on current voter turnout and enthusiasm, helpful in creating a data-based voting universe for their surveys. Now it’s just looking-in-the-rearview-mirror guesswork.

Why Early Poll Results Are Suspect

There are many more technical reasons why early poll results are suspect. But here’s one that’s not so technical: lies.

In 1989, I produced media for Doug Wilder, the first Black elected governor of Virginia. The week before the election, Wilder was comfortably ahead in most polls by 5 to 10 points. He won 50.2 to 49.8. It doesn’t take many respondent lies about who they are, what they believe and who they support to skew poll results. In our current political environment, my guess is that responses are less trustworthy than ever.

None of this means I’m anti-poll. I’ve been involved with literally hundreds of them, relying on results to suggest how campaign budgets are spent, the issues to highlight, candidate travel, campaign organization and more. That’s why campaigns take polls. The results provide candidates and their teams insights on how best to run their campaigns.

Putting Polls in Perspective

There’s nothing inherently wrong with polls if you design them correctly and don’t ask them to do more than what’s possible. The problem with media polls this early is that they get treated as the checkered flag before the race is really engaged, disadvantaging newcomers, influencing how the campaigns are reported, creating unwarranted optimism or despair, and making the media an important player rather than an observer.

My suggestion to the media: Go back and reread your polls from past elections. Once you see how wrong you were, make it right by being a lot more cautious about what and when you poll and how much news weight you give to the results.

Comments? Criticism? Contact Joe Rothstein at jrothstein@rothstein.net